new estimation and model selection procedures for semiparametric modeling in longitudinal data analysis semiparametric regression models are very useful for longitudinal data analysis the complexity of semiparametric models and the structure of longitudinal data pose new challenges to parametric inferences and model selection that frequently arise from longitudinal data analysis in this article two new approaches are proposed for estimating the regression coefficients in a semiparametric model the asymptotic normality of the resulting estimators is established an innovative class of variable selection procedures is proposed to select significant variables in the semiparametric models the proposed procedures are distinguished from others in that they simultaneously select significant variables and estimate unknown parameters rates of convergence of the resulting estimators are established with a proper choice of regularization parameters and penalty functions the proposed variable selection procedures are shown to perform as well as an oracle estimator a robust standard error formula is derived usi

efficient empirical bayes variable selection and estimation in linear models we propose an empirical bayes method for variable selection and coefficient estimation in linear regression models the method is based on a particular hierarchical bayes formulation and the empirical bayes estimator is shown to be closely related to the lasso estimator such a connection allows us to take advantage of the recently developed quick lasso algorithm to compute the empirical bayes estimate and provides a new way to select the tuning parameter in the lasso method unlike previous empirical bayes variable selection methods which in most practical situations can be implemented only through a greedy stepwise algorithm our method gives a global solution efficiently simulations and real examples show that the proposed method is very competitive in terms of variable selection estimation accuracy and computation speed compared with other variable selection and estimation methods

admm for high dimensional sparse penalized quantile regression abstractsparse penalized quantile regression is a useful tool for variable selection robust estimation and heteroscedasticity detection in high dimensional data analysis the computational issue of the sparse penalized quantile regression has not yet been fully resolved in the literature due to non smoothness of the quantile regression loss function we introduce fast alternating direction method of multipliers admm algorithms for computing the sparse penalized quantile regression the convergence properties of the proposed algorithms are established numerical examples demonstrate the competitive performance of our algorithm it significantly outperforms several other fast solvers for high dimensional penalized quantile regression supplementary materials for this article are available online

one step local quasi likelihood estimation summary local quasi likelihood estimation is a useful extension of local least squares methods but its computational cost and algorithmic convergence problems make the procedure less appealing particularly when it is iteratively used in methods such as the back fitting algorithm cross validation and bootstrapping a one step local quasi likelihood estimator is introduced to overcome the computational drawbacks of the local quasi likelihood method we demonstrate that as long as the initial estimators are reasonably good the one step estimator has the same asymptotic behaviour as the local quasi likelihood method our simulation shows that the one step estimator performs at least as well as the local quasi likelihood method for a wide range of choices of bandwidths a data driven bandwidth selector is proposed for the one step estimator based on the pre asymptotic substitution method of fan and gijbels it is then demonstrated that the datadriven one step local quasi likelihood estimator performs as well as the maximum local quasilikelihood estimator by using the ideal optimal bandwidth

heuristics of instability and stabilization in model selection in model selection usually a best predictor is chosen from a collection hat mu cdot s of predictors where hat mu cdot s is the minimum least squares predictor in a collection mathsf u s of predictors here s is a complexity parameter that is the smaller s the lower dimensional smoother the models in mathsf u s if mathsf l is the data used to derive the sequence hat mu cdot s the procedure is called unstable if a small change in mathsf l can cause large changes in hat mu cdot s with a crystal ball one could pick the predictor in hat mu cdot s having minimum prediction error without prescience one uses test sets cross validation and so forth the difference in prediction error between the crystal ball selection and the statistician s choice we call predictive loss for an unstable procedure the predictive loss is large this is shown by some analytics in a simple case and by simulation results in a more complex comparison of four different linear regression methods unstable procedures can be stabilized by perturbing the data getting a new predictor sequence hat mu cdot s and then averaging over many such predictor sequences

variable selection for multivariate failure time data in this paper we proposed a penalised pseudo partial likelihood method for variable selection with multivariate failure time data with a growing number of regression coefficients under certain regularity conditions we show the consistency and asymptotic normality of the penalised likelihood estimators we further demonstrate that for certain penalty functions with proper choices of regularisation parameters the resulting estimator can correctly identify the true model as if it were known in advance based on a simple approximation of the penalty function the proposed method can be easily carried out with the newton raphson algorithm we conduct extensive monte carlo simulation studies to assess the finite sample performance of the proposed procedures we illustrate the proposed method by analysing a dataset from the framingham heart study

local shrinkage rules levy processes and regularized regression we use l evy processes to generate joint prior distributions and therefore penalty functions for a location parameter b b1 bp as p grows large this generalizes the class of local global shrinkage rules based on scale mixtures of normals illuminates new connections among disparate methods and leads to new results for computing posterior means and modes under a wide class of priors we extend this framework to large scale regularized regression problems where p n and provide comparisons with other methodologies

nonparametric independence screening in sparse ultra high dimensional varying coefficient models the varying coefficient model is an important class of nonparametric statistical model which allows us to examine how the effects of covariates vary with exposure variables when the number of covariates is large the issue of variable selection arises in this article we propose and investigate marginal nonparametric screening methods to screen variables in sparse ultra high dimensional varying coefficient models the proposed nonparametric independence screening nis selects variables by ranking a measure of the nonparametric marginal contributions of each covariate given the exposure variable the sure independent screening property is established under some mild technical conditions when the dimensionality is of nonpolynomial order and the dimensionality reduction of nis is quantified to enhance the practical utility and finite sample performance two data driven iterative nis inis methods are proposed for selecting thresholding parameters and variables conditional permutation and greedy metho

decoupling shrinkage and selection in bayesian linear models a posterior summary perspective selecting a subset of variables for linear models remains an active area of research this article reviews many of the recent contributions to the bayesian model selection and shrinkage prior literature a posterior variable selection summary is proposed which distills a full posterior distribution over regression coefficients into a sequence of sparse linear predictors

shrinkage tuning parameter selection with a diverging number of parameters contemporary statistical research frequently deals with problems involving a diverging number of parameters for those problems various shrinkage methods e g the lasso and smoothly clipped absolute deviation are found to be particularly useful for variable selection nevertheless the desirable performances of those shrinkage methods heavily hinge on an appropriate selection of the tuning parameters with a fixed predictor dimension wang and co worker have demonstrated that the tuning parameters selected by a bayesian information criterion type criterion can identify the true model consistently in this work similar results are further extended to the situation with a diverging number of parameters for both unpenalized and penalized estimators consequently our theoretical results further enlarge not only the scope of applicability of the traditional bayesian information criterion type criteria but also that of those shrinkage estimation methods

penalized least squares regression methods and applications to neuroimaging the goals of this paper are to review the most popular methods of predictor selection in regression models to explain why some fail when the number p of explanatory variables exceeds the number n of participants and to discuss alternative statistical methods that can be employed in this case we focus on penalized least squares methods in regression models and discuss in detail two such methods that are well established in the statistical literature the lasso and elastic net we introduce bootstrap enhancements of these methods the be lasso and be enet that allow the user to attach a measure of uncertainty to each variable selected our work is motivated by a multimodal neuroimaging dataset that consists of morphometric measures volumes at several anatomical regions of interest white matter integrity measures from diffusion weighted data fractional anisotropy mean diffusivity axial diffusivity and radial diffusivity and clinical and demographic variables age education alcohol and drug history in this dataset the number p of explanatory variables exceeds the number n of participants we use the be lasso and be enet to provide the first statistical analysis that allows the assessment of neurocognitive performance from high dimensional neuroimaging and clinical predictors including their interactions the major novelty of this analysis is that biomarker selection and dimension reduction are accomplished with a view towards obtaining good predictions for the outcome of interest i e the neurocognitive indices unlike principal component analysis that are performed only on the predictors space independently of the outcome of interest

gemini graph estimation with matrix variate normal instances undirected graphs can be used to describe matrix variate distributions in this paper we develop new methods for estimating the graphical structures and underlying parameters namely the row and column covariance and inverse covariance matrices from the matrix variate data under sparsity conditions we show that one is able to recover the graphs and covariance matrices with a single random matrix from the matrix variate normal distribution our method extends with suitable adaptation to the general setting where replicates are available we establish consistency and obtain the rates of convergence in the operator and the frobenius norm we show that having replicates will allow one to estimate more complicated graphical structures and achieve faster rates of convergence we provide simulation evidence showing that we can recover graphical structures as well as estimating the precision matrices as predicted by theory

multivariate statistical process control using lasso this article develops a new multivariate statistical process control spc methodology based on adapting the lasso variable selection method to the spc problem the lasso method has the sparsity property of being able to select exactly the set of nonzero regression coefficients in multivariate regression modeling which is especially useful in cases where the number of nonzero coefficients is small in multivariate spc applications process mean vectors often shift in a small number of components our primary goals are to detect such a shift as soon as it occurs and to identify the shifted mean components using this connection between the two problems we propose a lasso based multivariate test statistic and then integrate this statistic into the multivariate ewma charting scheme for phase ii multivariate process monitoring we show that this approach balances protection against various shift levels and shift directions and thus provides an effective tool for multivariate spc applications this article

another look at distance weighted discrimination summary distance weighted discrimination dwd is a modern margin based classifier with an interesting geometric motivation it was proposed as a competitor to the support vector machine svm despite many recent references on dwd dwd is far less popular than the svm mainly because of computational and theoretical reasons we greatly advance the current dwd methodology and its learning theory we propose a novel thrifty algorithm for solving standard dwd and generalized dwd and our algorithm can be several hundred times faster than the existing state of the art algorithm based on second order cone programming in addition we exploit the new algorithm to design an efficient scheme to tune generalized dwd furthermore we formulate a natural kernel dwd approach in a reproducing kernel hilbert space and then establish the bayes risk consistency of the kernel dwd by using a universal kernel such as the gaussian kernel this result solves an open theoretical problem in the dwd literature a comparison study on 16 benchmark data sets shows that data driven generalized dwd consistently delivers higher classification accuracy with less computation time than the svm

a distributed one step estimator distributed statistical inference has recently attracted enormous attention many existing work focuses on the averaging estimator e g zhang and duchi j mach learn res 14 3321 3363 2013 together with many others we propose a one step approach to enhance a simple averaging based distributed estimator by utilizing a single newton raphson updating we derive the corresponding asymptotic properties of the newly proposed estimator we find that the proposed one step estimator enjoys the same asymptotic properties as the idealized centralized estimator in particular the asymptotic normality was established for the proposed estimator while other competitors may not enjoy the same property the proposed one step approach merely requires one additional round of communication in relative to the averaging estimator so the extra communication burden is insignificant the proposed one step approach leads to a lower upper bound of the mean squared error than other alternatives in finite sample cases numerical examples show that the proposed estimator outperforms the simple averaging estimator with a large margin in terms of the sample mean squared error a potential application of the one step approach is that one can use multiple machines to speed up large scale statistical inference with little compromise in the quality of estimators the proposed method becomes more valuable when data can only be available at distributed machines with limited communication bandwidth

a multiplicity result for second order impulsive differential equations via the leggett williams fixed point theorem using the leggett williams fixed point theorem we present in this paper criteria which guarantee the existence of three nonnegative solutions to second order impulsive equations

reduced rank regression via adaptive nuclear norm penalization we propose an adaptive nuclear norm penalization approach for low rank matrix approximation and use it to develop a new reduced rank estimation method for high dimensional multivariate regression the adaptive nuclear norm is defined as the weighted sum of the singular values of the matrix and it is generally non convex under the natural restriction that the weight decreases with the singular value however we show that the proposed non convex penalized regression method has a global optimal solution obtained from an adaptively soft thresholded singular value decomposition the method is computationally efficient and the resulting solution path is continuous the rank consistency of and prediction estimation performance bounds for the estimator are established for a high dimensional asymptotic regime simulation studies and an application in genetics demonstrate its efficacy

outlier models and prior distributions in bayesian linear regression summary bayesian inference in regression models is considered using heavy tailed error distri butions to accommodate outliers the particular class of distributions that can be con structed as scale mixtures of normal distributions are examined and use is made of them as both error models and prior distributions in bayesian linear modelling includ ing simple regression and more complex hierarchical models with structured priors depending on unknown hyperprior parameters the modelling of outliers in nominally normal linear regression models using alternative error distributions which are heavy tailed relative to the normal provides an automatic means of both detecting and accommodating possibly aberrant observations such realistic models do however often lead to analytically intractable analyses with complex posterior distributions in several dimensions that are difficult to summarize and understand in this paper we consider a special yet rather wide class of heavy tailed unimodal and symmetric error distributions for which the analyses though apparently intractable can be examined in some depth by exploiting certain properties of the assumed error form the distributions concerned are those that can be con structed as scale mixtures of normal distributions in his paper concerning location parameters de finetti 1961 discusses such distributions and suggests the hypothetical interpretation that each observation is taken using an instrument with normal error but each time chosen at random from a collection of instruments of different precisions the distribution of the

high dimensional variable selection via tilting the paper considers variable selection in linear regression models where the number of covariates is possibly much larger than the number of observations high dimensionality of the data brings in many complications such as possibly spurious high correlations between the variables which result in marginal correlation being unreliable as a measure of association between the variables and the response we propose a new way of measuring the contribution of each variable to the response which takes into account high correlations between the variables in a data driven way the proposed tilting procedure provides an adaptive choice between the use of marginal correlation and tilted correlation for each variable where the choice is made depending on the values of the hard thresholded sample correlation of the design matrix we study the conditions under which this measure can successfully discriminate between the relevant and the irrelevant variables and thus be used as a tool for variable selection finally an iterative variable screening algorithm is constructed to exploit the theoretical properties of tilted correlation and its good practical performance is demonstrated in a comparative simulation study

spatial shrinkage estimation of diffusion tensors on diffusion weighted imaging data diffusion tensor imaging dti based on the diffusion weighted imaging dwi data acquired from magnetic resonance experiments has been widely used to analyze the physical structure of white matter fibers in the human brain in vivo the raw dwi data however carry noise this contaminates the diffusion tensor dt estimates and introduces systematic bias into the induced eigenvalues these bias components affect the effectiveness of fiber tracking algorithms in this article we propose a two stage spatial shrinkage estimation spske procedure to accommodate the spatial information carried in dwi data in dt estimation and to reduce the bias components in the corresponding derived eigenvalues to this end in the framework of the heteroscedastic linear model spske incorporates l 1 type penalization and the locally weighted least square function the theoretical properties of spske are explored the effectiveness of spske is further illustrated by simulation and real data examples supplementary materi

nonconcave penalized likelihood with a diverging number of parameters a class of variable selection procedures for parametric models via nonconcave penalized likelihood was proposed by fan and li to simultaneously estimate parameters and select important variables they demonstrated that this class of procedures has an oracle property when the number of parameters is finite however in most model selection problems the number of parameters should be large and grow with the sample size in this paper some asymptotic properties of the nonconcave penalized likelihood are established for situations in which the number of parameters tends to as the sample size increases under regularity conditions we have established an oracle property and the asymptotic normality of the penalized likelihood estimators furthermore the consistency of the sandwich formula of the covariance matrix is demonstrated nonconcave penalized likelihood ratio statistics are discussed and their asymptotic distributions under the null hypothesis are obtained by imposing some mild conditions on the penalty functions the asymptotic results are augmented by a simulation study and the newly developed methodology is illustrated by an analysis of a court case on the sexual discrimination of salary

variable selection for cox s proportional hazards model and frailty model a class of variable selection procedures for parametric models via nonconcave penalized likelihood was proposed in fan and li 2001a it has been shown there that the resulting procedures perform as well as if the subset of significant variables were known in advance such a property is called an oracle property the proposed procedures were illustrated in the context of linear regression robust linear regression and generalized linear models in this paper the nonconcave penalized likelihood approach is extended further to the cox proportional hazards model and the cox proportional hazards frailty model two commonly used semi parametric models in survival analysis as a result new variable selection procedures for these two commonly used models are proposed it is demonstrated how the rates of convergence depend on the regularization parameter in the penalty function further with a proper choice of the regularization parameter and the penalty function the proposed estimators possess an oracle property standard error formulae are derived and their accuracies are empirically tested simulation studies show that the proposed procedures are more stable in prediction and more effective in computation than the best subset variable selection and they reduce model complexity as effectively as the best subset variable selection compared with the lasso which is the penalized likelihood method with the l 1 penalty proposed by tibshirani the newly proposed approaches have better theoretic properties and finite sample performance

high dimensional sparse additive hazards regression high dimensional sparse modeling with censored survival data is of great practical importance as exemplified by modern applications in high throughput genomic data analysis and credit risk analysis in this article we propose a class of regularization methods for simultaneous variable selection and estimation in the additive hazards model by combining the nonconcave penalized likelihood approach and the pseudoscore method in a high dimensional setting where the dimensionality can grow fast polynomially or nonpolynomially with the sample size we establish the weak oracle property and oracle property under mild interpretable conditions thus providing strong performance guarantees for the proposed methodology moreover we show that the regularity conditions required by the l 1 method are substantially relaxed by a certain class of sparsity inducing concave penalties as a result concave penalties such as the smoothly clipped absolute deviation minimax concave penalty and smooth integration of co

sparse signal estimation by maximally sparse convex optimization this paper addresses the problem of sparsity penalized least squares for applications in sparse signal processing e g sparse deconvolution this paper aims to induce sparsity more strongly than l1 norm regularization while avoiding non convex optimization for this purpose this paper describes the design and use of non convex penalty functions regularizers constrained so as to ensure the convexity of the total cost function f to be minimized the method is based on parametric penalty functions the parameters of which are constrained to ensure convexity of f it is shown that optimal parameters can be obtained by semidefinite programming sdp this maximally sparse convex msc approach yields maximally non convex sparsity inducing penalty functions constrained such that the total cost function f is convex it is demonstrated that iterative msc imsc can yield solutions substantially more sparse than the standard convex sparsity inducing approach i e l1 norm minimization

tuning parameter selection in high dimensional penalized likelihood summary determining how to select the tuning parameter appropriately is essential in penalized likelihood methods for high dimensional data analysis we examine this problem in the setting of penalized likelihood methods for generalized linear models where the dimensionality of covariates p is allowed to increase exponentially with the sample size n we propose to select the tuning parameter by optimizing the generalized information criterion with an appropriate model complexity penalty to ensure that we consistently identify the true model a range for the model complexity penalty is identified in the generlized information criterion we find that this model complexity penalty should diverge at the rate of some power of log p depending on the tail probability behaviour of the response variables this reveals that using the akaike information criterion or bayes information criterion to select the tuning parameter may not be adequate for consistently identifying the true model on the basis of our theoretical study we propose a uniform choice of the model complexity penalty and show that the approach proposed consistently identifies the true model among candidate models with asymptotic probability 1 we justify the performance of the procedure proposed by numerical simulations and a gene expression data analysis

a statistical view of some chemometrics regression tools chemometrics is a field of chemistry that studies the application of statistical methods to chemical data analysis in addition to borrowing many techniques from the statistics and engineering literatures chemometrics itself has given rise to several new data analytical methods this article examines two methods commonly used in chemometrics for predictive modeling partial least squares and principal components regression from a statistical perspective the goal is to try to understand their apparent successes and in what situations they can be expected to work well and to compare them with other statistical methods intended for those situations these methods include ordinary least squares variable subset selection and ridge regression

tuning parameter selection for the adaptive lasso using eric the adaptive lasso is a commonly applied penalty for variable selection in regression modeling like all penalties though its performance depends critically on the choice of the tuning parameter one method for choosing the tuning parameter is via information criteria such as those based on aic and bic however these criteria were developed for use with unpenalized maximum likelihood estimators and it is not clear that they take into account the effects of penalization in this article we propose the extended regularized information criterion eric for choosing the tuning parameter in adaptive lasso regression eric extends the bic to account for the effect of applying the adaptive lasso on the bias variance tradeoff this leads to a criterion whose penalty for model complexity is itself a function of the tuning parameter we show the tuning parameter chosen by eric is selection consistent when the number of variables grows with sample size and that this consistency holds in a wider range of contex

factor profiled sure independence screening we propose a method of factor profiled sure independence screening for ultrahigh dimensional variable selection the objective of this method is to identify nonzero components consistently from a sparse coefficient vector the new method assumes that the correlation structure of the high dimensional data can be well represented by a set of low dimensional latent factors which can be estimated consistently by eigenvalue eigenvector decomposition the estimated latent factors should then be profiled out from both the response and the predictors such an operation referred to as factor profiling produces uncorrelated predictors therefore sure independence screening can be applied subsequently and the resulting screening result is consistent for model selection a major advantage that standard sure independence screening does not share we refer to the new method as factor profiled sure independence screening numerical studies confirm its outstanding performance copyright 2012 oxford university press

sparsenet coordinate descent with nonconvex penalties we address the problem of sparse selection in linear models a number of nonconvex penalties have been proposed in the literature for this purpose along with a variety of convex relaxation algorithms for finding good solutions in this article we pursue a coordinate descent approach for optimization and study its convergence properties we characterize the properties of penalties suitable for this approach study their corresponding threshold functions and describe a df standardizing reparametrization that assists our pathwise algorithm the mc penalty is ideally suited to this task and we use it to demonstrate the performance of our algorithm certain technical derivations and experiments related to this article are included in the supplementary materials section

the group latent variable approach to probit binary classifications this paper considers the binary classification with the probit model under the expectation maximization em algorithm usually in the bayesian approach of the probit model the latent variables are introduced to handle with the intractable problem for each training sample there is a corresponding latent variable however the em algorithm requires matrix inversions which demand for the expansive computational cost when the number of training samples is large to overcome this problem we employ the group latent variable approach where for each of the training samples there are corresponding multiple latent variables instead of just one the major advantage of this approach which is originated from bayesian backfitting is that there are no requirements for matrix inversions in the em algorithm for the probit model in this paper to obtain sparse classifiers the laplacian prior is employed and the method to control the degree of sparseness is presented although the sparsity of the classifier is not determined in a full automatic way it can be controlled by specifying just one parameter in other words we are free to choose the degree of sparseness the proposed method is compared with support vector machine relevance vector machine and generalized lasso

identification of sparse fir systems using a general quantisation scheme this paper presents an identification scheme for sparse fir systems with quantised data we consider a general quantisation scheme which includes the commonly deployed static quantiser as a special case to tackle the sparsity issue we utilise a bayesian approach where an l1 a priori distribution for the parameters is used as a mechanism to promote sparsity the general framework used to solve the problem is maximum likelihood ml the ml problem is solved by using a generalised expectation maximisation algorithm

joint estimation of multiple graphical models gaussian graphical models explore dependence relationships between random variables through the estimation of the corresponding inverse covariance matrices in this paper we develop an estimator for such models appropriate for data from several graphical models that share the same variables and some of the dependence structure in this setting estimating a single graphical model would mask the underlying heterogeneity while estimating separate models for each category does not take advantage of the common structure we propose a method that jointly estimates the graphical models corresponding to the different categories present in the data aiming to preserve the common structure while allowing for differences between the categories this is achieved through a hierarchical penalty that targets the removal of common zeros in the inverse covariance matrices across categories we establish the asymptotic consistency and sparsity of the proposed estimator in the high dimensional case and illustrate its performance on a number of simulated networks an application to learning semantic connections between terms from webpages collected from computer science departments is included

one step sparse estimates in nonconcave penalized likelihood models fan and li propose a family of variable selection methods via penalized likelihood using concave penalty functions the nonconcave penalized likelihood estimators enjoy the oracle properties but maximizing the penalized likelihood function is computationally challenging because the objective function is nondifferentiable and nonconcave in this article we propose a new unified algorithm based on the local linear approximation lla for maximizing the penalized likelihood for a broad class of concave penalty functions convergence and other theoretical properties of the lla algorithm are established a distinguished feature of the lla algorithm is that at each lla step the lla estimator can naturally adopt a sparse representation thus we suggest using the one step lla estimator from the lla algorithm as the final estimates statistically we show that if the regularization parameter is appropriately chosen the one step lla estimates enjoy the oracle properties with good initial estimators computationally the one step lla estimation methods dramatically reduce the computational cost in maximizing the nonconcave penalized likelihood we conduct some monte carlo simulation to assess the finite sample performance of the one step sparse estimation methods the results are very encouraging

sparse estimation of conditional graphical models with application to gene networks in many applications the graph structure in a network arises from two sources intrinsic connections and connections due to external effects we introduce a sparse estimation procedure for graphical models that is capable of isolating the intrinsic connections by removing the external effects technically this is formulated as a conditional graphical model in which the external effects are modeled as predictors and the graph is determined by the conditional precision matrix we introduce two sparse estimators of this matrix using the reproduced kernel hilbert space combined with lasso and adaptive lasso we establish the sparsity variable selection consistency oracle property and the asymptotic distributions of the proposed estimators we also develop their convergence rate when the dimension of the conditional precision matrix goes to infinity the methods are compared with sparse estimators for unconditional graphical models and with the constrained maximum likelihood estimate that assumes a know

orthogonalizing em a design based least squares algorithm we introduce an efficient iterative algorithm intended for various least squares problems based on a design of experiments perspective the algorithm called orthogonalizing em oem works for ordinary least squares ols and can be easily extended to penalized least squares the main idea of the procedure is to orthogonalize a design matrix by adding new rows and then solve the original problem by embedding the augmented design in a missing data framework we establish several attractive theoretical properties concerning oem for the ols with a singular regression matrix an oem sequence converges to the moore penrose generalized inverse based least squares estimator for ordinary and penalized least squares with various penalties it converges to a point having grouping coherence for fully aliased regression matrices convergence and the convergence rate of the algorithm are examined finally we demonstrate that oem is highly efficient for large scale least squares and penalized least squares proble

piecewise linear regularized solution paths we consider the generic regularized optimization problem argmin l y x j efron hastie johnstone and tibshirani ann statist 32 2004 407 499 have shown that for the lasso that is if l is squared error loss and j 1 is the l 1 norm of the optimal coefficient path is piecewise linear that is is piecewise constant we derive a general characterization of the properties of loss l penalty j pairs which give piecewise linear coefficient paths such pairs allow for efficient generation of the full regularized coefficient paths we investigate the nature of efficient path following algorithms which arise we use our results to suggest robust versions of the lasso for regression and classification and to develop new efficient algorithms for existing problems in the literature including mammen and van de geer s locally adaptive regression splines

variable selection using mm algorithms variable selection is fundamental to high dimensional statistical modeling many variable selection techniques may be implemented by maximum penalized likelihood using various penalty functions optimizing the penalized likelihood function is often challenging because it may be nondifferentiable and or nonconcave this article proposes a new class of algorithms for finding a maximizer of the penalized likelihood for a broad class of penalty functions these algorithms operate by perturbing the penalty function slightly to render it differentiable then optimizing this differentiable function using a minorize maximize mm algorithm mm algorithms are useful extensions of the well known class of em algorithms a fact that allows us to analyze the local and global convergence of the proposed algorithm using some of the techniques employed for em algorithms in particular we prove that when our mm algorithms converge they must converge to a desirable point we also discuss conditions under which this convergence may be guaranteed we exploit the newton raphson like aspect of these algorithms to propose a sandwich estimator for the standard errors of the estimators our method performs well in numerical tests

forward regression for ultra high dimensional variable screening motivated by the seminal theory of sure independence screening fan and lv 2008 sis we investigate here another popular and classical variable screening method namely forward regression fr our theoretical analysis reveals that fr can identify all relevant predictors consistently even if the predictor dimension is substantially larger than the sample size in particular if the dimension of the true model is finite fr can discover all relevant predictors within a finite number of steps to practically select the best candidate from the models generated by fr the recently proposed bic criterion of chen and chen 2008 can be used the resulting model can then serve as an excellent starting point from where many existing variable selection methods e g scad and adaptive lasso can be applied directly fr s outstanding finite sample performances are confirmed by extensive numerical studies

shrinkage estimation of the varying coefficient model the varying coefficient model is a useful extension of the linear regression model nevertheless how to conduct variable selection for the varying coefficient model in a computationally efficient manner is poorly understood to solve the problem we propose here a novel method which combines the ideas of the local polynomial smoothing and the least absolute shrinkage and selection operator lasso the new method can do nonparametric estimation and variable selection simultaneously with a local constant estimator and the adaptive lasso penalty the new method can identify the true model consistently and that the resulting estimator can be as efficient as the oracle estimator numerical studies clearly confirm our theories extension to other shrinkage methods e g the scad i e the smoothly clipped absolute deviation and other smoothing methods is straightforward

hazard models with varying coefficients for multivariate failure time data statistical estimation and inference for marginal hazard models with varying coefficients for multivariate failure time data are important subjects in survival analysis a local pseudo partial likelihood procedure is proposed for estimating the unknown coefficient functions a weighted average estimator is also proposed in an attempt to improve the efficiency of the estimator the consistency and asymptotic normality of the proposed estimators are established and standard error formulas for the estimated coefficients are derived and empirically tested to reduce the computational burden of the maximum local pseudo partial likelihood estimator a simple and useful one step estimator is proposed statistical properties of the one step estimator are established and simulation studies are conducted to compare the performance of the one step estimator to that of the maximum local pseudo partial likelihood estimator the results show that the one step estimator can save computational cost without compromising performance both asymptotically and empirically and that an optimal weighted average estimator is more efficient than the maximum local pseudo partial likelihood estimator a data set from the busselton population health surveys is analyzed to illustrate our proposed methodology

i lamm for sparse learning simultaneous control of algorithmic complexity and statistical error we propose a computational framework named iterative local adaptive majorize minimization i lamm to simultaneously control algorithmic complexity and statistical error when fitting high dimensional models i lamm is a two stage algorithmic implementation of the local linear approximation to a family of folded concave penalized quasi likelihood the first stage solves a convex program with a crude precision tolerance to obtain a coarse initial estimator which is further refined in the second stage by iteratively solving a sequence of convex programs with smaller precision tolerances theoretically we establish a phase transition the first stage has a sublinear iteration complexity while the second stage achieves an improved linear rate of convergence though this framework is completely algorithmic it provides solutions with optimal statistical performances and controlled algorithmic complexity for a large family of nonconvex optimization problems the iteration effects on statistical errors are clearly demonstrated via a contraction property our theory relies on a localized version of the sparse restricted eigenvalue condition which allows us to analyze a large family of loss and penalty functions and provide optimality guarantees under very weak assumptions for example i lamm requires much weaker minimal signal strength than other procedures thorough numerical results are provided to support the obtained theory

generalized high dimensional trace regression via nuclear norm regularization abstract we study the generalized trace regression with a near low rank regression coefficient matrix which extends notion of sparsity for regression coefficient vectors specifically given a matrix covariate x the probability density function of the response y is f y x c y exp 1 y b where tr t x this model accommodates various types of responses and embraces many important problem setups such as reduced rank regression matrix regression that accommodates a panel of regressors matrix completion among others we estimate through minimizing empirical negative log likelihood plus nuclear norm penalty we first establish a general theory and then for each specific problem we derive explicitly the statistical rate of the proposed estimator they all match the minimax rates in the linear trace regression up to logarithmic factors numerical studies confirm the rates we established and demonstrate the advantage of generalized trace regression over linear trace regression when the response is dichotomous we also show the benefit of incorporating nuclear norm regularization in dynamic stock return prediction and in image classification

expandable factor analysis bayesian sparse factor models have proven useful for characterizing dependence in multivariate data but scaling computation to large numbers of samples and dimensions is problematic we propose expandable factor analysis for scalable inference in factor models when the number of factors is unknown the method relies on a continuous shrinkage prior for efficient maximum a posteriori estimation of a low rank and sparse loadings matrix the structure of the prior leads to an estimation algorithm that accommodates uncertainty in the number of factors we propose an information criterion to select the hyperparameters of the prior expandable factor analysis has better false discovery rates and true positive rates than its competitors across diverse simulation settings we apply the proposed approach to a gene expression study of ageing in mice demonstrating superior results relative to four competing methods

alternating linearization for structured regularization problems we adapt the alternating linearization method for proximal decomposition to structured regularization problems in particular to the generalized lasso problems the method is related to two well known operator splitting methods the douglas rachford and the peaceman rachford method but it has descent properties with respect to the objective function this is achieved by employing a special update test which decides whether it is beneficial to make a peaceman rachford step any of the two possible douglas rachford steps or none the convergence mechanism of the method is related to that of bundle methods of nonsmooth optimization we also discuss implementation for very large problems with the use of specialized algorithms and sparse data structures finally we present numerical results for several synthetic and real world examples including a three dimensional fused lasso problem which illustrate the scalability efficacy and accuracy of the method

informative estimation and selection of correlation structure for longitudinal data identifying an informative correlation structure is important in improving estimation efficiency for longitudinal data we approximate the empirical estimator of the correlation matrix by groups of known basis matrices that represent different correlation structures and transform the correlation structure selection problem to a covariate selection problem to address both the complexity and the informativeness of the correlation matrix we minimize an objective function that consists of two parts the difference between the empirical information and a model approximation of the correlation matrix and a penalty that penalizes models with too many basis matrices the unique feature of the proposed estimation and selection of correlation structure is that it does not require the specification of the likelihood function and therefore it is applicable for discrete longitudinal data we carry out the proposed method through a groupwise penalty strategy which is able to identify more complex structures the

guarding against spurious discoveries in high dimensions many data mining and statistical machine learning algorithms have been developed to select a subset of covariates to associate with a response variable spurious discoveries can easily arise in high dimensional data analysis due to enormous possibilities of such selections how can we know statistically our discoveries better than those by chance in this paper we define a measure of goodness of spurious fit which shows how good a response variable can be fitted by an optimally selected subset of covariates under the null model and propose a simple and effective lamm algorithm to compute it it coincides with the maximum spurious correlation for linear models and can be regarded as a generalized maximum spurious correlation we derive the asymptotic distribution of such goodness of spurious fit for generalized linear models and l1 regression such an asymptotic distribution depends on the sample size ambient dimension the number of variables used in the fit and the covariance information it can be consistently estimated by multiplier bootstrapping and used as a benchmark to guard against spurious discoveries it can also be applied to model selection which considers only candidate models with goodness of fits better than those by spurious fits the theory and method are convincingly illustrated by simulated examples and an application to the binary outcomes from german neuroblastoma trials

feature screening via distance correlation learning this article is concerned with screening features in ultrahigh dimensional data analysis which has become increasingly important in diverse scientific fields we develop a sure independence screening procedure based on the distance correlation dc sis the dc sis can be implemented as easily as the sure independence screening sis procedure based on the pearson correlation proposed by fan and lv however the dc sis can significantly improve the sis fan and lv established the sure screening property for the sis based on linear models but the sure screening property is valid for the dc sis under more general settings including linear models furthermore the implementation of the dc sis does not require model specification e g linear model or generalized linear model for responses or predictors this is a very appealing property in ultrahigh dimensional data analysis moreover the dc sis can be used directly to screen grouped predictor variables and multivariate response variables we establish

new efficient estimation and variable selection methods for semiparametric varying coefficient partially linear models the complexity of semiparametric models poses new challenges to statistical inference and model selection that frequently arise from real applications in this work we propose new estimation and variable selection procedures for the semiparametric varying coefficient partially linear model we first study quantile regression estimates for the nonparametric varying coefficient functions and the parametric regression coefficients to achieve nice efficiency properties we further develop a semiparametric composite quantile regression procedure we establish the asymptotic normality of proposed estimators for both the parametric and nonparametric parts and show that the estimators achieve the best convergence rate moreover we show that the proposed method is much more efficient than the least squares based method for many non normal errors and that it only loses a small amount of efficiency for normal errors in addition it is shown that the loss in efficiency is at most 11 1 for estimating varying coefficient functions and is no greater than 13 6 for estimating parametric components to achieve sparsity with high dimensional covariates we propose adaptive penalization methods for variable selection in the semiparametric varying coefficient partially linear model and prove that the methods possess the oracle property extensive monte carlo simulation studies are conducted to examine the finite sample performance of the proposed procedures finally we apply the new methods to analyze the plasma beta carotene level data

sparsistency and rates of convergence in large covariance matrix estimation this paper studies the sparsistency and rates of convergence for estimating sparse covariance and precision matrices based on penalized likelihood with nonconvex penalty functions here sparsistency refers to the property that all parameters that are zero are actually estimated as zero with probability tending to one depending on the case of applications sparsity priori may occur on the covariance matrix its inverse or its cholesky decomposition we study these three sparsity exploration problems under a unified framework with a general penalty function we show that the rates of convergence for these problems under the frobenius norm are of order s n log p n n 1 2 where s n is the number of nonzero elements p n is the size of the covariance matrix and n is the sample size this explicitly spells out the contribution of high dimensionality is merely of a logarithmic factor the conditions on the rate with which the tuning parameter n goes to 0 have been made explicit and compared under different penalties as a result for the l 1 penalty to guarantee the sparsistency and optimal rate of convergence the number of nonzero elements should be small sn o pn at most among o pn2 parameters for estimating sparse covariance or correlation matrix sparse precision or inverse correlation matrix or sparse cholesky factor where sn is the number of the nonzero elements on the off diagonal entries on the other hand using the scad or hard thresholding penalty functions there is no such a restriction

robust variable selection with exponential squared loss robust variable selection procedures through penalized regression have been gaining increased attention in the literature they can be used to perform variable selection and are expected to yield robust estimates however to the best of our knowledge the robustness of those penalized regression procedures has not been well characterized in this article we propose a class of penalized robust regression estimators based on exponential squared loss the motivation for this new procedure is that it enables us to characterize its robustness in a way that has not been done for the existing procedures while its performance is near optimal and superior to some recently developed methods specifically under defined regularity conditions our estimators are consistent and possess the oracle property importantly we show that our estimators can achieve the highest asymptotic breakdown point of 1 2 and that their influence functions are bounded with respect to the outliers in either the response or the covari

a gradient algorithm locally equivalent to the em algorithm in many problems of maximum likelihood estimation it is impossible to carry out either the e step or the m step of the em algorithm the present paper introduces a gradient algorithm that is closely related to the em algorithm this em gradient algorithm approximately solves the m step of the em algorithm by one iteration of newton s method since newton s method converges quickly the local properties of the em gradient algorithm are almost identical with those of the em algorithm any strict local maximum point of the observed likelihood locally attracts the em and em gradient algorithm at the same rate of convergence and near the maximum point the em gradient algorithm always produces an increase in the likelihood with proper modification the em gradient algorithm also exhibits global convergence properties that are similar to those of the em algorithm our proof of global convergence applies and improves existing theory for the em algorithm these theoretical points are reinforced by a discussion of three realistic examples illustrating how the em gradient algorithm can succeed where the em algorithm is intractable

penalized high dimensional empirical likelihood we propose penalized empirical likelihood for parameter estimation and variable selection for problems with diverging numbers of parameters our results are demonstrated for estimating the mean vector in multivariate analysis and regression coefficients in linear models by using an appropriate penalty function we showthat penalized empirical likelihood has the oracle property that is with probability tending to 1 penalized empirical likelihood identifies the true model and estimates the nonzero coefficients as efficiently as if the sparsity of the true model was known in advance the advantage of penalized empirical likelihood as a nonparametric likelihood approach is illustrated by testing hypotheses and constructing confidence regions numerical simulations confirm our theoretical findings

efficient global approximation of generalized nonlinear l1 regularized solution paths and its applications we consider efficient construction of nonlinear solution paths for general l1 regularization unlike the existing methods that incrementally build the solution path through a combination of local linear approximation and recalibration we propose an efficient global approximation to the whole solution path with the loss function approximated by a quadratic spline we show that the solution path can be computed using a generalized lars algorithm the proposed methodology avoids high dimensional numerical optimization and thus provides faster and more stable computation the methodology also can be easily extended to more general regularization framework we illustrate such flexibility with several examples including a generalization of the elastic net and a new method that effectively exploits the so called support vectors in kernel logistic regression

partially linear structure selection in cox models with varying coefficients summary to explore the nonlinear interactions between covariates and an index variable partially linear proportional hazards models have been proposed for censored survival data however specification of the partially linear structure was usually carried out in an ad hoc manner by first fitting a full varying coefficient model and visually examining the resulting fit to identify the linear part in this article we consider the problem of coefficient estimation and constant coefficient identification based on a double shrinkage approach variable selection is also considered in a coherent estimation framework resulting in a double penalization procedure under the mild assumptions we establish asymptotic properties for the procedure such as consistency sparesistency constansistency and asymptotic normality we evaluate the performance of the proposed method by numerical simulations and demonstrate its application using a breast cancer data set

hidden markov models with applications in cell adhesion experiments estimation of the number of hidden states is challenging in hidden markov models motivated by the analysis of a specific type of cell adhesion experiments a new framework based on a hidden markov model and double penalized order selection is proposed the order selection procedure is shown to be consistent in estimating the number of states a modified expectation maximization algorithm is introduced to efficiently estimate parameters in the model simulations show that the proposed framework outperforms existing methods applications of the proposed methodology to real data demonstrate the accuracy of estimating receptor ligand bond lifetimes and waiting times which are essential in kinetic parameter estimation supplementary materials for this article are available online

semiparametric model building for regression models with time varying parameters abstract this paper considers the problem of semiparametric model building for linear regression models with potentially time varying coefficients by allowing the response variable and explanatory variables be jointly a nonstationary process the proposed methods are widely applicable to nonstationary and dependent observations we propose a local linear shrinkage method that can simultaneously achieve parameter estimation and variable selection its selection consistency and the favorable oracle property are established due to the fear of losing efficiency an information criterion is further proposed for distinguishing between time varying and time constant components numerical examples are presented to illustrate the proposed methods

some notes on the nonnegative garrote some notes on the use of the nonnegative garrote ng are given in this article the main result is that compared with other penalized least squares methods the ng has a natural selection of penalty function according to an estimator of prediction risk furthermore two natural estimators of the tuning parameter which only involve very simple computations are proposed corresponding to the akaike information criterion aic and bayesian information criterion bic respectively this indicates that to select tuning parameters it may be unnecessary to optimize a model selection criterion repeatedly several reasonable ng estimators with natural tuning parameters are proposed for multicollinearity problems and other problems simulation results and a real data analysis are reported for testing the results obtained previously

weighted wilcoxon type smoothly clipped absolute deviation method summary shrinkage type variable selection procedures have recently seen increasing applications in biomedical research however their performance can be adversely influenced by outliers in either the response or the covariate space this article proposes a weighted wilcoxon type smoothly clipped absolute deviation ww scad method which deals with robust variable selection and robust estimation simultaneously the new procedure can be conveniently implemented with the statistical software r we establish that the ww scad correctly identifies the set of zero coefficients with probability approaching one and estimates the nonzero coefficients with the rate n 1 2 moreover with appropriately chosen weights the ww scad is robust with respect to outliers in both the x and y directions the important special case with constant weights yields an oracle type estimator with high efficiency in the presence of heavier tailed random errors the robustness of the ww scad is partly justified by its asymptotic performance under local shrinking contamination we propose a bayesian information criterion type tuning parameter selector for the ww scad the performance of the ww scad is demonstrated via simulations and by an application to a study that investigates the effects of personal characteristics and dietary factors on plasma beta carotene level

on selection of spatial linear models for lattice data summary spatial linear models are popular for the analysis of data on a spatial lattice but statistical techniques for selection of covariates and a neighbourhood structure are limited here we develop new methodology for simultaneous model selection and parameter estimation via penalized maximum likelihood under a spatial adaptive lasso a computationally efficient algorithm is devised for obtaining approximate penalized maximum likelihood estimates asymptotic properties of penalized maximum likelihood estimates and their approximations are established a simulation study shows that the method proposed has sound finite sample properties and for illustration we analyse an ecological data set in western canada

mixtures envelopes and hierarchical duality summary we develop a connection between mixture and envelope representations of objective functions that arise frequently in statistics we refer to this connection by using the term hierarchical duality our results suggest an interesting and previously underexploited relationship between marginalization and profiling or equivalently between the fenchel moreau theorem for convex functions and the bernstein widder theorem for laplace transforms we give several different sets of conditions under which such a duality result obtains we then extend existing work on envelope representations in several ways including novel generalizations to variance mean models and to multivariate gaussian location models this turns out to provide an elegant missing data interpretation of the proximal gradient method which is a widely used algorithm in machine learning we show several statistical applications in which the framework proposed leads to easily implemented algorithms including a robust version of the fused lasso non linear quantile regression via trend filtering and the binomial fused double pareto model code for the examples is available on github at https github com jgscott hierduals

regularization parameter selections via generalized information criterion we apply the nonconcave penalized likelihood approach to obtain variable selections as well as shrinkage estimators this approach relies heavily on the choice of regularization parameter which controls the model complexity in this paper we propose employing the generalized information criterion encompassing the commonly used akaike information criterion aic and bayesian information criterion bic for selecting the regularization parameter our proposal makes a connection between the classical variable selection criteria and the regularization parameter selections for the nonconcave penalized likelihood approaches we show that the bic type selector enables identification of the true model consistently and the resulting estimator possesses the oracle property in the terminology of fan and li 2001 in contrast however the aic type selector tends to overfit with positive probability we further show that the aic type selector is asymptotically loss efficient while the bic type selector is not

global solutions to folded concave penalized nonconvex learning this paper is concerned with solving nonconvex learning problems with folded concave penalty despite that their global solutions entail desirable statistical properties there lack optimization techniques that guarantee global optimality in a general setting in this paper we show that a class of nonconvex learning problems are equivalent to general quadratic programs this equivalence facilitates us in developing mixed integer linear programming reformulations which admit finite algorithms that find a provably global optimal solution we refer to this reformulation based technique as the mixed integer programming based global optimization mipgo to our knowledge this is the first global optimization scheme with a theoretical guarantee for folded concave penalized nonconvex learning with the scad penalty fan and li 2001 and the mcp penalty zhang 2010 numerical results indicate a significant outperformance of mipgo over the state of the art solution scheme local linear approximation and other alternative solution techniques in literature in terms of solution quality

majorized proximal alternating imputation for regularized rank constrained matrix completion abstract low rank approximation in the presence of missing data is ubiquitous in many areas of science and engineering as an np hard problem spectral regularization e g nuclear norm schatten p norm is widely used however solving this problem directly can be computationally expensive due to the unknown rank of variables and full rank singular value decompositions svd bilinear factorization is efficient and effectively benefits from the fast numerical optimization however most of existing methods require explicit knowledge of the rank and often tend to be flatlining or over fitting to balance and take advantage of both we formulate the matrix completion problem under the regularized rank constrained conditions we relax this problem to the surrogates with majorization minimization mm by employing a proximal alternating minimization method to optimize the corresponding surrogates we propose an efficient and accurate algorithm named majorized proximal alternating imputation mpa impute algorithm the proposed method iteratively replaces the missing data at each step with the most recent estimate and only involves simple matrix computation and small scale svd theoretically the proposed method is guaranteed to converge to the stationary point under mild conditions moreover the proposed method can be generalized to a family of non convex regularization problems extensive experiments over synthetic data and real world dataset demonstrate the superior performance of our method

partially functional linear regression in high dimensions summary in modern experiments functional and nonfunctional data are often encountered simultaneously when observations are sampled from random processes and high dimensional scalar covariates it is difficult to apply existing methods for model selection and estimation we propose a new class of partially functional linear models to characterize the regression between a scalar response and covariates of both functional and scalar types the new approach provides a unified and flexible framework that simultaneously takes into account multiple functional and ultrahigh dimensional scalar predictors enables us to identify important features and offers improved interpretability of the estimators the underlying processes of the functional predictors are considered to be infinite dimensional and one of our contributions is to characterize the effects of regularization on the resulting estimators we establish the consistency and oracle properties of the proposed method under mild conditions demonstrate its performance with simulation studies and illustrate its application using air pollution data

a consistent information criterion for support vector machines in diverging model spaces information criteria have been popularly used in model selection and proved to possess nice theoretical properties for classification claeskens et al 2008 proposed support vector machine information criterion for feature selection and provided encouraging numerical evidence yet no theoretical justification was given there this work aims to fill the gap and to provide some theoretical justifications for support vector machine information criterion in both fixed and diverging model spaces we first derive a uniform convergence rate for the support vector machine solution and then show that a modification of the support vector machine information criterion achieves model selection consistency even when the number of features diverges at an exponential rate of the sample size this consistency result can be further applied to selecting the optimal tuning parameter for various penalized support vector machine methods finite sample performance of the proposed information criterion is investigated using monte carlo studies and one real world gene selection problem

variable selection in semiparametric regression modeling in this paper we are concerned with how to select significant variables in semiparametric modeling variable selection for semiparametric regression models consists of two components model selection for nonparametric components and select significant variables for parametric portion thus it is much more challenging than that for parametric models such as linear models and generalized linear models because traditional variable selection procedures including stepwise regression and the best subset selection require model selection to nonparametric components for each submodel this leads to very heavy computational burden in this paper we propose a class of variable selection procedures for semiparametric regression models using nonconcave penalized likelihood the newly proposed procedures are distinguished from the traditional ones in that they delete insignificant variables and estimate the coefficients of significant variables simultaneously this allows us to establish the sampling properties of the resulting estimate we first establish the rate of convergence of the resulting estimate with proper choices of penalty functions and regularization parameters we then establish the asymptotic normality of the resulting estimate and further demonstrate that the proposed procedures perform as well as an oracle procedure semiparametric generalized likelihood ratio test is proposed to select significant variables in the nonparametric component we investigate the asymptotic behavior of the proposed test and demonstrate its limiting null distribution follows a chi squared distribution which is independent of the nuisance parameters extensive monte carlo simulation studies are conducted to examine the finite sample performance of the proposed variable selection procedures

blow up for the critical generalized korteweg de vries equation i dynamics near the soliton we consider the quintic generalized korteweg de vries equation gkdv u t u xx u 5 x 0 which is a canonical mass critical problem for initial data in h1 close to the soliton in earlier works on this problem finite or infinite time blow up was proved for non positive energy solutions and the solitary wave was shown to be the universal blow up profile see 16 26 and 20 for well localized initial data finite time blow up with an upper bound on blow up rate was obtained in 18