a simple two sample test in high dimensions based on l2 norm testing the equality of two means is a fundamental inference problem for high dimensional data the hotelling s t2 test either performs poorly or becomes inapplicable several modifications have b

high dimensional classification via nonparametric empirical bayes and maximum likelihood inference we propose new nonparametric empirical bayes methods for high dimensional classification our classifiers are designed to approximate the bayes classifier in a hypothesized hierarchical model where the prior distributions for the model parameters are estimated nonparametrically from the training data as is common with nonparametric empirical bayes the proposed classifiers are effective in high dimensional settings even when the underlying model parameters are in fact nonrandom we use nonparametric maximum likelihood estimates of the prior distributions following the elegant approach studied by kiefer wolfowitz in the 1950s however our implementation is based on a recent convex optimization framework for approximating these estimates that is well suited for large scale problems we derive new theoretical results on the accuracy of the approximate estimator which help control the misclassification rate of one of our classifiers we show that our methods outperform several existing methods in simulations and perform well when gene expression microarray data is used to classify cancer patients

covariance regularization by thresholding this paper considers regularizing a covariance matrix of p variables estimated from n observations by hard thresholding we show that the thresholded estimate is consistent in the operator norm as long as the true covariance matrix is sparse in a suitable sense the variables are gaussian or sub gaussian and log p n 0 and obtain explicit rates the results are uniform over families of covariance matrices which satisfy a fairly natural notion of sparsity we discuss an intuitive resampling scheme for threshold selection and prove a general cross validation result that justifies this approach we also compare thresholding to other covariance estimators in simulations and on an example from climate data 1 introduction estimation of covariance matrices is important in a number of areas of statistical analysis including dimension reduction by principal component analysis pca classification by linear or quadratic discriminant analysis lda and qda establishing independence and conditional independence relations in the context of graphical models and setting confidence intervals on linear functions of the means of the components in recent years many application areas where these tools are used have been dealing with very high dimensional datasets and sample sizes can be very small relative to dimension examples include genetic data brain imaging spectroscopic imaging climate data and many others it is well known by now that the empirical covariance matrix for samples of size n from a p variate gaussian distribution np p is not a good estimator of the population covariance if p is large many results in random matrix theory illustrate this from the classical mary law 29 to the more recent work of johnstone and his students on the theory of the largest eigenvalues 12 23 30 and associated eigenvectors 24 however with the exception of a method for estimating the covariance spectrum 11 these probabilistic results do not offer alternatives to the sample covariance matrix alternative estimators for large covariance matrices have therefore attracted a lot of attention recently two broad classes of covariance estimators have emerged those that rely on a natural ordering among variables and assume that variables

ideal spatial adaptation by wavelet shrinkage summary with ideal spatial adaptation an oracle furnishes information about how best to adapt a spatially variable estimator whether piecewise constant piecewise polynomial variable knot spline or variable bandwidth kernel to the unknown function estimation with the aid of an oracle offers dramatic advantages over traditional linear estimation by nonadaptive kernels however it is a priori unclear whether such performance can be obtained by a procedure relying on the data alone we describe a new principle for spatially adaptive estimation selective wavelet reconstruction we show that variable knot spline fits and piecewise polynomial fits when equipped with an oracle to select the knots are not dramatically more powerful than selective wavelet reconstruction with an oracle we develop a practical spatially adaptive method riskshrink which works by shrinkage of empirical wavelet coefficients riskshrink mimics the performance of an oracle for selective wavelet reconstruction as well as it is possible to do so a new inequality in multivariate normal decision theory which we call the oracle inequality shows that attained performance differs from ideal performance by at most a factor of approximately 2 log n where n is the sample size moreover no estimator can give a better guarantee than this within the class of spatially adaptive procedures riskshrink is essentially optimal relying only on the data it comes within a factor log 2 n of the performance of piecewise polynomial and variableknot spline methods equipped with an oracle in contrast it is unknown how or if piecewise polynomial methods could be made to function this well when denied access to an oracle and forced to rely on data alone

shrinking characteristics of precision matrix estimators we propose a framework to shrink a user specified characteristic of a precision matrix estimator that is needed to fit a predictive model estimators in our framework minimize the gaussian negative loglikelihood plus an l 1 penalty on a linear or affine function evaluated at the optimization variable corresponding to the precision matrix we establish convergence rate bounds for these estimators and propose an alternating direction method of multipliers algorithm for their computation our simulation studies show that our estimators can perform better than competitors when they are used to fit predictive models in particular we illustrate cases where our precision matrix estimators perform worse at estimating the population precision matrix but better at prediction

input output stabilization in the general two sided model matching setup the problem of input output stabilization in a general two sided model matching setup is studied as a first step the problem is reduced to a pair of uncoupled bilateral diophantine equations over rh infinity then recent results on bilateral diophantine equations are exploited to obtain a numerically tractable solution given in terms of explicit state space formulae the resulting solvability conditions rely on two uncoupled sylvester equations accompanied by algebraic constraints this is in contrast to the corresponding one sided stabilization where no sylvester equations are required it is shown that imposing a mild simplifying assumption is instrumental in obtaining convenient parameterization of all stabilizing solutions which is affine in a single rh infinity parameter this demonstrates that if the aforementioned assumption is imposed the general two sided stabilization problem is similar to its one sided counterpart in the sense that the constraints imposed by a stability requirement can be resolved without increasing problem complexity less

integrative linear discriminant analysis with guaranteed error rate improvement multiple types of data measured on a common set of subjects arise in many areas numerous empirical studies have found that integrative analysis of such data can result in better statistical performance in terms of prediction and feature selection however the advantages of integrative analysis have mostly been demonstrated empirically in the context of two class classification we propose an integrative linear discriminant analysis method and establish a theoretical guarantee that it achieves a smaller classification error than running linear discriminant analysis on each data type individually we address the issues of outliers and missing values frequently encountered in integrative analysis and illustrate our method through simulations and a neuroimaging study of alzheimer s disease

ultrahigh dimensional multiclass linear discriminant analysis by pairwise sure independence screening this article is concerned with the problem of feature screening for multiclass linear discriminant analysis under ultrahigh dimensional setting we allow the number of classes to be relatively large as a result the total number of relevant features is larger than usual this makes the related classification problem much more challenging than the conventional one where the number of classes is small very often two to solve the problem we propose a novel pairwise sure independence screening method for linear discriminant analysis with an ultrahigh dimensional predictor the proposed procedure is directly applicable to the situation with many classes we further prove that the proposed method is screening consistent simulation studies are conducted to assess the finite sample performance of the new procedure we also demonstrate the proposed methodology via an empirical analysis of a real life example on handwritten chinese character recognition

sparse linear discriminant analysis by thresholding for high dimensional data in many social economical biological and medical studies one objective is to classify a subject into one of several classes based on a set of variables observed from the subject because the probability distribution of the variables is usually unknown the rule of classification is constructed using a training sample the well known linear discriminant analysis lda works well for the situation where the number of variables used for classification is much smaller than the training sample size because of the advance in technologies modern statistical studies often face classification problems with the number of variables much larger than the sample size and the lda may perform poorly we explore when and why the lda has poor performance and propose a sparse lda that is asymptotically optimal under some sparsity conditions on the unknown parameters for illustration of application we discuss an example of classifying human cancer into two classes of leukemia based on a set of 7 129 genes and a training sample of size 72 a simulation is also conducted to check the performance of the proposed method

optimal rates of convergence for sparse covariance matrix estimation this paper considers estimation of sparse covariance matrices and establishes the optimal rate of convergence under a range of matrix operator norm and bregman divergence losses a major focus is on the derivation of a rate sharp minimax lower bound the problem exhibits new features that are significantly different from those that occur in the conventional nonparametric function estimation problems standard techniques fail to yield good results and new tools are thus needed we first develop a lower bound technique that is particularly well suited for treating two directional problems such as estimating sparse covariance matrices the result can be viewed as a generalization of le cam s method in one direction and assouad s lemma in another this lower bound technique is of independent interest and can be used for other matrix estimation problems we then establish a rate sharp minimax lower bound for estimating sparse covariance matrices under the spectral norm by applying the general lower bound technique a thresholding estimator is shown to attain the optimal rate of convergence under the spectral norm the results are then extended to

estimating sparse precision matrix optimal rates of convergence and adaptive estimation precision matrix is of significant importance in a wide range of applications in multivariate analysis this paper considers adaptive minimax estimation of sparse precision matrices in the high dimensional setting optimal rates of convergence are established for a range of matrix norm losses a fully data driven estimator based on adaptive constrained l1 minimization is proposed and its rate of convergence is obtained over a collection of parameter spaces the estimator called aclime is easy to implement and performs well numerically a major step in establishing the minimax rate of convergence is the derivation of a rate sharp lower bound a two directional lower bound technique is applied to obtain the minimax lower bound the upper and lower bounds together yield the optimal rates of convergence for sparse precision matrix estimation and show that the aclime estimator is adaptively minimax rate optimal for a collection of parameter spaces and a range of matrix norm losses simultaneously

sign consistency for the linear programming discriminant rule abstract linear discriminant analysis lda is an important conventional model for data classification classical theory shows that lda is bayes consistent for a fixed data dimensionality p and a large training sample size n however in high dimensional settings when p n lda is difficult due to the inconsistent estimation of the covariance matrix and the mean vectors of populations recently a linear programming discriminant lpd rule was proposed for high dimensional linear discriminant analysis based on the sparsity assumption over the discriminant function it is shown that the lpd rule is bayes consistent in high dimensional settings in this paper we further show that the lpd rule is sign consistent under the sparsity assumption such sign consistency ensures the lpd rule to select the optimal discriminative features for high dimensional data classification problems evaluations on both synthetic and real data validate our result on the sign consistency of the lpd rule

two stage regularized linear discriminant analysis for 2 d data fisher linear discriminant analysis lda involves within class and between class covariance matrices for 2 d data such as images regularized lda rlda can improve lda due to the regularized eigenvalues of the estimated within class matrix however it fails to consider the eigenvectors and the estimated between class matrix to improve these two matrices simultaneously we propose in this paper a new two stage method for 2 d data namely a bidirectional lda blda in the first stage and the rlda in the second stage where both blda and rlda are based on the fisher criterion that tackles correlation blda performs the lda under special separable covariance constraints that incorporate the row and column correlations inherent in 2 d data the main novelty is that we propose a simple but effective statistical test to determine the subspace dimensionality in the first stage as a result the first stage reduces the dimensionality substantially while keeping the significant discriminant information in the data this enables the second stage to perform rlda in a much lower dimensional subspace and thus improves the two estimated matrices simultaneously experiments on a number of 2 d synthetic and real world data sets show that blda rlda outperforms several closely related competitors

a new approach to synthesis problems in linear system theory we consider a rational nonlinear equation which is involved in the general synthesis design problem of linear multivariable systems the complete solution of this equation subject to causality stability and minimality constraints is provided formal power series and in particular the theory of partial realizations prove of central importance for the design problem

two way string matching a new string matching algorithm is presented which can be viewed as an intermediate between the classical algorithms of knuth morris and pratt on the one hand and boyer and moore on the other hand the algorithm is linear in time and uses constant space as the algorithm of galil and seiferas it presents the advantage of being remarkably simple which consequently makes its analysis possible the algorithm relies on a previously known result in combinatorics on words called the critical factorization theorem which relates the global period of a word to its local repetitions of blocks

operator norm consistent estimation of large dimensional sparse covariance matrices estimating covariance matrices is a problem of fundamental importance in multivariate statistics in practice it is increasingly frequent to work with data matrices x of dimension n x p where p and n are both large results from random matrix theory show very clearly that in this setting standard estimators like the sample covariance matrix perform in general very poorly in this large n large p setting it is sometimes the case that practitioners are willing to assume that many elements of the population covariance matrix are equal to 0 and hence this matrix is sparse we develop an estimator to handle this situation the estimator is shown to be consistent in operator norm when for instance we have p n as n oo in other words the largest singular value of the difference between the estimator and the population covariance matrix goes to zero this implies consistency of all the eigenvalues and consistency of eigenspaces associated to isolated eigenvalues we also propose a notion of sparsity for matrices that is compatible with spectral analysis and is independent of the ordering of the variables

general solution of robust tracking problem in two degree of freedom control systems in this note we study linear time invariant multivariable systems with two degree of freedom compensators first we give a new parametrization of all stabilizing compensators from the viewpoint of independent design of transfer function and robustness property second utilizing this parametrization we solve a general robust tracking problem including the case of nonsquare plants and show that tracking and robustness can be independently specified in two degree of freedom control systems

sliced inverse regression for dimension reduction abstract modern advances in computing power have greatly widened scientists scope in gathering and investigating information from many variables information which might have been ignored in the past yet to effectively scan a large pool of variables is not an easy task although our ability to interact with data has been much enhanced by recent innovations in dynamic graphics in this article we propose a novel data analytic tool sliced inverse regression sir for reducing the dimension of the input variable x without going through any parametric or nonparametric model fitting process this method explores the simplicity of the inverse view of regression that is instead of regressing the univariate output variable y against the multivariate x we regress x against y forward regression and inverse regression are connected by a theorem that motivates this method the theoretical properties of sir are investigated under a model of the form y f 1 x k x e where the k s are the unknown

regularized estimation of large covariance matrices this paper considers estimating a covariance matrix of p variables from n observations by either banding the sample covariance matrix or estimating a banded version of the inverse of the covariance we show that these estimates are consistent in the operator norm as long as logp 2 n 0 and obtain explicit rates the results are uniform over some fairly natural well conditioned families of covariance matrices we also introduce an analogue of the gaussian white noise model and show that if the population covariance is embeddable in that model and well conditioned then the banded approximations produce consistent estimates of the eigenvalues and associated eigenvectors of the covariance matrix the results can be extended to smooth versions of banding and to non gaussian distributions with su ciently short tails a resampling approach is proposed for choosing the banding parameter in practice this approach is illustrated numerically on both simulated and real data

simple real time constant space string matching string matching is the classical problem of finding all occurrences of a pattern in a text a real time string matching algorithm takes worst case constant time to check if a pattern occurrence ends at each text location we derive a real time variation of the elegant crochemore perrin constant space string matching algorithm that has a simple and efficient control structure we use observations about the locations of critical factorizations to deploy two tightly coupled simplified real time instances of the crochemore perrin algorithm that search for complementary parts of the pattern whose simultaneous occurrence indicates an occurrence of the complete pattern

generalized quadratic discriminant analysis in linear discriminant analysis the assumption of equality of the dispersion matrices of different classes leads to a classification rule based on minimum mahalanobis distance from the class centres however without this assumption the resulting quadratic discriminant classifier involves in addition to the mahalanobis distances the ratio of the determinants of the dispersion matrices as a factor in fact it has been observed that for discriminating between populations with underlying elliptically symmetric distributions such classifiers also incorporate similar factors apart from the mahalanobis distances in this paper a nonparametric classification technique which generalizes discriminant analysis has been proposed the method of cross validation is used to make the technique adaptive to a given dataset an extensive simulation study is presented to illustrate the potential of the method finally through implementation on a number of real life data sets it has been demonstrated that the proposed generalized quadratic discriminant analysis gqda compares very favourably with other nonparametric methods and is computationally cost effective highlightsa nonparametric classification technique has been proposed it generalizes fisher s discriminant analysis cross validation is used to make the technique adaptive to given data found empirically to compare very favourably with other nonparametric methods computationally cost effective

on maximal suffixes and constant space linear time versions of kmp algorithm constant space linear time string matching algorithms are usually very sophisticated most of them consist of two phases very technical preprocessing phase and searching phase an exception is one phase crochemore s algorithm theoret comput sci 92 1992 33 it is an on line version of knuth morris pratt algorithm kmp with on the fly computation of pattern shifts as approximate periods in this paper we explore further crochemore s approach and construct alternative algorithms which are differently structured in crochemore s algorithm the approximate period function is restarted from inside which means that several internal variables of this function are changing globally also crochemore s algorithm strongly depends on the concrete implementation of approximate periods computation we present a simple modification of kmp algorithm which works in o 1 space o n time for any function which computes periods or approximate periods in o 1 space and linear time the approximate period function can be treated as a black box we identify class of patterns self maximal words which are especially well suited for crochemore style string matching a new o 1 space string matching algorithm maxsuffix matching is proposed in the paper which gives yet another example of applicability of maximal suffixes

coda high dimensional copula discriminant analysis we propose a high dimensional classification method named the copula discriminant analysis coda the coda generalizes the normal based linear discriminant analysis to the larger gaussian copula models or the nonparanormal as proposed by liu et al 2009 to simultaneously achieve estimation efficiency and robustness the nonparametric rank based methods including the spearman s rho and kendall s tau are exploited in estimating the covariance matrix in high dimensional settings we prove that the sparsity pattern of the discriminant features can be consistently recovered with the parametric rate and the expected misclassification error is consistent to the bayes risk our theory is backed up by careful numerical experiments which show that the extra flexibility gained by the coda method incurs little efficiency loss even when the data are truly gaussian these results suggest that the coda method can be an alternative choice besides the normal based high dimensional linear discriminant analysis

adaptive covariance matrix estimation through block thresholding estimation of large covariance matrices has drawn considerable recent attention and the theoretical focus so far has mainly been on developing a minimax theory over a fixed parameter space in this paper we consider adaptive covariance matrix estimation where the goal is to construct a single procedure which is minimax rate optimal simultaneously over each parameter space in a large collection a fully data driven block thresholding estimator is proposed the estimator is constructed by carefully dividing the sample covariance matrix into blocks and then simultaneously estimating the entries in a block by thresholding the estimator is shown to be optimally rate adaptive over a wide range of bandable covariance matrices a simulation study is carried out and shows that the block thresholding estimator performs well numerically some of the technical tools developed in this paper can also be of independent interest

optimal classification in sparse gaussian graphic model consider a two class classification problem where the number of features is much larger than the sample size the features are masked by gaussian noise with mean zero and covariance matrix sigma where the precision matrix omega sigma 1 is unknown but is presumably sparse the useful features also unknown are sparse and each contributes weakly i e rare and weak to the classification decision by obtaining a reasonably good estimate of omega we formulate the setting as a linear regression model we propose a two stage classification method where we first select features by the method of innovated thresholding it and then use the retained features and fisher s lda for classification in this approach a crucial problem is how to set the threshold of it we approach this problem by adapting the recent innovation of higher criticism thresholding hct we find that when useful features are rare and weak the limiting behavior of hct is essentially just as good as the limiting behavior of ideal threshold the threshold one would choose if the underlying distribution of the signals is known if only somewhat surprisingly when omega is sufficiently sparse its off diagonal coordinates usually do not have a major influence over the classification decision compared to recent work in the case where omega is the identity matrix proc natl acad sci usa 105 2008 14790 14795 philos trans r soc lond ser a math phys eng sci 367 2009 4449 4470 the current setting is much more general which needs a new approach and much more sophisticated analysis one key component of the analysis is the intimate relationship between hct and fisher s separation another key component is the tight large deviation bounds for empirical processes for data with unconventional correlation structures where graph theory on vertex coloring plays an important role

multiclass linear discriminant analysis with ultrahigh dimensional features within the framework of fisher s discriminant analysis we propose a multiclass classification method which embeds variable screening for ultrahigh dimensional predictors leveraging interfeature correlations we show that the proposed linear classifier recovers informative features with probability tending to one and can asymptotically achieve a zero misclassification rate we evaluate the finite sample performance of the method via extensive simulations and use this method to classify posttransplantation rejection types based on patients gene expressions

a state space parametrization of discrete time two degree of freedom integral controllers abstract using the observer based integral controller proposed by ishihara h j guo and h takeda 1992 automatica 28 599 603 as a basic controller we propose a state space parametrization of discrete time two degree of freedom integral controllers accounting computation delay the proposed parametrization can easily be obtained by using the solution of an observer based regulator problem for a plant clear interpretation of the parametrization as an observer based integral controller is given as application we discuss the loop transfer recovery and the dead beat control using the free parameters

high dimensional semiparametric gaussian copula graphical models ing the spearman s rho and kendall s tau we prove that the nonparanormal skeptic achieves the optimal parametric rates of convergence for both graph recovery and parameter estimation this result suggests that the nonparanormal graphical models can be used as a safe replacement of the popular gaussian graphical models even when the data are truly gaussian besides theoretical analysis we also conduct thorough numerical simulations to compare the graph recovery performance of dierent estimators under both ideal and noisy settings the proposed methods are then applied on a largescale genomic dataset to illustrate their empirical usefulness the r package huge implementing the proposed methods is available on the comprehensive r archive network http cran r project org

optimization approaches to supervised classification abstract the supervised classification problem one of the oldest and most recurrent problems in applied data analysis has always been analyzed from many different perspectives when the emphasis is placed on its overall goal of developing classification rules with minimal classification cost supervised classification can be understood as an optimization problem on the other hand when the focus is in modeling the uncertainty involved in the classification of future unknown entities it can be formulated as a statistical problem other perspectives that pay particular attention to pattern recognition and machine learning aspects of supervised classification have also a long history that has lead to influential insights and different methodologies in this review two approaches to supervised classification strongly related to optimization theory will be discussed and compared in particular we will review methodologies based on mathematical programming models that optimize observable criteria linked to the true objective of misclassification error or cost minimization and approaches derived from the minimization of known bounds on the true misclassification error the former approach is known as the mathematical programming approach to supervised classification while the latter is in the origin of the well known classification support vector machines throughout the review two group as well as general multi group problems will be considered and the review will conclude with a discussion of the most promising research directions in this area

covariate adjusted tensor classification in high dimensions in contemporary scientific research it is often of great interest to predict a categorical response based on a high dimensional tensor i e multi dimensional array and additional covariates mo

simultaneous sparse estimation of canonical vectors in the p n setting abstractthis article considers the problem of sparse estimation of canonical vectors in linear discriminant analysis when p n several methods have been proposed in the literature that estimate one canonical vector in the two group case however g 1 canonical vectors can be considered if the number of groups is g in the multi group context it is common to estimate canonical vectors in a sequential fashion moreover separate prior estimation of the covariance structure is often required we propose a novel methodology for direct estimation of canonical vectors in contrast to existing techniques the proposed method estimates all canonical vectors at once performs variable selection across all the vectors and comes with theoretical guarantees on the variable selection and classification consistency first we highlight the fact that in the n p setting the canonical vectors can be expressed in a closed form up to an orthogonal transformation secondly we propose an extension of this form to the

high dimensional classification using features annealed independence rules classification using high dimensional features arises frequently in many contemporary statistical studies such as tumor classification using microarray or other high throughput data the impact of dimensionality on classifications is largely poorly understood in a seminal paper bickel and levina 2004 show that the fisher discriminant performs poorly due to diverging spectra and they propose to use the independence rule to overcome the problem we first demonstrate that even for the independence classification rule classification using all the features can be as bad as the random guessing due to noise accumulation in estimating population centroids in high dimensional feature space in fact we demonstrate further that almost all linear discriminants can perform as bad as the random guessing thus it is paramountly important to select a subset of important features for high dimensional classification resulting in features annealed independence rules fair the conditions under which all the important features can be selected by the two sample t statistic are established the choice of the optimal number of features or equivalently the threshold value of the test statistics are proposed based on an upper bound of the classification error simulation studies and real data analysis support our theoretical results and demonstrate convincingly the advantage of our new classification procedure

one step sparse estimates in nonconcave penalized likelihood models fan and li propose a family of variable selection methods via penalized likelihood using concave penalty functions the nonconcave penalized likelihood estimators enjoy the oracle properties but maximizing the penalized likelihood function is computationally challenging because the objective function is nondifferentiable and nonconcave in this article we propose a new unified algorithm based on the local linear approximation lla for maximizing the penalized likelihood for a broad class of concave penalty functions convergence and other theoretical properties of the lla algorithm are established a distinguished feature of the lla algorithm is that at each lla step the lla estimator can naturally adopt a sparse representation thus we suggest using the one step lla estimator from the lla algorithm as the final estimates statistically we show that if the regularization parameter is appropriately chosen the one step lla estimates enjoy the oracle properties with good initial estimators computationally the one step lla estimation methods dramatically reduce the computational cost in maximizing the nonconcave penalized likelihood we conduct some monte carlo simulation to assess the finite sample performance of the one step sparse estimation methods the results are very encouraging

piecewise linear regularized solution paths we consider the generic regularized optimization problem argmin l y x j efron hastie johnstone and tibshirani ann statist 32 2004 407 499 have shown that for the lasso that is if l is squared error loss and j 1 is the l 1 norm of the optimal coefficient path is piecewise linear that is is piecewise constant we derive a general characterization of the properties of loss l penalty j pairs which give piecewise linear coefficient paths such pairs allow for efficient generation of the full regularized coefficient paths we investigate the nature of efficient path following algorithms which arise we use our results to suggest robust versions of the lasso for regression and classification and to develop new efficient algorithms for existing problems in the literature including mammen and van de geer s locally adaptive regression splines

covariance enhanced discriminant analysis linear discriminant analysis has been widely used to characterize or separate multiple classes via linear combinations of features however the high dimensionality of features from modern biological experiments defies traditional discriminant analysis techniques possible interfeature correlations present additional challenges and are often underused in modelling in this paper by incorporating possible interfeature correlations we propose a covariance enhanced discriminant analysis method that simultaneously and consistently selects informative features and identifies the corresponding discriminable classes under mild regularity conditions we show that the method can achieve consistent parameter estimation and model selection and can attain an asymptotically optimal misclassification rate extensive simulations have verified the utility of the method which we apply to a renal transplantation trial

adaptive thresholding for sparse covariance matrix estimation in this article we consider estimation of sparse covariance matrices and propose a thresholding procedure that is adaptive to the variability of individual entries the estimators are fully data driven and demonstrate excellent performance both theoretically and numerically it is shown that the estimators adaptively achieve the optimal rate of convergence over a large class of sparse covariance matrices under the spectral norm in contrast the commonly used universal thresholding estimators are shown to be suboptimal over the same parameter spaces support recovery is discussed as well the adaptive thresholding estimators are easy to implement the numerical performance of the estimators is studied using both simulated and real data simulation results demonstrate that the adaptive thresholding estimators uniformly outperform the universal thresholding estimators the method is also illustrated in an analysis on a dataset from a small round blue cell tumor microarray experiment a supplement to this ar

sparsifying the fisher linear discriminant by rotation many high dimensional classification techniques have been proposed in the literature based on sparse linear discriminant analysis lda to efficiently use them sparsity of linear classifiers is a prerequisite however this might not be readily available in many applications and rotations of data are required to create the needed sparsity in this paper we propose a family of rotations to create the required sparsity the basic idea is to use the principal components of the sample covariance matrix of the pooled samples and its variants to rotate the data first and to then apply an existing high dimensional classifier this rotate and solve procedure can be combined with any existing classifiers and is robust against the sparsity level of the true model we show that these rotations do create the sparsity needed for high dimensional classifications and provide theoretical understanding why such a rotation works empirically the effectiveness of the proposed method is demonstrated by a number of simulated and real data examples and the improvements of our method over some popular high dimensional classification rules are clearly shown

model selection and estimation in the gaussian graphical model summary we propose penalized likelihood methods for estimating the concentration matrix in the gaussian graphical model the methods lead to a sparse and shrinkage estimator of the concentration matrix that is positive definite and thus conduct model selection and estimation simultaneously the implementation of the methods is nontrivial because of the positive definite constraint on the concentration matrix but we show that the computation can be done effectively by taking advantage of the efficient maxdet algorithm developed in convex optimization we propose a bic type criterion for the selection of the tuning parameter in the penalized likelihood methods the connection between our methods and existing methods is illustrated simulations and real examples demonstrate the competitive performance of the new methods

the zooming method a recursive approach to time space efficient string matching a new approach to time space efficient string matching is presented the method is flexible its implementation depends whether or not the alphabet is linearly ordered the only known linear time constant space algorithm for string matching over nonordered alphabets is the galil seiferas algorithm see crochemore 1993 and galil 1983 which are rather complicated the zooming method gives probably the simplest string matching algorithm working in constant space and linear time for nonordered alphabets the novel feature of our algorithm is the application of the searching phase which is usually simpler than preprocessing in the preprocessing phase the preprocessing has a recursive structure similar to selection in linear time see aho 1974 for ordered alphabets the preprocessing part is much simpler its basic component is a simple and well known algorithm for finding the maximal suffix see duval 1983 hence we demonstrate a new application of this algorithm see also crochemore 1991 the idea of the zooming method was applied by crochemore et al 1995 to two dimensional patterns

optimal rates of convergence for covariance matrix estimation covariance matrix plays a central role in multivariate statistical analysis significant advances have been made recently on developing both theory and methodology for estimating large covariance matrices however a minimax theory has yet been developed in this paper we establish the optimal rates of convergence for estimating the covariance matrix under both the operator norm and frobenius norm it is shown that optimal procedures under the two norms are different and consequently matrix estimation under the operator norm is fundamentally different from vector estimation the minimax upper bound is obtained by constructing a special class of tapering estimators and by studying their risk properties a key step in obtaining the optimal rate of convergence is the derivation of the minimax lower bound the technical analysis requires new ideas that are quite different from those used in the more conventional function sequence estimation problems

a constrained l1 minimization approach to sparse precision matrix estimation this article proposes a constrained l1 minimization method for estimating a sparse inverse covariance matrix based on a sample of n iid p variate random variables the resulting estimator is shown to have a number of desirable properties in particular the rate of convergence between the estimator and the true s sparse precision matrix under the spectral norm is when the population distribution has either exponential type tails or polynomial type tails we present convergence rates under the elementwise l norm and frobenius norm in addition we consider graphical model selection the procedure is easily implemented by linear programming numerical performance of the estimator is investigated using both simulated and real data in particular the procedure is applied to analyze a breast cancer dataset and is found to perform favorably compared with existing methods

optimal feature selection in high dimensional discriminant analysis we consider the high dimensional discriminant analysis problem for this problem different methods have been proposed and justified by establishing exact convergence rates for the classification risk as well as the ell 2 convergence results to the discriminative rule however sharp theoretical analysis for the variable selection performance of these procedures have not been established even though model interpretation is of fundamental importance in scientific data analysis this paper bridges the gap by providing sharp sufficient conditions for consistent variable selection using the sparse discriminant analysis through careful analysis we establish rates of convergence that are significantly faster than the best known results and admit an optimal scaling of the sample size n dimensionality p and sparsity level s in the high dimensional setting sufficient conditions are complemented by the necessary information theoretic limits on the variable selection problem in the context of high dimensional discriminant analysis exploiting a numerical equivalence result our method also establish the optimal results for the road estimator and the sparse optimal scoring estimator furthermore we analyze an exhaustive search procedure whose performance serves as a benchmark and show that it is variable selection consistent under weaker conditions extensive simulations demonstrating the sharpness of the bounds are also provided

high dimensional linear discriminant analysis optimality adaptive algorithm and missing data the paper develops optimality theory for linear discriminant analysis in the high dimensional setting a data driven and tuning free classification rule which is based on an adaptive constrained l1 minimization approach is proposed and analysed minimax lower bounds are obtained and this classification rule is shown to be simultaneously rate optimal over a collection of parameter spaces in addition we consider classification with incomplete data under the missingness completely at random model an adaptive classifier with theoretical guarantees is introduced and the optimal rate of convergence for high dimensional linear discriminant analysis under the missingness completely at random model is established the technical analysis for the case of missing data is much more challenging than that for complete data we establish a large deviation result for the generalized sample covariance matrix which serves as a key technical tool and can be of independent interest an application to lung cancer and leukaemia studies is also discussed

sparsistency and rates of convergence in large covariance matrix estimation this paper studies the sparsistency and rates of convergence for estimating sparse covariance and precision matrices based on penalized likelihood with nonconvex penalty functions here sparsistency refers to the property that all parameters that are zero are actually estimated as zero with probability tending to one depending on the case of applications sparsity priori may occur on the covariance matrix its inverse or its cholesky decomposition we study these three sparsity exploration problems under a unified framework with a general penalty function we show that the rates of convergence for these problems under the frobenius norm are of order s n log p n n 1 2 where s n is the number of nonzero elements p n is the size of the covariance matrix and n is the sample size this explicitly spells out the contribution of high dimensionality is merely of a logarithmic factor the conditions on the rate with which the tuning parameter n goes to 0 have been made explicit and compared under different penalties as a result for the l 1 penalty to guarantee the sparsistency and optimal rate of convergence the number of nonzero elements should be small sn o pn at most among o pn2 parameters for estimating sparse covariance or correlation matrix sparse precision or inverse correlation matrix or sparse cholesky factor where sn is the number of the nonzero elements on the off diagonal entries on the other hand using the scad or hard thresholding penalty functions there is no such a restriction

robust estimation of high dimensional covariance and precision matrices high dimensional data are often most plausibly generated from distributions with complex structure and leptokurtosis in some or all components covariance and precision matrices provide a useful summary of such structure yet the performance of popular matrix estimators typically hinges upon a sub gaussianity assumption this paper presents robust matrix estimators whose performance is guaranteed for a much richer class of distributions the proposed estimators under a bounded fourth moment assumption achieve the same minimax convergence rates as do existing methods under a sub gaussianity assumption consistency of the proposed estimators is also established under the weak assumption of bounded 2 e moments for e 0 2 the associated convergence rates depend on e

a road to classification in high dimensional space the regularized optimal affine discriminant for high dimensional classification it is well known that naively performing the fisher discriminant rule leads to poor results due to diverging spectra and noise accumulation therefore researchers proposed independence rules to circumvent the diverging spectra and sparse independence rules to mitigate the issue of noise accumulation however in biological applications there are often a group of correlated genes responsible for clinical outcomes and the use of the covariance information can significantly reduce misclassification rates in theory the extent of such error rate reductions is unveiled by comparing the misclassification rates of the fisher discriminant rule and the independence rule to materialize the gain based on finite samples a regularized optimal affine discriminant road is proposed road selects an increasing number of features as the regularization relaxes further benefits can be achieved when a screening method is employed to narrow the feature pool before hitting the road an efficient constrained coordinate descent algorithm ccd is also developed to solve the associated optimization problems sampling properties of oracle type are established simulation studies and real data analysis support our theoretical results and demonstrate the advantages of the new classification procedure under a variety of correlation structures a delicate result on continuous piecewise linear solution path for the road optimization problem at the population level justifies the linear interpolation of the ccd algorithm

on the class of all stabilizing regulators an alternative characterization is given of the class of stabilizing regulators which is convenient to achieve online or offline h sup 2 h sup infinity optimization for performance and robustness the characterization uses disturbance signal reconstruction and nulling filters the theory allows a reinterpretation of the necessary and sufficient conditions for regulation also connections between structurally stable regulation and the internal model principle are exposed

matrix linear discriminant analysis we propose a novel linear discriminant analysis lda approach for the classification of high dimensional matrix valued data that commonly arises from imaging studies motivated by the equivalence

prediction by supervised principal components in regression problems where the number of predictors greatly exceeds the number of observations conventional regression techniques may produce unsatisfactory results we describe a technique called supervised principal components that can be applied to this type of problem supervised principal components is similar to conventional principal components analysis except that it uses a subset of the predictors selected based on their association with the outcome supervised principal components can be applied to regression and generalized regression problems such as survival analysis it compares favorably to other techniques for this type of problem and can also account for the effects of other covariates and help identify which predictor variables are most important we also provide asymptotic consistency results to help support our empirical findings these methods could become important tools for dna microarray data where they may be used to more accurately diagnose and treat cancer

the nonparanormal semiparametric estimation of high dimensional undirected graphs recent methods for estimating sparse undirected graphs for real valued data in high dimensional problems rely heavily on the assumption of normality we show how to use a semiparametric gaussian copula or nonparanormal for high dimensional inference just as additive models extend linear models by replacing linear functions with a set of one dimensional smooth functions the nonparanormal extends the normal by transforming the variables by smooth functions we derive a method for estimating the nonparanormal study the method s theoretical properties and show that it works well in many examples

on the solvability of the general feedback control problem a new necessary and sufficient condition for the solvability of the general feedback control problem in linear systems is presented its advantage lies in the fact that it allows the complexity of the feedback controller to be incorporated in the theory of feedback design

a note on the connection and equivalence of three sparse linear discriminant analysis methods in this article we reveal the connection between and equivalence of three sparse linear discriminant analysis methods the l1 fisher s discriminant analysis proposed by wu et al in 2008 the sparse optimal scoring proposed by clemmensen et al in 2011 and the direct sparse discriminant analysis dsda proposed by mai et al in 2012 it is shown that for any sequence of penalization parameters the normalized solutions of dsda equal the normalized solutions of the other two methods at different penalization parameters a prostate cancer dataset is used to demonstrate the theory

mwpcr multiscale weighted principal component regression for high dimensional prediction abstractwe propose a multiscale weighted principal component regression mwpcr framework for the use of high dimensional features with strong spatial features e g smoothness and correlation to predict an outcome variable such as disease status this development is motivated by identifying imaging biomarkers that could potentially aid detection diagnosis assessment of prognosis prediction of response to treatment and monitoring of disease status among many others the mwpcr can be regarded as a novel integration of principal components analysis pca kernel methods and regression models in mwpcr we introduce various weight matrices to prewhitten high dimensional feature vectors perform matrix decomposition for both dimension reduction and feature extraction and build a prediction model by using the extracted features examples of such weight matrices include an importance score weight matrix for the selection of individual features at each location and a spatial weight matrix for the incor