a simple two sample test in high dimensions based on l2 norm testing the equality of two means is a fundamental inference problem for high dimensional data the hotelling s t2 test either performs poorly or becomes inapplicable several modifications have b

high dimensional classification via nonparametric empirical bayes and maximum likelihood inference we propose new nonparametric empirical bayes methods for high dimensional classification our classifiers are designed to approximate the bayes classifier in a hypothesized hierarchical model where the prior distributions for the model parameters are estimated nonparametrically from the training data as is common with nonparametric empirical bayes the proposed classifiers are effective in high dimensional settings even when the underlying model parameters are in fact nonrandom we use nonparametric maximum likelihood estimates of the prior distributions following the elegant approach studied by kiefer wolfowitz in the 1950s however our implementation is based on a recent convex optimization framework for approximating these estimates that is well suited for large scale problems we derive new theoretical results on the accuracy of the approximate estimator which help control the misclassification rate of one of our classifiers we show that our methods outperform several existing methods in simulations and perform well when gene expression microarray data is used to classify cancer patients

ideal spatial adaptation by wavelet shrinkage summary with ideal spatial adaptation an oracle furnishes information about how best to adapt a spatially variable estimator whether piecewise constant piecewise polynomial variable knot spline or variable bandwidth kernel to the unknown function estimation with the aid of an oracle offers dramatic advantages over traditional linear estimation by nonadaptive kernels however it is a priori unclear whether such performance can be obtained by a procedure relying on the data alone we describe a new principle for spatially adaptive estimation selective wavelet reconstruction we show that variable knot spline fits and piecewise polynomial fits when equipped with an oracle to select the knots are not dramatically more powerful than selective wavelet reconstruction with an oracle we develop a practical spatially adaptive method riskshrink which works by shrinkage of empirical wavelet coefficients riskshrink mimics the performance of an oracle for selective wavelet reconstruction as well as it is possible to do so a new inequality in multivariate normal decision theory which we call the oracle inequality shows that attained performance differs from ideal performance by at most a factor of approximately 2 log n where n is the sample size moreover no estimator can give a better guarantee than this within the class of spatially adaptive procedures riskshrink is essentially optimal relying only on the data it comes within a factor log 2 n of the performance of piecewise polynomial and variableknot spline methods equipped with an oracle in contrast it is unknown how or if piecewise polynomial methods could be made to function this well when denied access to an oracle and forced to rely on data alone

shrinking characteristics of precision matrix estimators we propose a framework to shrink a user specified characteristic of a precision matrix estimator that is needed to fit a predictive model estimators in our framework minimize the gaussian negative loglikelihood plus an l 1 penalty on a linear or affine function evaluated at the optimization variable corresponding to the precision matrix we establish convergence rate bounds for these estimators and propose an alternating direction method of multipliers algorithm for their computation our simulation studies show that our estimators can perform better than competitors when they are used to fit predictive models in particular we illustrate cases where our precision matrix estimators perform worse at estimating the population precision matrix but better at prediction

central limit theorems and uniform laws of large numbers for arrays of random fields over the last decades spatial interaction models have been increasingly used in economics however the development of a sufficiently general asymptotic theory for nonlinear spatial models has been hampered by a lack of relevant central limit theorems clts uniform laws of large numbers ullns and pointwise laws of large numbers llns these limit theorems form the essential building blocks towards developing the asymptotic theory of m estimators including maximum likelihood and generalized method of moments estimators the paper establishes a clt ulln and lln for spatial processes or random fields that should be applicable to a broad range of data processes

integrative linear discriminant analysis with guaranteed error rate improvement multiple types of data measured on a common set of subjects arise in many areas numerous empirical studies have found that integrative analysis of such data can result in better statistical performance in terms of prediction and feature selection however the advantages of integrative analysis have mostly been demonstrated empirically in the context of two class classification we propose an integrative linear discriminant analysis method and establish a theoretical guarantee that it achieves a smaller classification error than running linear discriminant analysis on each data type individually we address the issues of outliers and missing values frequently encountered in integrative analysis and illustrate our method through simulations and a neuroimaging study of alzheimer s disease

hac estimation in a spatial framework abstract we suggest a non parametric heteroscedasticity and autocorrelation consistent hac estimator of the variance covariance vc matrix for a vector of sample moments within a spatial context we demonstrate consistency under a set of assumptions that should be satisfied by a wide class of spatial models we allow for more than one measure of distance each of which may be measured with error monte carlo results suggest that our estimator is reasonable in finite samples we then consider a spatial model containing various complexities and demonstrate that our hac estimator can be applied in the context of that model

the incidental parameter problem since 1948 this paper was written to mark the 50th anniversary of neyman and scott s econometrica paper defining the incidental parameter problem it surveys the history both of the paper and of the problem in the statistics and econometrics literature

ultrahigh dimensional multiclass linear discriminant analysis by pairwise sure independence screening this article is concerned with the problem of feature screening for multiclass linear discriminant analysis under ultrahigh dimensional setting we allow the number of classes to be relatively large as a result the total number of relevant features is larger than usual this makes the related classification problem much more challenging than the conventional one where the number of classes is small very often two to solve the problem we propose a novel pairwise sure independence screening method for linear discriminant analysis with an ultrahigh dimensional predictor the proposed procedure is directly applicable to the situation with many classes we further prove that the proposed method is screening consistent simulation studies are conducted to assess the finite sample performance of the new procedure we also demonstrate the proposed methodology via an empirical analysis of a real life example on handwritten chinese character recognition

ridge regression biased estimation for nonorthogonal problems in multiple regression it is shown that parameter estimates based on minimum residual sum of squares have a high probability of being unsatisfactory if not incorrect if the prediction vectors are not orthogonal proposed is an estimation procedure based on adding small positive quantities to the diagonal of x x introduced is the ridge trace a method for showing in two dimensions the effects of nonorthogonality it is then shown how to augment x x to obtain biased estimates with smaller mean square error

weak exponential stability of linear time varying differential behaviors abstract we develop a new approach to exponential stability of linear time varying ltv differential behaviors that is analogous to that in our paper on exponential stability of discrete ltv behaviors h bourles et al 2015 5 stability theory for differential state space systems with smooth coefficients is an important subject in the literature for differential ltv behaviors with arbitrary smooth coefficients there is no reasonable stability theory therefore we restrict the smooth varying coefficients to functions that are defined by means of locally convergent puiseux series all rational functions are of this type we introduce a new kind of behaviors and prove a module behavior duality for these we define a new notion of weak exponential stability w e s of a behavior b and its associated finitely generated f g module m and show that the w e s modules and behaviors are closed under isomorphisms subobjects factor objects and extensions the standard uniform exponential stability of state space equations is not preserved under behavior isomorphisms and unsuitable for a behavioral theory in the main result we assume a nonzero f g torsion module m and its associated autonomous behavior b such a module may be regular or irregular singular according to the galois theory of differential equations if it is nonzero and regular singular it is never w e s for irregular singular m we characterize w e s of most b algebraically and constructively

testing predictor contributions in sufficient dimension reduction we develop tests of the hypothesis of no effect for selected predictors in regression without assuming a model for the conditional distribution of the response given the predictors predictor effects need not be limited to the mean function and smoothing is not required the general approach is based on sufficient dimension reduction the idea being to replace the predictor vector with a lower dimensional version without loss of information on the regression methodology using sliced inverse regression is developed in detail

a compression approach to support vector model selection in this paper we investigate connections between statistical learning theory and data compression on the basis of support vector machine svm model selection inspired by several generalization bounds we construct compression coefficients for svms which measure the amount by which the training labels can be compressed by a code built from the separating hyperplane the main idea is to relate the coding precision to geometrical concepts such as the width of the margin or the shape of the data in the feature space the so derived compression coefficients combine well known quantities such as the radius margin term r2 2 the eigenvalues of the kernel matrix and the number of support vectors to test whether they are useful in practice we ran model selection experiments on benchmark data sets as a result we found that compression coefficients can fairly accurately predict the parameters for which the test error is minimized

sparse linear discriminant analysis by thresholding for high dimensional data in many social economical biological and medical studies one objective is to classify a subject into one of several classes based on a set of variables observed from the subject because the probability distribution of the variables is usually unknown the rule of classification is constructed using a training sample the well known linear discriminant analysis lda works well for the situation where the number of variables used for classification is much smaller than the training sample size because of the advance in technologies modern statistical studies often face classification problems with the number of variables much larger than the sample size and the lda may perform poorly we explore when and why the lda has poor performance and propose a sparse lda that is asymptotically optimal under some sparsity conditions on the unknown parameters for illustration of application we discuss an example of classifying human cancer into two classes of leukemia based on a set of 7 129 genes and a training sample of size 72 a simulation is also conducted to check the performance of the proposed method

a new insight into serre s reduction problem abstract the purpose of this paper is to study the connections existing between serre s reduction of linear functional systems which aims at finding an equivalent system defined by fewer equations and fewer unknowns and the decomposition problem which aims at finding an equivalent system having a diagonal block structure in which one of the diagonal blocks is assumed to be the identity matrix in order to do that we further develop results on serre s reduction problem and on the decomposition problem obtained in 2 3 finally these techniques are used to analyze the decomposability of linear systems of partial differential equations studied in hydrodynamics such as the oseen equations

modified profile likelihoods in models with stratum nuisance parameters it is well known at least through many examples that when there are many nuisance parameters modified profile likelihoods often perform much better than the profile likelihood ordinary asymptotics almost totally fail to deal with this issue for this reason we study asymptotic properties of the profile and modified profile likelihoods in models for stratified data in a two index asymptotics setting this means that both the sample size of the strata m and the dimension of the nuisance parameter q may increase to infinity it is shown that in this asymptotic setting modified profile likelihoods give improvements with respect to the profile likelihood in terms of consistency of estimators and of asymptotic distributional properties in particular the modified profile likelihood based statistics have the usual asymptotic distribution provided that l m o q 1 3 while the analogous condition for the profile likelihood is l m o q 1

asymptotic properties of a robust variance matrix estimator for panel data when t is large abstract i consider the asymptotic properties of a commonly advocated covariance matrix estimator for panel data under asymptotics where the cross section dimension n grows large with the time dimension t fixed the estimator is consistent while allowing essentially arbitrary correlation within each individual however many panel data sets have a non negligible time dimension i extend the usual analysis to cases where n and t go to infinity jointly and where t with n fixed i provide conditions under which t and f statistics based on the covariance matrix estimator provide valid inference and illustrate the properties of the estimator in a simulation study

sign consistency for the linear programming discriminant rule abstract linear discriminant analysis lda is an important conventional model for data classification classical theory shows that lda is bayes consistent for a fixed data dimensionality p and a large training sample size n however in high dimensional settings when p n lda is difficult due to the inconsistent estimation of the covariance matrix and the mean vectors of populations recently a linear programming discriminant lpd rule was proposed for high dimensional linear discriminant analysis based on the sparsity assumption over the discriminant function it is shown that the lpd rule is bayes consistent in high dimensional settings in this paper we further show that the lpd rule is sign consistent under the sparsity assumption such sign consistency ensures the lpd rule to select the optimal discriminative features for high dimensional data classification problems evaluations on both synthetic and real data validate our result on the sign consistency of the lpd rule

estimating subject specific dependent competing risk profile with censored event time observations in a longitudinal study suppose that the primary endpoint is the time to a specific event this response variable however may be censored by an independent censoring variable or by the occurrence of one of several dependent competing events for each study subject a set of baseline covariates is collected the question is how to construct a reliable prediction rule for the future subject s profile of all competing risks of interest at a specific time point for risk benefit decision makings in this paper we propose a two stage procedure to make inferences about such subject specific profiles for the first step we use a parametric model to obtain a univariate risk index score system we then estimate consistently the average competing risks for subjects which have the same parametric index score via a nonparametric function estimation procedure we illustrate this new proposal with the data from a randomized clinical trial for evaluating the efficacy of a treatment for prostate cancer the primary endpoint for this study was the time to prostate cancer death but had two types of dependent competing events one from cardiovascular death and the other from death of other causes

two stage regularized linear discriminant analysis for 2 d data fisher linear discriminant analysis lda involves within class and between class covariance matrices for 2 d data such as images regularized lda rlda can improve lda due to the regularized eigenvalues of the estimated within class matrix however it fails to consider the eigenvectors and the estimated between class matrix to improve these two matrices simultaneously we propose in this paper a new two stage method for 2 d data namely a bidirectional lda blda in the first stage and the rlda in the second stage where both blda and rlda are based on the fisher criterion that tackles correlation blda performs the lda under special separable covariance constraints that incorporate the row and column correlations inherent in 2 d data the main novelty is that we propose a simple but effective statistical test to determine the subspace dimensionality in the first stage as a result the first stage reduces the dimensionality substantially while keeping the significant discriminant information in the data this enables the second stage to perform rlda in a much lower dimensional subspace and thus improves the two estimated matrices simultaneously experiments on a number of 2 d synthetic and real world data sets show that blda rlda outperforms several closely related competitors

integrated likelihood functions for non bayesian inference consider a model with parameter 0 where is the parameter of interest and let l denote the likelihood function one approach to likelihood inference for is to use an integrated likelihood function in which is eliminated from l by integrating with respect to a density function the goal of this paper is to consider the problem of selecting so that the resulting integrated likelihood function is useful for non bayesian likelihood inference the desirable properties of an integrated likelihood function are analyzed and these suggest that should be chosen by finding a nuisance parameter that is unrelated to and then taking the prior density for to be independent of such an unrelated parameter is constructed and the resulting integrated likelihood is shown to be closely related to the modified profile likelihood

tensored polynomial models the theme of the present paper is the study of two different versions of tensor products of functional models and present some applications to various problems related to system theory among those are stability of higher order systems tangent spaces spaces of intertwining maps invariant factors of tensor products of linear transformations and the solvability of sylvester equations

the second order bias and mean squared error of nonlinear estimators abstract despite the now widespread use of nonlinear estimators their finite sample properties have received very little attention in either the statistics or econometrics literature we partially redress this problem by deriving and examining the second order bias and mean squared error of a fairly wide class of nonlinear estimators which includes nonlinear least squares maximum likelihood and many generalized method of moments estimators as special cases a number of examples are provided the results from a monte carlo exercise demonstrate how the results can be applied for improved inferences

sliced inverse regression for dimension reduction abstract modern advances in computing power have greatly widened scientists scope in gathering and investigating information from many variables information which might have been ignored in the past yet to effectively scan a large pool of variables is not an easy task although our ability to interact with data has been much enhanced by recent innovations in dynamic graphics in this article we propose a novel data analytic tool sliced inverse regression sir for reducing the dimension of the input variable x without going through any parametric or nonparametric model fitting process this method explores the simplicity of the inverse view of regression that is instead of regressing the univariate output variable y against the multivariate x we regress x against y forward regression and inverse regression are connected by a theorem that motivates this method the theoretical properties of sir are investigated under a model of the form y f 1 x k x e where the k s are the unknown

asymptotically unbiased inference for a dynamic panel model with fixed effects when both n and t are large we consider a dynamic panel ar 1 model with fixed effects when both n and t are large under the t fixed n large asymptotic approximation the ordinary least squares ols or gaussian maximum likelihood estimator mle is known to be inconsistent due to the well known incidental parameter problem we consider an alternative asymptotic approximation where n and t grow at the same rate it is shown that although ols or the mle is asymptotically biased a relatively simple fix to ols or the mle results in an asymptotically unbiased estimator under the assumption of gaussian innovations the bias corrected mle is shown to be asymptotically efficient by a hajek type convolution theorem

bias reduction in nonlinear and dynamic panels in the presence of cross section dependence abstract fixed effects estimation of nonlinear dynamic panel models is subject to the incidental parameter issue leading to a biased asymptotic distribution while this problem has been studied extensively in the literature a general analysis allowing for both serial and cross sectional dependence is missing in this paper we investigate the large n t theory of the profile and integrated likelihood estimators allowing for dependence across both dimensions we show that under stronger dependence types the asymptotic bias disappears but a o p 1 t small sample bias remains we provide bias correction and inference methods and also obtain primitive conditions for asymptotic normality under various dependence settings

a behavioral approach to control of distributed systems this paper develops a theory of control for distributed systems i e those defined by systems of constant coefficient partial differential operators via the behavioral approach of willems the study here is algebraic in the sense that it relates behaviors of distributed systems to submodules of free modules over the polynomial ring in several indeterminates as in the lumped case behaviors of distributed arma systems can be reduced to ar behaviors this paper first studies the notion of ar controllable distributed systems following the corresponding definition for lumped systems due to willems it shows that as in the lumped case the class of controllable ar systems is precisely the class of ma systems it then shows that controllable 2 d distributed systems are necessarily given by free submodules whereas this is not the case for n d distributed systems n ge 3 this therefore points out an important difference between these two cases this paper then defines two notions of autonomous distributed systems which mimic different properties of lumped autonomous systems control is the process of restricting a behavior to a specific desirable autonomous subbehavior a notion of stability generalizing bounded input bounded output stability of lumped systems is proposed and the pole placement problem is defined for distributed systems this paper then solves this problem for a class of distributed behaviors

estimating dynamic panel data discrete choice models with fixed effects abstract this paper considers the estimation of dynamic binary choice panel data models with fixed effects it is shown that the modified maximum likelihood estimator mmle used in this paper reduces the order of the bias in the maximum likelihood estimator from o t 1 to o t 2 without increasing the asymptotic variance no orthogonal reparametrization is needed monte carlo simulations are used to evaluate its performance in finite samples where t is not large in probit and logit models containing lags of the endogenous variable and exogenous variables the estimator is found to have a small bias in a panel with eight periods a distinctive advantage of the mmle is its general applicability estimation and relevance of different policy parameters of interest in this kind of models are also addressed

a note on the connection and equivalence of three sparse linear discriminant analysis methods in this article we reveal the connection between and equivalence of three sparse linear discriminant analysis methods the l1 fisher s discriminant analysis proposed by wu et al in 2008 the sparse optimal scoring proposed by clemmensen et al in 2011 and the direct sparse discriminant analysis dsda proposed by mai et al in 2012 it is shown that for any sequence of penalization parameters the normalized solutions of dsda equal the normalized solutions of the other two methods at different penalization parameters a prostate cancer dataset is used to demonstrate the theory

algorithms for the quillen suslin theorem abstract we give an algorithm for computing a free basis of a projective c x1 xn module which is presented as the image kernel or cokernel of a polynomial matrix our method can be implemented using grobner bases and it provides an elementary constructive proof of the quillen suslin theorem

factoring and decomposing a class of linear functional systems abstract within a constructive homological algebra approach we study the factorization and decomposition problems for a class of linear functional determined over determined under determined systems using the concept of ore algebras of functional operators e g ordinary partial differential operators shift operators time delay operators we first concentrate on the computation of morphisms from a finitely presented left module m over an ore algebra to another one m where m resp m is a module intrinsically associated with the linear functional system ry 0 resp r z 0 these morphisms define applications sending solutions of the system r z 0 to solutions of r y 0 we explicitly characterize the kernel image cokernel and coimage of a general morphism we then show that the existence of a non injective endomorphism of the module m is equivalent to the existence of a non trivial factorization r r 2 r 1 of the system matrix r the corresponding system can then be integrated in cascade under certain conditions we also show that the system ry 0 is equivalent to a system r z 0 where r is a block triangular matrix of the same size as r we show that the existence of idempotents of the endomorphism ring of the module m allows us to reduce the integration of the system ry 0 to the integration of two independent systems r 1 y 1 0 and r 2 y 2 0 furthermore we prove that under certain conditions idempotents provide decompositions of the system ry 0 i e they allow us to compute an equivalent system r z 0 where r is a block diagonal matrix of the same size as r applications of these results in mathematical physics and control theory are given finally the different algorithms of the paper are implemented in the maple package m orphisms based on the library oremodules

analysis of additive risk model with high dimensional covariates using partial least squares in this paper we construct a partial additive regression par model to predict the survival times of cancer patients based on microarray gene expression data with right censoring the area under time dependent receiver operating characteristic curve is used as a model evaluation criterion we conduct a simulation study to compare the proposed method with other methods i e partial cox regression and supervised principal component analysis two data sets of breast cancer and diffuse large b cell lymphoma are analyzed to illustrate our procedure the outcome indicates great predictive performance on both dimension reduction and predictive performance of the proposed par model

generalized quadratic discriminant analysis in linear discriminant analysis the assumption of equality of the dispersion matrices of different classes leads to a classification rule based on minimum mahalanobis distance from the class centres however without this assumption the resulting quadratic discriminant classifier involves in addition to the mahalanobis distances the ratio of the determinants of the dispersion matrices as a factor in fact it has been observed that for discriminating between populations with underlying elliptically symmetric distributions such classifiers also incorporate similar factors apart from the mahalanobis distances in this paper a nonparametric classification technique which generalizes discriminant analysis has been proposed the method of cross validation is used to make the technique adaptive to a given dataset an extensive simulation study is presented to illustrate the potential of the method finally through implementation on a number of real life data sets it has been demonstrated that the proposed generalized quadratic discriminant analysis gqda compares very favourably with other nonparametric methods and is computationally cost effective highlightsa nonparametric classification technique has been proposed it generalizes fisher s discriminant analysis cross validation is used to make the technique adaptive to given data found empirically to compare very favourably with other nonparametric methods computationally cost effective

individual and time effects in nonlinear panel models with large n t fixed eects estimators of nonlinear panel data models can be severely biased because of the well known incidental parameter problem we develop analytical and jackknife bias corrections for nonlinear models with both individual and time eects under asymptotic sequences where the time dimension t grows with the cross sectional dimension n the time eects introduce additional incidental parameter bias as the existing bias corrections apply to models with only individual eects we derive the appropriate corrections the basis for the corrections are general asymptotic expansions of xed eects estimators with incidental parameters in multiple dimensions we apply the expansions to m estimators with concave objective functions in parameters for panel models with additive individual and time eects these estimators cover xed eects estimators of the most popular limited dependent variable models such as logit probit ordered probit tobit and poisson models our analysis therefore extends the use of large t bias adjustments to an important class of models we also develop bias corrections for functions of the data parameters and individual and time eects including average partial eects in this case the incidental parameter bias is asymptotically of second order in general but the corrections improve nite sample properties

coda high dimensional copula discriminant analysis we propose a high dimensional classification method named the copula discriminant analysis coda the coda generalizes the normal based linear discriminant analysis to the larger gaussian copula models or the nonparanormal as proposed by liu et al 2009 to simultaneously achieve estimation efficiency and robustness the nonparametric rank based methods including the spearman s rho and kendall s tau are exploited in estimating the covariance matrix in high dimensional settings we prove that the sparsity pattern of the discriminant features can be consistently recovered with the parametric rate and the expected misclassification error is consistent to the bayes risk our theory is backed up by careful numerical experiments which show that the extra flexibility gained by the coda method incurs little efficiency loss even when the data are truly gaussian these results suggest that the coda method can be an alternative choice besides the normal based high dimensional linear discriminant analysis

volatility forecast comparison using imperfect volatility proxies the use of a conditionally unbiased but imperfect volatility proxy can lead to undesirable outcomes in standard methods for comparing conditional variance forecasts we motivate our study with analytical results on the distortions caused by some widely used loss functions when used with standard volatility proxies such as squared returns the intra daily range or realised volatility we then derive necessary and sufficient conditions on the functional form of the loss function for the ranking of competing volatility forecasts to be robust to the presence of noise in the volatility proxy and derive some useful special cases of this class of robust loss functions the methods are illustrated with an application to the volatility of returns on ibm over the period 1993 to 2003

the lasso method for variable selection in the cox model summary i propose a new method for variable selection and shrinkage in cox s proportional hazards model my proposal minimizes the log partial likelihood subject to the sum of the absolute values of the parameters being bounded by a constant because of the nature of this constraint it shrinks coe cients and produces some coe cients that are exactly zero as a result it reduces the estimation variance while providing an interpretable final model the method is a variation of the lasso proposal of tibshirani designed for the linear regression context simulations indicate that the lasso can be more accurate than stepwise selection in this setting

robust priors in nonlinear panel data models many approaches to estimation of panel models are based on an average or integrated likelihood that assigns weights to different values of the individual effects fixed effects random effects and bayesian approaches all fall in this category we provide a characterization of the class of weights or priors that produce estimators that are first order unbiased we show that such bias reducing weights must depend on the data unless an orthogonal reparameterization or an essentially equivalent condition is available two intuitively appealing weighting schemes are discussed we argue that asymptotically valid confidence intervals can be read from the posterior distribution of the common parameters when n and t grow at the same rate finally we show that random effects estimators are not bias reducing in general and discuss important exceptions three examples and some monte carlo experiments illustrate the results

optimal classification in sparse gaussian graphic model consider a two class classification problem where the number of features is much larger than the sample size the features are masked by gaussian noise with mean zero and covariance matrix sigma where the precision matrix omega sigma 1 is unknown but is presumably sparse the useful features also unknown are sparse and each contributes weakly i e rare and weak to the classification decision by obtaining a reasonably good estimate of omega we formulate the setting as a linear regression model we propose a two stage classification method where we first select features by the method of innovated thresholding it and then use the retained features and fisher s lda for classification in this approach a crucial problem is how to set the threshold of it we approach this problem by adapting the recent innovation of higher criticism thresholding hct we find that when useful features are rare and weak the limiting behavior of hct is essentially just as good as the limiting behavior of ideal threshold the threshold one would choose if the underlying distribution of the signals is known if only somewhat surprisingly when omega is sufficiently sparse its off diagonal coordinates usually do not have a major influence over the classification decision compared to recent work in the case where omega is the identity matrix proc natl acad sci usa 105 2008 14790 14795 philos trans r soc lond ser a math phys eng sci 367 2009 4449 4470 the current setting is much more general which needs a new approach and much more sophisticated analysis one key component of the analysis is the intimate relationship between hct and fisher s separation another key component is the tight large deviation bounds for empirical processes for data with unconventional correlation structures where graph theory on vertex coloring plays an important role

a statistical view of some chemometrics regression tools chemometrics is a field of chemistry that studies the application of statistical methods to chemical data analysis in addition to borrowing many techniques from the statistics and engineering literatures chemometrics itself has given rise to several new data analytical methods this article examines two methods commonly used in chemometrics for predictive modeling partial least squares and principal components regression from a statistical perspective the goal is to try to understand their apparent successes and in what situations they can be expected to work well and to compare them with other statistical methods intended for those situations these methods include ordinary least squares variable subset selection and ridge regression

multiclass linear discriminant analysis with ultrahigh dimensional features within the framework of fisher s discriminant analysis we propose a multiclass classification method which embeds variable screening for ultrahigh dimensional predictors leveraging interfeature correlations we show that the proposed linear classifier recovers informative features with probability tending to one and can asymptotically achieve a zero misclassification rate we evaluate the finite sample performance of the method via extensive simulations and use this method to classify posttransplantation rejection types based on patients gene expressions

the effective calculation of all assignable polynomials to a single input system over a bezout domain the set of all feedback assignable polynomials to a given single input system over a bezout domain is obtained the general case is reduced to the case of weakly reachable systems for these systems a canonical form for the algebraic equivalence is obtained algorithms and examples over a euclidean domain and in particular over the ring z of rational integers are also given the problem of deciding and feedback assignment of a given polynomial to a given non reachable n dimensional linear system is solved effectively up to a cost of o n4 arithmetic operations

inference with dependent data using cluster covariance estimators this paper presents a novel way to conduct inference using dependent data in time series spatial and panel data applications our method involves constructing t and wald statistics utilizing a cluster covariance matrix estimator cce we then use an approximation that takes the number of clusters groups as xed and the number of observations per group to be large and calcu late limiting distributions of the t and wald statistics this approximation is analogous to xed b asymptotics of kiefer and vogelsang 2002 2005 kv for heteroskedasticity and autocorrelation consistent inference but in our case yields standard t and f distributions where the number of groups essentially plays the role of sample size we provide simulation evidence that demonstrates our procedure outperforms conventional inference procedures and performs comparably to kv

optimization approaches to supervised classification abstract the supervised classification problem one of the oldest and most recurrent problems in applied data analysis has always been analyzed from many different perspectives when the emphasis is placed on its overall goal of developing classification rules with minimal classification cost supervised classification can be understood as an optimization problem on the other hand when the focus is in modeling the uncertainty involved in the classification of future unknown entities it can be formulated as a statistical problem other perspectives that pay particular attention to pattern recognition and machine learning aspects of supervised classification have also a long history that has lead to influential insights and different methodologies in this review two approaches to supervised classification strongly related to optimization theory will be discussed and compared in particular we will review methodologies based on mathematical programming models that optimize observable criteria linked to the true objective of misclassification error or cost minimization and approaches derived from the minimization of known bounds on the true misclassification error the former approach is known as the mathematical programming approach to supervised classification while the latter is in the origin of the well known classification support vector machines throughout the review two group as well as general multi group problems will be considered and the review will conclude with a discussion of the most promising research directions in this area

on behavior homomorphisms and system equivalence in this paper we introduce the concept of a behavior homomorphisms and isomorphisms and use it to present a uni ed approach to the study of equivalence of di erent behavior representations concentrating on a special representation we call normalized arma narma representation c 2001 elsevier science b v all rights reserved

covariate adjusted tensor classification in high dimensions in contemporary scientific research it is often of great interest to predict a categorical response based on a high dimensional tensor i e multi dimensional array and additional covariates mo

simultaneous sparse estimation of canonical vectors in the p n setting abstractthis article considers the problem of sparse estimation of canonical vectors in linear discriminant analysis when p n several methods have been proposed in the literature that estimate one canonical vector in the two group case however g 1 canonical vectors can be considered if the number of groups is g in the multi group context it is common to estimate canonical vectors in a sequential fashion moreover separate prior estimation of the covariance structure is often required we propose a novel methodology for direct estimation of canonical vectors in contrast to existing techniques the proposed method estimates all canonical vectors at once performs variable selection across all the vectors and comes with theoretical guarantees on the variable selection and classification consistency first we highlight the fact that in the n p setting the canonical vectors can be expressed in a closed form up to an orthogonal transformation secondly we propose an extension of this form to the

jackknife and analytical bias reduction for nonlinear panel models fixed effects estimators of panel models can be severely biased because of the well known incidental parameters problem we show that this bias can be reduced by using a panel jackknife or an analytical bias correction motivated by large t we give bias corrections for averages over the fixed effects as well as model parameters we find large bias reductions from using these approaches in examples we consider asymptotics where t grows with n as an approximation to the properties of the estimators in econometric applications we show that if t grows at the same rate as n the fixed effects estimator is asymptotically biased so that asymptotic confidence intervals are incorrect but that they are correct for the panel jackknife we show t growing faster than n 1 3 suffices for correctness of the analytic correction a property we also conjecture for the jackknife

high dimensional classification using features annealed independence rules classification using high dimensional features arises frequently in many contemporary statistical studies such as tumor classification using microarray or other high throughput data the impact of dimensionality on classifications is largely poorly understood in a seminal paper bickel and levina 2004 show that the fisher discriminant performs poorly due to diverging spectra and they propose to use the independence rule to overcome the problem we first demonstrate that even for the independence classification rule classification using all the features can be as bad as the random guessing due to noise accumulation in estimating population centroids in high dimensional feature space in fact we demonstrate further that almost all linear discriminants can perform as bad as the random guessing thus it is paramountly important to select a subset of important features for high dimensional classification resulting in features annealed independence rules fair the conditions under which all the important features can be selected by the two sample t statistic are established the choice of the optimal number of features or equivalently the threshold value of the test statistics are proposed based on an upper bound of the classification error simulation studies and real data analysis support our theoretical results and demonstrate convincingly the advantage of our new classification procedure

one step sparse estimates in nonconcave penalized likelihood models fan and li propose a family of variable selection methods via penalized likelihood using concave penalty functions the nonconcave penalized likelihood estimators enjoy the oracle properties but maximizing the penalized likelihood function is computationally challenging because the objective function is nondifferentiable and nonconcave in this article we propose a new unified algorithm based on the local linear approximation lla for maximizing the penalized likelihood for a broad class of concave penalty functions convergence and other theoretical properties of the lla algorithm are established a distinguished feature of the lla algorithm is that at each lla step the lla estimator can naturally adopt a sparse representation thus we suggest using the one step lla estimator from the lla algorithm as the final estimates statistically we show that if the regularization parameter is appropriately chosen the one step lla estimates enjoy the oracle properties with good initial estimators computationally the one step lla estimation methods dramatically reduce the computational cost in maximizing the nonconcave penalized likelihood we conduct some monte carlo simulation to assess the finite sample performance of the one step sparse estimation methods the results are very encouraging

bias in dynamic panel estimation with fixed effects incidental trends and cross section dependence abstract explicit asymptotic bias formulae are given for dynamic panel regression estimators as the cross section sample size n the results extend earlier work by nickell 1981 biases in dynamic models with fixed effects econometrica 49 1417 1426 and later authors in several directions that are relevant for practical work including models with unit roots deterministic trends predetermined and exogenous regressors and errors that may be cross sectionally dependent the asymptotic bias is found to be so large when incidental linear trends are fitted and the time series sample size is small that it changes the sign of the autoregressive coefficient another finding of interest is that when there is cross section error dependence the probability limit of the dynamic panel regression estimator is a random variable rather than a constant which helps to explain the substantial variability observed in dynamic panel estimates when there is cross section dependence even in situations where n is very large some proposals for bias correction are suggested and finite sample performance is analyzed in simulations

covariance enhanced discriminant analysis linear discriminant analysis has been widely used to characterize or separate multiple classes via linear combinations of features however the high dimensionality of features from modern biological experiments defies traditional discriminant analysis techniques possible interfeature correlations present additional challenges and are often underused in modelling in this paper by incorporating possible interfeature correlations we propose a covariance enhanced discriminant analysis method that simultaneously and consistently selects informative features and identifies the corresponding discriminable classes under mild regularity conditions we show that the method can achieve consistent parameter estimation and model selection and can attain an asymptotically optimal misclassification rate extensive simulations have verified the utility of the method which we apply to a renal transplantation trial

piecewise linear regularized solution paths we consider the generic regularized optimization problem argmin l y x j efron hastie johnstone and tibshirani ann statist 32 2004 407 499 have shown that for the lasso that is if l is squared error loss and j 1 is the l 1 norm of the optimal coefficient path is piecewise linear that is is piecewise constant we derive a general characterization of the properties of loss l penalty j pairs which give piecewise linear coefficient paths such pairs allow for efficient generation of the full regularized coefficient paths we investigate the nature of efficient path following algorithms which arise we use our results to suggest robust versions of the lasso for regression and classification and to develop new efficient algorithms for existing problems in the literature including mammen and van de geer s locally adaptive regression splines

stringing high dimensional data for functional analysis we propose stringing a class of methods where one views high dimensional observations as functional data stringing takes advantage of the high dimension by representing such data as discretized and noisy observations that originate from a hidden smooth stochastic process assuming that the observations result from scrambling the original ordering of the observations of the process stringing reorders the components of the high dimensional vectors followed by transforming the high dimensional vector observations into functional data established techniques from functional data analysis can be applied for further statistical analysis once an underlying stochastic process and the corresponding random trajectory for each subject have been identified stringing of high dimensional data is implemented with distance based metric multidimensional scaling mapping high dimensional data to locations on a real interval such that predictors that are close in a suitable sample metric also are located close to each

sparsifying the fisher linear discriminant by rotation many high dimensional classification techniques have been proposed in the literature based on sparse linear discriminant analysis lda to efficiently use them sparsity of linear classifiers is a prerequisite however this might not be readily available in many applications and rotations of data are required to create the needed sparsity in this paper we propose a family of rotations to create the required sparsity the basic idea is to use the principal components of the sample covariance matrix of the pooled samples and its variants to rotate the data first and to then apply an existing high dimensional classifier this rotate and solve procedure can be combined with any existing classifiers and is robust against the sparsity level of the true model we show that these rotations do create the sparsity needed for high dimensional classifications and provide theoretical understanding why such a rotation works empirically the effectiveness of the proposed method is demonstrated by a number of simulated and real data examples and the improvements of our method over some popular high dimensional classification rules are clearly shown

forecasting economic time series using targeted predictors abstract this paper studies two refinements to the method of factor forecasting first we consider the method of quadratic principal components that allows the link function between the predictors and the factors to be non linear second the factors used in the forecasting equation are estimated in a way to take into account that the goal is to forecast a specific series this is accomplished by applying the method of principal components to targeted predictors selected using hard and soft thresholding rules our three main findings can be summarized as follows first we find improvements at all forecast horizons over the current diffusion index forecasts by estimating the factors using fewer but informative predictors allowing for non linearity often leads to additional gains second forecasting the volatile one month ahead inflation warrants a high degree of targeting to screen out the noisy predictors a handful of variables notably relating to housing starts and interest rates are found to have systematic predictive power for inflation at all horizons third the targeted predictors selected by both soft and hard thresholding changes with the forecast horizon and the sample period holding the set of predictors fixed as is the current practice of factor forecasting is unnecessarily restrictive

a note on pseudolikelihood constructed from marginal densities for likelihood based inference involving distributions in which high dimensional dependencies are present it may be useful to use approximate likelihoods based for example on the univariate or bivariate marginal distributions the asymptotic properties of formal maximum likelihood estimators in such cases are outlined in particular applications in which only a single q x i vector of observations is observed are examined conditions under which consistent estimators of parameters result from the approximate likelihood using only pairwise joint distributions are studied some examples are analysed in detail

optimal sufficient dimension reduction in regressions with categorical predictors though partial sliced inverse regression partial sir chiaromonte et al 2002 sufficient dimension reduction in regressions with categorical predictors ann statist 30 475 497 extended the scope of sufficient dimension reduction to regressions with both continuous and categorical predictors its requirement of homogeneous predictor covariances across the subpopulations restricts its application in practice when this condition fails partial sir may provide misleading results in this article we propose a new estimation method via a minimum discrepancy approach without this restriction our method is optimal in terms of asymptotic efficiency and its test statistic for testing the dimension of the partial central subspace always has an asymptotic chi squared distribution it also gives us the ability to test predictor effects an asymptotic chi squared test of the conditional independence hypothesis that the response is independent of a selected subset of the continuous predictors given the remaining predictors is obtained

bootstrap refinements for qml estimators of the garch 1 1 parameters a b s t r a c t this paper reconsiders a block bootstrap procedure for quasi maximum likelihood estimation of garch models based on the resampling of the likelihood function as proposed by goncalves and white 2004 maximum likelihood and the bootstrap for nonlinear dynamic models journal of econometrics 119 199 219 first we provide necessary conditions and sufficient conditions in terms of moments of the innovation process for the existence of the edgeworth expansion of the garch 1 1 estimator up to the k th term second we provide sufficient conditions for higher order refinements for equally tailed and symmetric test statistics in particular the bootstrap estimator based on resampling the likelihood has the same higher order improvements in terms of error in the rejection probabilities as those in andrews 2002 higher order improvements of a computationally attractive k step bootstrap for extremum estimators econometrica 70 119 162

fully exponential laplace approximations to expectations and variances of nonpositive functions abstract tierney and kadane 1986 presented a simple second order approximation for posterior expectations of positive functions they used laplace s method for asymptotic evaluation of integrals in which the integrand is written as f exp nh and the function h is approximated by a quadratic the form in which they applied laplace s method however was fully exponential the integrand was written instead as exp nh log f this allowed first order approximations to be used in the numerator and denominator of a ratio of integrals to produce a second order expansion for the ratio other second order expansions hartigan 1965 johnson 1970 lindley 1961 1980 mosteller and wallace 1964 require computation of more derivatives of the log likelihood function in this article we extend the fully exponential method to apply to expectations and variances of nonpositive functions to obtain a second order approximation to an expectation e g we use the fully exponential method to approximate

optimal feature selection in high dimensional discriminant analysis we consider the high dimensional discriminant analysis problem for this problem different methods have been proposed and justified by establishing exact convergence rates for the classification risk as well as the ell 2 convergence results to the discriminative rule however sharp theoretical analysis for the variable selection performance of these procedures have not been established even though model interpretation is of fundamental importance in scientific data analysis this paper bridges the gap by providing sharp sufficient conditions for consistent variable selection using the sparse discriminant analysis through careful analysis we establish rates of convergence that are significantly faster than the best known results and admit an optimal scaling of the sample size n dimensionality p and sparsity level s in the high dimensional setting sufficient conditions are complemented by the necessary information theoretic limits on the variable selection problem in the context of high dimensional discriminant analysis exploiting a numerical equivalence result our method also establish the optimal results for the road estimator and the sparse optimal scoring estimator furthermore we analyze an exhaustive search procedure whose performance serves as a benchmark and show that it is variable selection consistent under weaker conditions extensive simulations demonstrating the sharpness of the bounds are also provided

parameter orthogonality and approximate conditional inference on propose une statistique du rapport de vraisemblance construite a partir de la distribution conditionnelle des observations etant donne les estimateurs du maximum de vraisemblance pour les parametres de nuisance

profile likelihood and conditionally parametric models in this paper we outline a general approach to estimating the parametric component of a semiparametric model for the case of a scalar parametric component the method is based on the idea of first estimating a one dimensional subproblem of the original problem that is least favorable in the sense of stein the likelihood function for the scalar parameter along this estimated subproblem may be viewed as a generalization of the profile likelihood for the problem the scalar parameter is then estimated by maximizing this generalized profile likelihood this method of estimation is applied to a particular class of semiparametric models where it is shown that the resulting estimator is asymptotically efficient

inference for density families using functional principal component analysis we consider t 1 t samples of iid observations x1t xntt from unknown population densities ft to characterize differences and similarities of ft we assume their expansions into the first l principal components from the given observations xit we study inference on the components and on their required number l a detailed asymptotic theory is presented our method is applied in the analysis of yearly cross sectional samples of british households interpretation of the estimated principal components and their scores provides new insights into the evolution and interplay of household income and age distributions from 1968 1988 from estimating their required numbersl we draw conclusions on the dimensionality of mixture models for describing the densities

econometric analysis of realized volatility and its use in estimating stochastic volatility models summary the availability of intraday data on the prices of speculative assets means that we can use quadratic variation like measures of activity in financial markets called realized volatility to study the stochastic properties of returns here under the assumption of a rather general stochastic volatility model we derive the moments and the asymptotic distribution of the realized volatility error the difference between realized volatility and the discretized integrated volatility which we call actual volatility these properties can be used to allow us to estimate the parameters of stochastic volatility models without recourse to the use of simulation intensive methods

gmm estimation with cross sectional dependence this paper presents a spatial model of dependence among agents using a metric of economic distance measurements of this economic distance provide cross sectional data with a structure similar to that provided by the time index in time series data generalized method of moments estimators using such dependent data are shown to be consistent and asymptotically normal this paper presents a class of non parametric positive semi definite covariance matrix estimators that allow for general forms of dependence characterized by economic distance these covariance matrix estimators are shown to remain consistent when economic distances are not precisely observed

panel data models with interactive fixed effects this paper considers large n and large t panel data models with unobservable multiple interactive eects these models are useful for both micro and macro econometric modelings in earnings studies for example workers motivation persistence and diligence combined to influence the earnings in addition to the usual argument of innate ability in macroeconomics the interactive eects represent unobservable common shocks and their heterogeneous responses over cross sections since the interactive eects are allowed to be correlated with the regressors they are treated as fixed eects parameters to be estimated along with the common slope coecients the model is estimated by the least squares method which provides the interactive eect s counterpart of the within estimator we first consider model identification and then derive the rate of convergence and the limiting distribution of the interactive eect s estimator of the common slope coecients the estimator is shown to be p nt consistent this rate is valid even in the presence of correlations and heteroskedasticities in both dimensions a striking contrast with fixed t framework in which serial correlation and heteroskedasticity imply unidentification the asymptotic distribution is not necessarily centered at zero biased corrected estimators are derived we also derive the constrained estimator and its limiting distribution imposing additivity coupled with interactive eects the problem of testing additive versus interactive eects is also studied we also derive identification conditions for models with grand mean time invariant regressors and common regressors it is shown that there exists a set of necessary and sucient identification conditions for those models given identification the rate of convergence and limiting results continue to hold

a jordan holder theorem for differential algebraic groups abstract we show that a differential algebraic group can be filtered by a finite subnormal series of differential algebraic groups such that successive quotients are almost simple that is have no normal subgroups of the same type we give a uniqueness result prove several properties of almost simple groups and in the ordinary differential case classify almost simple linear differential algebraic groups

slicing regression a link free regression method slicing regression a link free regression m e t h o d author s naihua duan and k e r c h a u l i s o u r c e the annals of statistics v o l 19 n o 2 t u n 1991 p p 5 0 5 5 3 0 p u b l i s h e d b y institute of mathematical statistics s t a b l e u r l http www jstor org stable 2242072 accessed 16 05 2011 your use of the jstor archive indicates your acceptance of jstor s terms and conditions of use available at http www jstor org page info about policies terms jsp jstor s terms and conditions of use provides in part that unless you have obtained prior permission you may not download an entire issue of a journal or multiple copies of articles and you may use content in the jstor archive only for your personal non commercial use please contact the publisher regarding any further use of this work publisher contact information may be obtained at http www jstor org action showpublisher publishercode ims each copy of any part of a jstor transmission must contain the same copyright notice that appears on the screen or printed page of such transmission jstor is a not for profit service that helps scholars researchers and students discover use and build upon a wide range of content in a trusted digital archive we use information technology and tools to increase productivity and facilitate new forms of scholarship for more information about jstor please contact support jstor org http www jstor org

matrix linear discriminant analysis we propose a novel linear discriminant analysis lda approach for the classification of high dimensional matrix valued data that commonly arises from imaging studies motivated by the equivalence

prediction by supervised principal components in regression problems where the number of predictors greatly exceeds the number of observations conventional regression techniques may produce unsatisfactory results we describe a technique called supervised principal components that can be applied to this type of problem supervised principal components is similar to conventional principal components analysis except that it uses a subset of the predictors selected based on their association with the outcome supervised principal components can be applied to regression and generalized regression problems such as survival analysis it compares favorably to other techniques for this type of problem and can also account for the effects of other covariates and help identify which predictor variables are most important we also provide asymptotic consistency results to help support our empirical findings these methods could become important tools for dna microarray data where they may be used to more accurately diagnose and treat cancer

localization and parametrization of linear multidimensional control systems abstract we study the link existing between the parametrization of differential operators by potential like arbitrary functions and the localization of differential modules while applying these results to the parametrization of linear multidimensional control systems we show that the localization of differential modules is a natural way to generalize some well known results on transfer matrix classically obtained by using laplace transform to time varying ordinary differential control systems and to partial differential control systems with variable coefficients in particular we show that the parametrizations obtained by localization are simpler than those obtained by formal duality but are worse in the sense of palamodov kashiwara s classification of differential modules

a road to classification in high dimensional space the regularized optimal affine discriminant for high dimensional classification it is well known that naively performing the fisher discriminant rule leads to poor results due to diverging spectra and noise accumulation therefore researchers proposed independence rules to circumvent the diverging spectra and sparse independence rules to mitigate the issue of noise accumulation however in biological applications there are often a group of correlated genes responsible for clinical outcomes and the use of the covariance information can significantly reduce misclassification rates in theory the extent of such error rate reductions is unveiled by comparing the misclassification rates of the fisher discriminant rule and the independence rule to materialize the gain based on finite samples a regularized optimal affine discriminant road is proposed road selects an increasing number of features as the regularization relaxes further benefits can be achieved when a screening method is employed to narrow the feature pool before hitting the road an efficient constrained coordinate descent algorithm ccd is also developed to solve the associated optimization problems sampling properties of oracle type are established simulation studies and real data analysis support our theoretical results and demonstrate the advantages of the new classification procedure under a variety of correlation structures a delicate result on continuous piecewise linear solution path for the road optimization problem at the population level justifies the linear interpolation of the ccd algorithm

exponentially stable linear time varying discrete behaviors we study implicit systems of linear time varying ltv difference equations with rational coefficients of arbitrary order and their solution spaces called discrete ltv behaviors the signals are sequences i e functions from the discrete time set of natural numbers into the complex numbers the difference field of rational functions with complex coefficients gives rise to a noncommutative skew polynomial algebra of difference operators that act on sequences via left shift for this paper it is decisive that the ring of operators is a principal ideal domain and that nonzero rational functions have only finitely many poles and zeros and grow at most polynomially due to the poles a new definition of behaviors is required for the latter we derive the important categorical duality between finitely generated left modules over the ring of operators and behaviors the duality theorem implies the usual consequences for willems elimination the fundamental principle input output decompositions and controlla

fixed effects estimation of structural parameters and marginal effects in panel probit models fixed effects estimators of nonlinear panel models can be severely biased due to the incidental parameters problem in this paper i characterize the leading term of a large t expansion of the bias of the mle and estimators of average marginal effects in parametric fixed effects panel binary choice models for probit index coefficients the former term is proportional to the true value of the coefficients being estimated this result allows me to derive a lower bound for the bias of the mle i then show that the resulting fixed effects estimates of ratios of coefficients and average marginal effects exhibit no bias in the absence of heterogeneity and negligible bias for a wide variety of distributions of regressors and individual effects in the presence of heterogeneity i subsequently propose new bias corrected estimators of index coefficients and marginal effects with improved finite sample properties for linear and nonlinear models with predetermined regressors

mwpcr multiscale weighted principal component regression for high dimensional prediction abstractwe propose a multiscale weighted principal component regression mwpcr framework for the use of high dimensional features with strong spatial features e g smoothness and correlation to predict an outcome variable such as disease status this development is motivated by identifying imaging biomarkers that could potentially aid detection diagnosis assessment of prognosis prediction of response to treatment and monitoring of disease status among many others the mwpcr can be regarded as a novel integration of principal components analysis pca kernel methods and regression models in mwpcr we introduce various weight matrices to prewhitten high dimensional feature vectors perform matrix decomposition for both dimension reduction and feature extraction and build a prediction model by using the extracted features examples of such weight matrices include an importance score weight matrix for the selection of individual features at each location and a spatial weight matrix for the incor