on principal hessian directions for data visualization and dimension reduction another application of stein s lemma abstract modern graphical tools have enhanced our ability to learn many things from data directly with much user friendly graphical software available we are encouraged to plot a lot more often than before the benefits from direct interaction with graphics have been enormous but trailing behind these high tech advances is the issue of appropriate guidance on what to plot there are too many directions to project a high dimensional data set and unguided plotting can be time consuming and fruitless in a recent article li set up a statistical framework for study on this issue based on a notion of effective dimension reduction edr directions they are the directions to project a high dimensional input variable for the purpose of effectively viewing and studying its relationship with an output variable a methodology sliced inverse regression was introduced and shown to be useful in finding edr directions this article introduces another method for finding edr directions it begins with the observat

estimation of the mean of a multivariate normal distribution estimation of the means of independent normal random variables is considered using sum of squared errors as loss an unbiased estimate of risk is obtained for an arbitrary estimate and certain special classes of estimates are then discussed the results are applied to smoothing by use of moving averages and to trimmed analogs of the james stein estimate a suggestion is made for calculating approximate confidence sets for the mean vector centered at an arbitrary estimate

on model selection criteria in multimodel analysis 1 hydrologic systems are open and complex rendering them prone to multiple conceptualizations and mathematical descriptions there has been a growing tendency to postulate several alternative hydrologic models for a site and use model selection criteria to 1 rank these models 2 eliminate some of them and or 3 weigh and average predictions and statistics generated by multiple models this has led to some debate among hydrogeologists about the merits and demerits of common model selection also known as model discrimination or information criteria such as aic aicc bic and kic and some lack of clarity about the proper interpretation and mathematical representation of each criterion we examine the model selection literature to find that 1 all published rigorous derivations of aic and aicc require that the true model having generated the observational data be in the set of candidate models 2 though bic and kic were originally derived by assuming that such a model is in the set bic has been rederived by cavanaugh and neath 1999 without the need for such an assumption and 3 kic reduces to bic as the number of observations becomes large relative to the number of adjustable model parameters implying that it likewise does not require the existence of a true model in the set of alternatives we explain why kic is the only criterion accounting validly for the likelihood of prior parameter estimates elucidate the unique role that the fisher information matrix plays in kic and demonstrate through an example that it imbues kic with desirable model selection properties not shared by aic aicc or bic our example appears to provide the first comprehensive test of how aic aicc bic and kic weigh and rank alternative models in light of the models predictive performance under cross validation with real hydrologic data

inverse regression estimation for censored data an inverse regression methodology for assessing predictor performance in the censored data setup is developed along with inference procedures and a computational algorithm the technique developed here allows for conditioning on the unobserved failure time along with a weighting mechanism that accounts for the censoring the implementation is nonparametric and computationally fast this provides an efficient methodological tool that can be used especially in cases where the usual modeling assumptions are not applicable to the data under consideration it can also be a good diagnostic tool that can be used in the model selection process we have provided theoretical justification of consistency and asymptotic normality of the methodology simulation studies and two data analyses are provided to illustrate the practical utility of the procedure

optimal m estimation in high dimensional regression we consider in the modern setting of high dimensional statistics the classic problem of optimizing the objective function in regression using m estimates when the error distribution is assumed to be known we propose an algorithm to compute this optimal objective function that takes into account the dimensionality of the problem although optimality is achieved under assumptions on the design matrix that will not always be satisfied our analysis reveals generally interesting families of dimension dependent objective functions

regularization theory and neural networks architectures we had previously shown that regularization principles lead to approximation schemes that are equivalent to networks with one layer of hidden units called regularization networks in particular s

likelihood based sufficient dimension reduction we obtain the maximum likelihood estimator of the central subspace under conditional normality of the predictors given the response analytically and in simulations we found that our new estimator can preform much better than sliced inverse regression sliced average variance estimation and directional regression and that it seems quite robust to deviations from normality

graphical procedures for multiple comparisons under general dependence abstractit has been more than half a century since tukey first introduced graphical displays that relate nonoverlap of confidence intervals to statistically significant differences between parameter estimates in this article we show how tukey s graphical overlap procedure can be modified to accommodate general forms of dependence within and across samples we also develop a procedure that can be used to more effectively resolve rankings within the tails of the distributions of parameter values thereby generalizing existing methods for multiple comparisons with the best we show that these new procedures retain the simplicity of tukey s original procedure while maintaining asymptotic control of the familywise error rate under very general conditions simple examples are used throughout to illustrate the procedures supplementary materials for this article are available online

high dimensionality effects in the markowitz problem and other quadratic programs with linear constraints risk underestimation we study the properties of solutions of quadratic programs with linear equality constraints whose parameters are estimated from data in the high dimensional setting where p the number of variables in the problem is of the same order of magnitude as n the number of observations used to estimate the parameters the markowitz problem in finance is a subcase of our study assuming normality and independence of the observations we relate the ecient frontier computed empirically to the true ecient frontier our computations show that there is a separation of the errors induced by estimating the mean of the observations and estimating the covariance matrix in particular the price paid for estimating the covariance matrix is an underestimation of the variance by a factor roughly equal to 1 p n therefore the risk of the optimal population solution is underestimated when we estimate it by solving a similar quadratic program with estimated parameters we also characterize the statistical behavior of linear functionals of the empirical optimal vector and show that they are biased estimators of the corresponding population quantities we investigate the robustness of our gaussian results by extending the study to certain elliptical models and models where our n observations are correlated in time we show a lack of robustness of the gaussian results but are still able to get results concerning rst order properties of the quantities of interest even in the case of relatively heavy tailed data we require two moments risk underestimation is still present in the elliptical case and more pronounced that in the gaussian case we discuss properties of the non parametric and parametric bootstrap in this context we show several results including the interesting fact that standard applications of the bootstrap generally yields inconsistent estimates of bias finally we propose some strategies to correct these problems and practically validate them in some simulations in all the paper we will assume that p n and n p tend to innity and p n

confidence sets for a multivariate distribution on etudie une version multivariable alternative de la bande de confiance unidimensionnelle de kolmogorov smirnov qui preserve l invariance affine

superconvergence and extrapolation for mixed finite element methods on rectangular domains asymptotic expansions for the rt raviart thomas mixed finite element approximation by the lowest order rectangular element associated with a second order elliptic equation on a rectangular domain are derived superconvergence for the vector field along the gauss lines is obtained as a result of the expansion a procedure of postprocessed extrapolation is presented for the scalar field as well as procedures of pure richardson extrapolation for both the vector and the scalar fields

identifying regression outliers and mixtures graphically regressions in practice can include outliers and other unknown subpopulation structures for example mixtures of regressions occur if there is an omitted categorical predictor like gender or location and different regressions occur within each category the theory of regression graphics based on central subspaces can be used to construct graphical solutions to long standing problems of this type it is argued that in practice the central subspace automatically expands to incorporate outliers and regression mixtures thus methods of estimating the central subspace can be used to identify these phenomena without specifying a model examples illustrating the power of the theory are presented

finite element galerkin approximation of the eigenvalues and eigenvectors of selfadjoint problems abstract this paper establishes some refined estimates for the approximation of the eigenvalues and eigenvectors of selfadjoint eigenvalue problems by finite element or more generally galerkin methods suppose lambda is an eigenvalue of multiplicity q of a selfajoint problem and let m lambda denote the space of eigenvectors corresponding to lambda keywords hilbert space

eigenvalue approximation by mixed and hybrid methods note univ maryland dept math college pk md 20742 ecole polytech ctr math appl f 91128 palaiseau france univ paris 6 f 75230 paris 05 france mercier b cea limeil f 94190 villeneuve france isi document delivery no lq584times cited 29cited reference count 44 reference asn article 1981 005doi 10 2307 2007651 record created on 2006 08 24 modified on 2017 05 12

on the conditional distributions of low dimensional projections from high dimensional data we study the conditional distribution of low dimensional projections from high dimensional data where the conditioning is on other low dimensional projections to fix ideas consider a random d vector z that has a lebesgue density and that is standardized so that ez 0 and ezz id moreover consider two projections defined by unit vectors and namely a response y z and an explanatory variable x z it has long been known that the conditional mean of y given x is approximately linear in x under some regularity conditions cf hall and li 1993 however a corresponding result for the conditional variance has not been available so far we here show that the conditional variance of y given x is approximately constant in x again under some regularity conditions these results hold uniformly in and for most s provided only that the dimension of z is large in that sense we see that most linear submodels of a high dimensional overall model are approximately correct our findings provide new insights in a variety of modeling scenarios we discuss several examples including sparse linear modeling generalized linear models under potential link violation sliced inverse regression sliced average variance estimation and kernel learning machines

a note on visualizing response transformations in regression a new graphical method for assessing parametric transformations of the response in linear regression is given we simply regress the response variable y on the predictors find the fitted values and then dynamically plot the transformed response y against those fitted values by varying the transformation parameter until the plot is linear the method can be used also to assess the success of numerical response transformation methods and to discover influential observations modifications using robust estimators can be used as well

leave one out approach for matrix completion primal and dual analysis in this paper we introduce a powerful technique based on leave one out analysis to the study of low rank matrix completion problems using this technique we develop a general approach for obtaining fine grained entrywise bounds for iterative stochastic procedures in the presence of probabilistic dependency we demonstrate the power of this approach in analyzing two of the most important algorithms for matrix completion i the non convex approach based on projected gradient descent pgd for a rank constrained formulation also known as the singular value projection algorithm and ii the convex relaxation approach based on nuclear norm minimization nnm using this approach we establish the first convergence guarantee for the original form of pgd without regularization or sample splitting and in particular shows that it converges linearly in the infinity norm for nnm we use this approach to study a fictitious iterative procedure that arises in the dual analysis our results show that nnm recovers an d by d rank r matrix with mathcal o mu r log mu r d log d observed entries this bound has optimal dependence on the matrix dimension and is independent of the condition number to the best of our knowledge none of previous sample complexity results for tractable matrix completion algorithms satisfies these two properties simultaneously

sliced inverse regression for dimension reduction abstract modern advances in computing power have greatly widened scientists scope in gathering and investigating information from many variables information which might have been ignored in the past yet to effectively scan a large pool of variables is not an easy task although our ability to interact with data has been much enhanced by recent innovations in dynamic graphics in this article we propose a novel data analytic tool sliced inverse regression sir for reducing the dimension of the input variable x without going through any parametric or nonparametric model fitting process this method explores the simplicity of the inverse view of regression that is instead of regressing the univariate output variable y against the multivariate x we regress x against y forward regression and inverse regression are connected by a theorem that motivates this method the theoretical properties of sir are investigated under a model of the form y f 1 x k x e where the k s are the unknown

fourth order eigenvalue approximation by extrapolation on domains with reentrant corners the eigenvalue problem of the laplace operator is considered on a non convex domain composed of rectangles this model problem may be solved by the finite element method with bilinear elements on a rectangular mesh it is known thato h 2 1 convergence can be obtained for the eigenvalues if the mesh haso h 2 points a simple extrapolation scheme is presented which on appropriately graded meshes increases the rate of convergence too h 4

graph connection laplacian methods can be made robust to noise recently several data analytic techniques based on graph connection laplacian gcl ideas have appeared in the literature at this point the properties of these methods are starting to be understood in the setting where the data is observed without noise we study the impact of additive noise on these methods and show that they are remarkably robust as a by product of our analysis we propose modifications of the standard algorithms that increase their robustness to noise we illustrate our results in numerical simulations

robust kernel principal component analysis this letter discusses the robustness issue of kernel principal component analysis a class of new robust procedures is proposed based on eigenvalue decomposition of weighted covariance the proposed procedures will place less weight on deviant patterns and thus be more resistant to data contamination and model deviation theoretical influence functions are derived and numerical examples are presented as well both theoretical and numerical results indicate that the proposed robust method outperforms the conventional approach in the sense of being less sensitive to outliers our robust method and results also apply to functional principal component analysis

dimension reduction in regression without matrix inversion summary regressions in which the fixed number of predictors p exceeds the number of independent observational units n occur in a variety of scientific fields sufficient dimension reduction provides a promising approach to such problems by restricting attention to d n linear combinations of the original p predictors however standard methods of sufficient dimension reduction require inversion of the sample predictor covariance matrix we propose a method for estimating the central subspace that eliminates the need for such inversion and is applicable regardless of the n p relationship simulations show that our method compares favourably with standard large sample techniques when the latter are applicable we illustrate our method with a genomics application

on distribution weighted partial least squares with diverging number of highly correlated predictors summary because highly correlated data arise from many scientific fields we investigate parameter estimation in a semiparametric regression model with diverging number of predictors that are highly correlated for this we first develop a distribution weighted least squares estimator that can recover directions in the central subspace then use the distribution weighted least squares estimator as a seed vector and project it onto a krylov space by partial least squares to avoid computing the inverse of the covariance of predictors thus distrbution weighted partial least squares can handle the cases with high dimensional and highly correlated predictors furthermore we also suggest an iterative algorithm for obtaining a better initial value before implementing partial least squares for theoretical investigation we obtain strong consistency and asymptotic normality when the dimension p of predictors is of convergence rate o n1 2 log n and o n1 3 respectively where n is the sample size when there are no other constraints on the covariance of predictors the rates n1 2 and n1 3 are optimal we also propose a bayesian information criterion type of criterion to estimate the dimension of the krylov space in the partial least squares procedure illustrative examples with a real data set and comprehensive simulations demonstrate that the method is robust to non ellipticity and works well even in small n large p problems

maximum likelihood bayesian averaging of spatial variability models in unsaturated fractured tuff 1 hydrologic analyses typically rely on a single conceptual mathematical model yet hydrologic environments are open and complex rendering them prone to multiple interpretations and mathematical descriptions adopting only one of these may lead to statistical bias and underestimation of uncertainty bayesian model averaging bma hoeting et al 1999 provides an optimal way to combine the predictions of several competing models and to assess their joint predictive uncertainty however it tends to be computationally demanding and relies heavily on prior information about model parameters neuman 2002 2003 proposed a maximum likelihood version mlbma of bma to render it computationally feasible and to allow dealing with cases where reliable prior information is lacking we apply mlbma to seven alternative variogram models of log air permeability data from single hole pneumatic injection tests in six boreholes at the apache leap research site alrs in central arizona unbiased ml estimates of variogram and drift parameters are obtained using adjoint state maximum likelihood cross validation samper and neuman 1989a in conjunction with universal kriging and generalized least squares standard information criteria provide an ambiguous ranking of the models which does not justify selecting one of them and discarding all others as is commonly done in practice instead we eliminate some of the models based on their negligibly small posterior probabilities and use the rest to project the measured log permeabilities by kriging onto a rock volume containing the six boreholes we then average these four projections and associated kriging variances using the posterior probability of each model as weight finally we cross validate the results by eliminating from consideration all data from one borehole at a time repeating the above process and comparing the predictive capability of mlbma with that of each individual model we find that mlbma is superior to any individual geostatistical model of log permeability among those we consider at the alrs

a probabilistic analysis of em for mixtures of separated spherical gaussians we show that given data from a mixture of k well separated spherical gaussians in d a simple two round variant of em will with high probability learn the parameters of the gaussians to near optimal precision if the dimension is high d ln k we relate this to previous theoretical and empirical work on the em algorithm

a comparison of three methods for selecting values of input variables in the analysis of output from a computer code two types of sampling plans are examined as alternatives to simple random sampling in monte carlo studies these plans are shown to be improvements over simple random sampling with respect to variance for a class of estimators which includes the sample mean and the empirical distribution function

self consistency and principal component analysis abstract i examine the self consistency of a principal component axis that is when a distribution is centered about a principal component axis a principal component axis of a random vector x is self consistent if each point on the axis corresponds to the mean of x given that x projects orthogonally onto that point a large class of symmetric multivariate distributions are examined in terms of self consistency of principal component subspaces elliptical distributions are characterized by the preservation of self consistency of principal component axes after arbitrary linear transformations a lack of fit test is proposed that tests for self consistency of a principal axis the test is applied to two real datasets

new families of skewed higher order kernel estimators to solve the bss ica problem for multimodal sources mixtures this letter suggests two new types of asymmetrical higher order kernels hok that are generated using the orthogonal polynomials laguerre positive or right skew and bessel negative or left skew these skewed hok are implemented in the blind source separation independent component analysis bss ica algorithm the tests for these proposed hok are accomplished using three scenarios to simulate a real environment using actual sound sources an environment of mixtures of multimodal fast changing probability density function pdf sources that represent a challenge to the symmetrical hok and an environment of an adverse case near gaussian the separation is performed by minimizing the mutual information mi among the mixed sources the performance of the skewed kernels is compared to the performance of the standard kernels such as epanechnikov bisquare trisquare and gaussian and the performance of the symmetrical hok generated using the polynomials chebyshev1 chebyshev2 gegenbauer jacobi and legendre to the tenth order the gaussian hok are generated using the hermite polynomial and the wand and schucany procedure the comparison among the 96 kernels is based on the average intersymbol interference ratio aisir and the time needed to complete the separation in terms of aisir the skewed kernels performance is better than that of the standard kernels and rivals most of the symmetrical kernels performance the importance of these new skewed hok is manifested in the environment of the multimodal pdf mixtures in such an environment the skewed hok come in first place compared with the symmetrical hok these new families can substitute for symmetrical hoks in such applications

a combined parameter scaling and inverse technique to upscale the unsaturated hydraulic parameters for heterogeneous soils 1 determining a large number of soil hydraulic parameters for heterogeneous soils remains a challenge because inverting for too many parameters can lead to parameter values that are nonunique furthermore such inversions may need very long simulation times for example months or more when inverting field scale problems in this research a combined parameter scaling and inverse technique cpsit is proposed to upscale hydraulic parameters from the local scale to the field scale the cpsit approach includes two steps 1 parameter scaling and 2 inverse modeling in step 1 the number of parameters to be estimated at field scale fs is reduced by applying parameter scaling whereby a heterogeneous soil is treated as a composition of multiple equivalent homogeneous media ehms in step 2 the fs parameters for the reference ehm are determined using the inverse technique and observations from well designed field experiments the advantages of the cpsit approach are that the number of parameters to be inverted is reduced by a factor of the number m of ehms and the simulation time is reduced by a factor of about m2 the cpsit approach was tested by upscaling the hydraulic parameters using a field injection experiment at the hanford site results show that when the cpsit upscaled parameters were used to simulated flow the mean squared residual was reduced by 83 2 relative to that when the local scale parameters were used

robust regression based on infinitesimal neighbourhoods we study robust estimation in the general normal regression model with random carriers permitting small departures from the model the framework is that of bickel 1981 we obtain solutions of huber 1982 kraskerhampel 1980 and krasker welsch 1982 as special cases as well as some new procedures our calculations indicate that the optimality properties of these estimates are more limited than suggested by krasker and welsch

asymptotics for high dimensional regression m estimates fixed design results we investigate the asymptotic distributions of coordinates of regression m estimates in the moderate p n regime where the number of covariates p grows proportionally with the sample size n under appropriate regularity conditions we establish the coordinate wise asymptotic normality of regression m estimates assuming a fixed design matrix our proof is based on the second order poincare inequality and leave one out analysis some relevant examples are indicated to show that our regularity conditions are satisfied by a broad class of design matrices we also show a counterexample namely an anova type design to emphasize that the technical assumptions are not just artifacts of the proof finally numerical experiments confirm and complement our theoretical results

sudakov s typical marginals random linear functionals and a conditional central limit theorem v n sudakov sud78 proved that the one dimensional marginals of a high dimensional second order measure are close to each other in most directions extending this and a related result in the context of projection pursuit of p diaconis and d freedman dia84 we give for a probability measure p and a random a s linear functional f on a hilbert space simple sufficient conditions under which most of the one dimensional images of p under f are close to their canonical mixture which turns out to be almost a mixed normal distribution using the concept of approximate conditioning we deduce a conditional central limit theorem theorem 3 for random averages of triangular arrays of random variables which satisfy only fairly weak asymptotic orthogonality conditions

investigations on two kinds of two grid mixed finite element methods for the elliptic eigenvalue problem two kinds of two grid new mixed finite element schemes for the elliptic eigenvalue problem based on less regularity of flux are considered in the new mixed variational formulation the flux belongs to the square integrable space instead of the classical h div and the choices of finite element pairs become simple and easy the two grid methods consist of solving a small elliptic eigenvalue problem on the coarse mesh and then solving a linear algebraic system on the fine mesh which can still maintain asymptotically optimal accuracy finally the performance of two kinds of two grid schemes are compared in efficiency and precision aspects by a series of numerical experiments numerical results show that the accelerated two grid method is a viable choice for the lowest order approximations of the elliptic eigenvalue problem

sensitivity analysis for box cox power transformation model contrast parameters summary sensitivity analysis is an important tool for the evaluation of scientific models we consider the box cox power transformation model and study the sensitivity for the estimated slope vector when the transformation parameter is perturbed this is a key issue in the debate among bickel doksum 1981 box cox 1982 and hinkley runger 1984 on the application of the power transformation model hinkley runger 1984 conjectured that box cox s 1964 z transformation would eliminate asymptotically the sensitivity for the estimated slope vector we establish this conjecture under appropriate symmetry conditions on the joint distribution for the regressors x when the true transformation is logarithmic the conjecture holds if the distribution for x is axially symmetric for other power transformations the conjecture holds if x is multivariate normal we also consider a weaker version of hinkley runger s conjecture namely the z transformation would achieve the best possible reduction in the sensitivity we establish this weaker conjecture under less restrictive symmetry conditions which only involve the marginal distribution for xfl both conjectures might fail when those symmetry conditions are not satisfied two examples are given to illustrate the limitations of the conjectures sensitivity analysis is an important tool for the evaluation of scientific models many scientific models include some nuisance parameters which are not known precisely it is usually desirable for the model output to be stable with respect to those parameters the performance of the model can be questionable if the model output is highly sensitive to those parameters it is therefore useful to carry out a sensitivity analysis in which those parameters are perturbed the model output is derived under the perturbed parameters and compared with the model output prior to the perturbation we will study the sensitivity analysis for the box cox power transformation model with respect to the specific transformation taken let yi xi i 1 n be the observed data where yi denotes a positive scalar outcome and xi denotes a p dimensional row vector of regressors let the transformed outcome be

the central limit problem for convex bodies it is shown that every symmetric convex body which satisfies a kind of weak law of large numbers has the property that almost all its marginal distributions are approximately gaussian several quite broad classes of bodies are shown to satisfy the condition

modeling water retention curves of sandy soils using neural networks we used neural networks nns to model the drying water retention curve wrc of 204 sandy soil samples from particle size distribution psd soil organic matter content som and bulk density bd neural networks can relate multiple model input data to multiple model output data without the need of an a priori model concept in this way a high performance black box model is created which is very useful in a data exploration effort to assess the maximum obtainable prediction accuracy we used a series of nn models with an increasing parametrization of input and output variables to get a better interpretability of model results in the first two models we used the nine psd fractions bd and som as input while we predicted the nine points of the water retention curve these nns had 12 input and 9 output variables predicting wrcs with an average root mean square residual rmsr water content of 0 020 cm3 cm 3 after a few intermediary models with increasing parametrization of psd and wrc using adapted van genuchten 1980 equations we arrived at a final nn model that used six input variables to predict three van genuchten 1980 parameters resulting in a rmsr of 0 024 cm3 cm 3 we found saturated and residual water contents to be unrelated to the psd bd or som therefore the saturated water content was considered to be an independent input variable while the residual water content was set to zero sensitivity analyses showed that the psd had a major influence on the shape of the wrc while bd and som were less important on the basis of these sensitivity analyses we established more explicit equations that demonstrated similarity relations between psd and wrc and incorporated effects of som and bd in an empirical way despite the fact that we considered a large number of linear and nonlinear variants these equations had a weaker performance rmsr 0 029 cm3 cm 3 than the nn models proving the modeling power of that technique

sensitivity analysis and assessment of prior model probabilities in mlbma with application to unsaturated fractured tuff 1 previous application of maximum likelihood bayesian model averaging mlbma neuman 2002 2003 to alternative variogram models of log air permeability data in fractured tuff has demonstrated its effectiveness in quantifying conceptual model uncertainty and enhancing predictive capability ye et al 2004 a question remained how best to ascribe prior probabilities to competing models in this paper we examine the extent to which lead statistics of posterior log permeability predictions are sensitive to prior probabilities of seven corresponding variogram models we then explore the feasibility of quantifying prior model probabilities by 1 maximizing shannon s entropy h shannon 1948 subject to constraints reflecting a single analyst s or a group of analysts prior perception about how plausible each alternative model or a group of models is relative to others and 2 selecting a posteriori the most likely among such maxima corresponding to alternative prior perceptions of various analysts or groups of analysts another way to select among alternative prior model probability sets which however is not guaranteed to yield optimum predictive performance though it did so in our example and would therefore not be our preferred option is a minimum maximum approach according to which one selects a priori the set corresponding to the smallest value of maximum entropy whereas maximizing h subject to the prior perception of a single analyst or group maximizes the potential for further information gain through conditioning selecting the smallest among such maxima gives preference to the most informed prior perception among those of several analysts or groups we use the same variogram models and log permeability data as ye et al 2004 to demonstrate that our proposed approach yields the least amount of posterior entropy residual uncertainty after conditioning and enhances predictive model performance as compared to 1 the noninformative neutral case in which all prior model probabilities are set equal to each other and 2 an informed case that nevertheless violates the principle of parsimony

exploratory projection pursuit abstract a new projection pursuit algorithm for exploring multivariate data is presented that has both statistical and computational advantages over previous methods a number of practical issues concerning its application are addressed a connection to multivariate density estimation is established and its properties are investigated through simulation studies and application to real data the goal of exploratory projection pursuit is to use the data to find low one two or three dimensional projections that provide the most revealing views of the full dimensional data with these views the human gift for pattern recognition can be applied to help discover effects that may not have been anticipated in advance since linear effects are directly captured by the covariance structure of the variable pairs which are straightforward to estimate the emphasis here is on the discovery of nonlinear effects such as clustering or other general nonlinear associations among the variables although arbitrary

classification of fmri time series in a low dimensional subspace with a spatial prior we propose a new method for detecting activation in functional magnetic resonance imaging fmri data we project the fmri time series on a low dimensional subspace spanned by wavelet packets in order to create projections that are as non gaussian as possible our approach achieves two goals it reduces the dimensionality of the problem by explicitly constructing a sparse approximation to the dataset and it also creates meaningful clusters allowing the separation of the activated regions from the clutter formed by the background time series we use a mixture of gaussian densities to model the distribution of the wavelet packet coefficients we expect activated areas that are connected and impose a spatial prior in the form of a markov random field our approach was validated with in vivo data and realistic synthetic data where it outperformed a linear model equipped with the knowledge of the true hemodynamic response

dimension reduction for conditional mean in regression in many situations regression analysis is mostly concerned with inferring about the conditional mean of the response given the predictors and less concerned with the other aspects of the conditional distribution in this paper we develop dimension reduction methods that incorporate this consideration we introduce the notion of the central mean subspace cms a natural inferential object for dimension reduction when the mean function is of interest we study properties of the cms and develop methods to estimate it these methods include a new class of estimators which requires fewer conditions than phd and which displays a clear advantage when one of the conditions for phd is violated cms also reveals a transparent distinction among the existing methods for dimension reduction ols phd sir and save we apply the new methods to a data set involving recumbent cows

determining the dimension of iterative hessian transformation the central mean subspace cms and iterative hessian transformation iht have been introduced recently for dimension reduction when the conditional mean is of interest suppose that x is a vector valued predictor and y is a scalar response the basic problem is to find a lower dimensional predictor t x such that e y x e y t x the cms defines the inferential object for this problem and iht provides an estimating procedure compared with other methods iht requires fewer assumptions and has been shown to perform well when the additional assumptions required by those methods fail in this paper we give an asymptotic analysis of iht and provide stepwise asymptotic hypothesis tests to determine the dimension of the cms as estimated by iht here the original iht method has been modified to be invariant under location and scale transformations to provide empirical support for our asymptotic results we will present a series of simulation studies these agree well with the theory the method is applied to analyze an ozone data set

multimodality exploration by an unsupervised projection pursuit neural network graphical inspection of multimodality is demonstrated using unsupervised lateral inhibition neural networks three projection pursuit indexes are compared on low dimensional simulated and real world data principal components legendre polynomial and projection pursuit network

a two grid discretization scheme for eigenvalue problems a two grid discretization scheme is proposed for solving eigenvalue problems including both partial differential equations and integral equations with this new scheme the solution of an eigenvalue problem on a fine grid is reduced to the solution of an eigenvalue problem on a much coarser grid and the solution of a linear algebraic system on the fine grid and the resulting solution still maintains an asymptotically optimal accuracy

an assumption free exact test for fixed design linear models with exchangeable errors we propose the cyclic permutation test cpt to test general linear hypotheses for linear models this test is non randomized and valid in finite samples with exact type i error alpha for an arbitrary fixed design matrix and arbitrary exchangeable errors whenever 1 alpha is an integer and n p ge 1 alpha 1 the test involves applying the marginal rank test to 1 alpha linear statistics of the outcome vector where the coefficient vectors are determined by solving a linear system such that the joint distribution of the linear statistics is invariant with respect to a non standard cyclic permutation group under the null hypothesis the power can be further enhanced by solving a secondary non linear travelling salesman problem for which the genetic algorithm can find a reasonably good solution extensive simulation studies show that the cpt has comparable power to existing tests when testing for a single contrast of coefficients an exact confidence interval can be obtained by inverting the test furthermore we provide a selective yet extensive literature review of the century long efforts on this problem highlighting the novelty of our test

analysis of some finite elements for the stokes problem we study some finite elements which are used in the approximation of the stokes problem so as to obtain error estimates of optimal order resume nous etudions deux elements finis utilises pour l approximation du probleme de stokes et obtenons des estimations d erreur d ordre optimal

asymptotic behavior of m estimators of p regression parameters when p 2 n is large ii normal approximation in a general linear model y x beta r with y and r n dimensional x a n times p matrix and beta p dimensional let hat beta be an m estimator of beta satisfying 0 sum x i psi y i x i beta let p rightarrow infty such that p log n 3 2 n rightarrow 0 then max i x i hat beta beta rightarrow p 0 and it is possible to find a uniform normal approximation for the distribution of hat beta under which arbitrary linear combinations a n hat beta beta are asymptotically normal when appropriately normalized and hat beta beta x x hat beta beta is approximately chi 2 p

the multilevel mixed finite element discretizations based on local defect correction for the stokes eigenvalue problem abstract based on the work of xu and zhou 2000 this paper combines mixed finite element method and the local defect correction technique to establish new local and parallel multilevel discretization schemes for the stokes eigenvalue problem theoretical analysis and numerical experiments show that the computational approach proposed in this paper is simple and easy to carry out in parallel and can be used to solve singular stokes eigenvalue problem efficiently this paper also discusses local error estimates of mixed finite element approximations it is a new feature here that the estimates apply to the local domains containing corner points

a model free test for reduced rank in multivariate regression we propose a test of dimension in multivariate regression this test is in the spirit of tests on the rank of the coefficient matrix in a multivariate linear model but it does not require a prespecified model the test may be particularly useful at the outset of an analysis before a multivariate model is posited because it can lead to low dimensional summary plots that are inferred to contain all of the sample information on the multivariate mean function

a multivariate distribution for sub images a new method for obtaining multivariate distributions for sub images of natural images is described the information in each sub image is summarized by a measurement vector in a measurement space the dimension of the measurement space is reduced by applying a random projection to the truncated output of the discrete cosine transforms of the sub images the measurement space is then reparametrized such that a gaussian distribution is a good model for the measurement vectors in the reparametrized space an ornstein uhlenbeck process associated with the gaussian distribution is used to model the differences between measurement vectors obtained from matching sub images the probability of a false alarm and the probability of accepting a correct match are calculated the accuracy of the resulting statistical model for matching sub images is tested using images from the middlebury stereo database with promising results in particular if the probability of accepting a correct match is relatively large then there is good agreement between the calculated and the experimental probabilities of obtaining a unique match that is also a correct match

high dimensional robust m estimation asymptotic variance via approximate message passing in a recent article el karoui et al proc natl acad sci 110 36 14557 14562 2013 study the distribution of robust regression estimators in the regime in which the number of parameters p is of the same order as the number of samples n using numerical simulations and highly plausible heuristic arguments they unveil a striking new phenomenon namely the regression coefficients contain an extra gaussian noise component that is not explained by classical concepts such as the fisher information matrix we show here that that this phenomenon can be characterized rigorously using techniques that were developed by the authors for analyzing the lasso estimator under high dimensional asymptotics we introduce an approximate message passing amp algorithm to compute m estimators and deploy state evolution to evaluate the operating characteristics of amp and so also m estimates our analysis clarifies that the extra gaussian noise encountered in this problem is fundamentally similar to phenomena already studied for regularized least squares in the setting n p

on the impact of predictor geometry on the performance on high dimensional ridge regularized generalized robust regression estimators we study ridge regularized generalized robust regression estimators i e begin aligned widehat beta mathrm argmin beta in mathbb r p frac 1 n sum i 1 n rho i y i x i beta frac tau 2 beta 2 quad text where quad y i epsilon i x i beta 0 end aligned in the situation where p n tends to a finite non zero limit our study here focuses on the situation where the errors epsilon i s are heavy tailed and x i s have an elliptical like distribution our assumptions are quite general and we do not require homoskedasticity of epsilon i s for instance we obtain a characterization of the limit of widehat beta beta 0 as well as several other results including central limit theorems for the entries of widehat beta

optimal rates of convergence for noisy sparse phase retrieval via thresholded wirtinger flow this paper considers the noisy sparse phase retrieval problem recovering a sparse signal mathbf x in mathbb r p from noisy quadratic measurements y j mathbf a j mathbf x 2 varepsilon j j 1 ldots m with independent sub exponential noise varepsilon j the goals are to understand the effect of the sparsity of mathbf x on the estimation precision and to construct a computationally feasible estimator to achieve the optimal rates adaptively inspired by the wirtinger flow ieee trans inform theory61 2015 1985 2007 proposed for non sparse and noiseless phase retrieval a novel thresholded gradient descent algorithm is proposed and it is shown to adaptively achieve the minimax optimal rates of convergence over a wide range of sparsity levels when the mathbf a j s are independent standard gaussian random vectors provided that the sample size is sufficiently large compared to the sparsity of mathbf x

nonconvex rectangular matrix completion via gradient descent without l regularization the analysis of nonconvex matrix completion has recently attracted much attention in the community of machine learning thanks to its computational convenience existing analysis on this problem however usually relies on ell 2 infty projection or regularization that involves unknown model parameters although they are observed to be unnecessary in numerical simulations in this paper we extend the analysis of the vanilla gradient descent for positive semidefinite matrix completion in the literature to the rectangular case and more significantly improve the required sampling rate from o mathrm poly kappa mu 3 r 3 log 3 n n to o mu 2 r 2 kappa 14 log n n our technical ideas and contributions are potentially useful in improving the leave one out analysis in other related problems

on directional regression for dimension reduction we introduce directional regression dr as a method for dimension reduction like contour regression dr is derived from empirical directions but achieves higher accuracy and requires substantially less computation dr naturally synthesizes the dimension reduction estimators based on conditional moments such as sliced inverse regression and sliced average variance estimation and in doing so combines the advantages of these methods under mild conditions it provides exhaustive and consistent estimate of the dimension reduction space we develop the asymptotic distribution of the dr estimator and from that a sequential test procedure to determine the dimension of the central space we compare the performance of dr with that of existing methods by simulation and find strong evidence of its advantage over a wide range of models finally we apply dr to analyze a data set concerning the identification of hand written digits

quantification of uncertainty in pedotransfer function based parameter estimation for unsaturated flow modeling 1 while pedotransfer functions ptfs have long been applied to estimate soil hydraulic parameters for unsaturated flow and solute transport modeling the uncertainty associated with the estimates is often ignored the objective of this study is to evaluate uncertainty of the ptf estimated soil hydraulic parameters and its effect on numerical simulation of moisture flow contributing to the parameter estimation uncertainty are 1 the ptf intrinsic uncertainty caused by limited data used for ptf training and 2 the ptf input uncertainty in pedotransfer variables i e ptf inputs the ptf intrinsic uncertainty is assessed using the bootstrap method by generating multiple bootstrap realizations of the soil hydraulic parameters the realizations follow normal or lognormal distributions the ptf input variables i e bulk density and soil texture are obtained using the cokriging technique the ptf input uncertainty is quantified by assuming that the cokriging estimates follow a normal distribution our results show that the ptf input uncertainty dominates over the ptf intrinsic uncertainty and determines the spatial distribution of the ptf parameter estimation uncertainty when the parameter estimation uncertainty is included the spatial variability of the measured soil hydraulic parameters is better captured this is also the case for the observed moisture contents whose spatial variability is well bracketed by the prediction intervals however this is only possible after the ptf input uncertainty is considered these results suggest that additional sample acquisition for the ptf input variables would have a more favorable impact on reduction of the parameter estimation uncertainty than collecting additional soil hydraulic parameter measurements for ptf development

a central limit theorem for convex sets we show that there exists a sequence varepsilon n searrow0 for which the following holds let k n be a compact convex set with a non empty interior let x be a random vector that is distributed uniformly in k then there exist a unit vector in n t0 and 0 such that sup a subset mathbb r left textit prob langle x theta rangle in a frac 1 sqrt 2 pi sigma int ae frac t t 0 2 2 sigma 2 dt right leq varepsilon n qquad ast where the supremum runs over all measurable sets a and where denotes the usual scalar product in n furthermore under the additional assumptions that the expectation of x is zero and that the covariance matrix of x is the identity matrix we may assert that most unit vectors satisfy with t0 0 and 1 corresponding principles also hold for multi dimensional marginal distributions of convex sets

on robust regression with high dimensional predictors we study regression m estimates in the setting where p the number of covariates and n the number of observations are both large but p n we find an exact stochastic representation for the distribution of argmin p i 1 n y i x i at fixed p and n under various assumptions on the objective function and our statistical model a scalar random variable whose deterministic limit r can be studied when p n 0 plays a central role in this representation we discover a nonlinear system of two deterministic equations that characterizes r interestingly the system shows that r depends on through proximal mappings of as well as various aspects of the statistical model underlying our study several surprising results emerge in particular we show that when p n is large enough least squares becomes preferable to least absolute deviations for double exponential errors

the discrimination power of projection pursuit with different density estimators summary we explore the properties of projection pursuit discriminant analysis this discriminant method is very powerful but relies heavily on a univariate density estimate we show that the procedure based on wavelets maintains the same rate of convergence as with univariate wavelet density estimation we also show the bayes risk strong consistency of both the kernel and wavelet based methods simulated data and real data concerning character recognition show that the method is effective and robust against the curse of dimensionality the wavelet alternative seems more likely than the kernel counterpart to find an interesting projection wavelets are often criticised for giving too wiggly an estimate and for being too localised to give good global properties in the above context these potential drawbacks do not weaken the method but the use of wavelets seems to enhance it a multiple projection generalisation is also considered

rates of growth and sample moduli for weighted empirical processes indexed by sets probability inequalities are obtained for the supremum of a weighted empirical process indexed by a vapnik cervonenkis class c of sets these inequalities are particularly useful under the assumption p c c p c t 0 as t 0 they are used to obtain almost sure bounds on the rate of growth of the process as the sample size approaches infinity to find an asymptotic sample modulus for the unweighted empirical process and to study the ratio pn p of the empirical measure to the actual measure

nonuniformity of p values can occur early in diverging dimensions evaluating the joint significance of covariates is of fundamental importance in a wide range of applications to this end p values are frequently employed and produced by algorithms that are powered by classical large sample asymptotic theory it is well known that the conventional p values in gaussian linear model are valid even when the dimensionality is a non vanishing fraction of the sample size but can break down when the design matrix becomes singular in higher dimensions or when the error distribution deviates from gaussianity a natural question is when the conventional p values in generalized linear models become invalid in diverging dimensions we establish that such a breakdown can occur early in nonlinear models our theoretical characterizations are confirmed by simulation studies

robust estimation with many instruments abstract linear instrumental variables models are widely used in empirical work but often associated with low estimator precision this paper proposes an estimator that is robust to outliers and shows that the estimator is minimax optimal in a class of estimators that includes the limited maximum likelihood estimator liml intuitively this optimal robust estimator combines liml with winsorization of the structural residuals and the winsorization leads to improved precision under thick tailed error distributions consistency and asymptotic normality of the estimator are established under many instruments asymptotics and a consistent variance estimator which allows for asymptotically valid inference is provided

a finite element splitting extrapolation for second order hyperbolic equations splitting extrapolation is an efficient technique for solving large scale scientific and engineering problems in parallel this article discusses a finite element splitting extrapolation for second order hyperbolic equations with time dependent coefficients this method possesses a higher degree of parallelism less computational complexity and more flexibility than richardson extrapolation while achieving the same accuracy by means of domain decomposition and isoparametric mapping some grid parameters are chosen according to the problem the multiparameter asymptotic expansion of the d quadratic finite element error is also established the splitting extrapolation formulas are developed from this expansion an approximation with higher accuracy on a globally fine grid can be computed by solving a set of smaller discrete subproblems on different coarser grids in parallel some a posteriori error estimates are also provided numerical examples show that this method is efficient for solving discontinuous problems and nonlinear hyperbolic equations

projection pursuit in high dimensions projection pursuit is a classical exploratory data analysis method to detect interesting low dimensional structures in multivariate data originally projection pursuit was applied mostly to data of moderately low dimension motivated by contemporary applications we here study its properties in high dimensional settings specifically we analyze the asymptotic properties of projection pursuit on structureless multivariate gaussian data with an identity covariance as both dimension p and sample size n tend to infinity with p n 0 our main results are that i if then there exist projections whose corresponding empirical cumulative distribution function can approximate any arbitrary distribution and ii if 0 not all limiting distributions are possible however depending on the value of various non gaussian distributions may still be approximated in contrast if we restrict to sparse projections involving only a few of the p variables then asymptotically all empirical cumulative distribution functions are gaussian and iii if 0 then asymptotically all projections are gaussian some of these results extend to mean centered sub gaussian data and to projections into k dimensions hence in the small n large p setting unless sparsity is enforced and regardless of the chosen projection index projection pursuit may detect an apparent structure that has no statistical significance furthermore our work reveals fundamental limitations on the ability to detect non gaussian signals in high dimensional data in particular through independent component analysis and related non gaussian component analysis

a modern maximum likelihood theory for high dimensional logistic regression students in statistics or data science usually learn early on that when the sample size n is large relative to the number of variables p fitting a logistic model by the method of maximum likelihood produces estimates that are consistent and that there are well known formulas that quantify the variability of these estimates which are used for the purpose of statistical inference we are often told that these calculations are approximately valid if we have 5 to 10 observations per unknown parameter this paper shows that this is far from the case and consequently inferences produced by common software packages are often unreliable consider a logistic model with independent features in which n and p become increasingly large in a fixed ratio we prove that i the maximum likelihood estimate mle is biased ii the variability of the mle is far greater than classically estimated and iii the likelihood ratio test lrt is not distributed as a 2 the bias of the mle yields wrong predictions for the probability of a case based on observed values of the covariates we present a theory which provides explicit expressions for the asymptotic bias and variance of the mle and the asymptotic distribution of the lrt we empirically demonstrate that these results are accurate in finite samples our results depend only on a single measure of signal strength which leads to concrete proposals for obtaining accurate inference in finite samples through the estimate of this measure

estimating the structural dimension of regressions via parametric inverse regression a new estimation method for the dimension of a regression at the outset of an analysis is proposed a linear subspace spanned by projections of the regressor vector x which contains part or all of the modelling information for the regression of a vector y on x and its dimension are estimated via the means of parametric inverse regression smooth parametric curves are fitted to the p inverse regressions via a multivariate linear model no restrictions are placed on the distribution of the regressors the estimate of the dimension of the regression is based on optimal estimation procedures a simulation study shows the method to be more powerful than sliced inverse regression in some situations

linear dimensionality reduction survey insights and generalizations linear dimensionality reduction methods are a cornerstone of analyzing high dimensional data due to their simple geometric interpretations and typically attractive computational properties these methods capture many data features of interest such as covariance dynamical structure correlation between data sets input output relationships and margin between data classes methods have been developed with a variety of names and motivations in many fields and perhaps as a result the connections between all these methods have not been highlighted here we survey methods from this disparate literature as optimization programs over matrix manifolds we discuss principal component analysis factor analysis linear multidimensional scaling fisher s linear discriminant analysis canonical correlations analysis maximum autocorrelation factors slow feature analysis sufficient dimensionality reduction undercomplete independent component analysis linear regression distance metric learning and more this optimization framework gives insight to some rarely discussed shortcomings of well known methods such as the suboptimality of certain eigenvector solutions modern techniques for optimization over matrix manifolds enable a generic linear dimensionality reduction solver which accepts as input data and an objective to be optimized and returns as output an optimal low dimensional projection of the data this simple optimization framework further allows straightforward generalizations and novel variants of classical methods which we demonstrate here by creating an orthogonal projection canonical correlations analysis more broadly this survey and generic solver suggest that linear dimensionality reduction can move toward becoming a blackbox objective agnostic numerical technology

measurement error regression with unknown link dimension reduction and data visualization abstract a general nonlinear regression problem is considered with measurement error in the predictors we assume that the response is related to an unknown linear combination of a multidimensional predictor through an unknown link function instead of observing the predictor we instead observe a surrogate with the property that its expectation is linearly related to the true predictor with constant variance we identify an important transformation of the surrogate variable using this transformed variable we show that if one proceeds with the usual analysis ignoring measurement error then both ordinary least squares and sliced inverse regression yield estimates which consistently estimate the true regression parameter up to a constant of proportionality we derive the asymptotic distribution of the estimates a simulation study is conducted applying sliced inverse regression in this context

the full approximation accuracy for the stream function vorticity pressure method by means of identity techniques in this paper we develop the stream function vorticity pressure method and obtain the full approximation convergence and global superconvergence estimates for the stokes equations

pattern classification using projection pursuit abstract this article discusses the adaptation of recently developed regression techniques to classifier design apart from finite sample effects projection pursuit pp regression may be used to model a desired response class as a sum of ridge functions according to a minimum expected squared error criterion this approach can be shown to furnish an optimal discriminant function which can satisfy the neyman pearson criterion over all possible thresholds basis function expansions are used instead of smoothed histograms to reduce computation since good approximation of a discriminant by a linear combination of moderate number of ridge functions may not be easy we introduce an improved method utilizing a nonlinear weighting function

dimension reduction for censored regression data without parametric assumptions high dimensional regression analysis is already complex this is made even harder when data are subject to censoring in this article we seek ways of reducing the dimensionality of the regressor before applying nonparametric smoothing techniques if the censoring time is independent of the lifetime then the method of sliced inverse regression can be applied directly otherwise modification is needed to adjust for the censoring bias a key identity leading to the bias correction is derived and the root n consistency of the modified estimate is established patterns of censoring can also be studied under a similar dimension reduction framework some simulation results and an application to a real data set are reported 1 introduction survival data are often subject to censoring when this occurs the incompleteness of the observed data may induce a substantial bias in the sample several approaches have been suggested to overcome the associated difficulties in regression including the accelerated failure time model censored linear regression the cox proportional hazard model and many others survival analysis becomes even more intricate when the dimension of the regressor increases to apply any of the aforementioned methods users are required to specify a functional form which relates the outcome variables to the input ones however in reality knowledge needed for an appropriate model specification is often inadequate as a matter of fact the acquisition of such information may well turn out to be one of the primary goals of the study itself under such circumstances it seems preferable to have exploratory tools that rely less on such model specification this is the issue to be addressed in this article the dimension reduction approach of li 1991 will be extended to settings which allow for censoring in the data we shall offer methods of finding low dimensional projections of the data for visually examining the censoring pattern we shall show how censored regression data can still be analyzed without assuming the functional form a priori

inference in linear regression models with many covariates and heteroskedasticity the linear regression model is widely used in empirical work in economics statistics and many other disciplines researchers often include many covariates in their linear model specification in a

gradient descent with random initialization fast global convergence for nonconvex phase retrieval this paper considers the problem of solving systems of quadratic equations namely recovering an object of interest varvec x natural in mathbb r n from m quadratic equations samples y i varvec a i top varvec x natural 2 1 le i le m this problem also dubbed as phase retrieval spans multiple domains including physical sciences and machine learning we investigate the efficacy of gradient descent or wirtinger flow designed for the nonconvex least squares problem we prove that under gaussian designs gradient descent when randomly initialized yields an epsilon accurate solution in o big log n log 1 epsilon big iterations given nearly minimal samples thus achieving near optimal computational and sample complexities at once this provides the first global convergence guarantee concerning vanilla gradient descent for phase retrieval without the need of i carefully designed initialization ii sample splitting or iii sophisticated saddle point escaping schemes all of these are achieved by exploiting the statistical models in analyzing optimization algorithms via a leave one out approach that enables the decoupling of certain statistical dependency between the gradient descent iterates and the data

asymptotic expansions and extrapolations of eigenvalues for the stokes problem by mixed finite element methods this paper derives a general procedure to produce an asymptotic expansion for eigenvalues of the stokes problem by mixed finite elements by means of integral expansion technique the asymptotic error expansions for the approximations of the stokes eigenvalue problem by bernadi raugel element and q 2 p 1 element are given based on such expansions the extrapolation technique is applied to improve the accuracy of the approximations

sparse non gaussian component analysis non gaussian component analysis ngca introduced in offered a method for high dimensional data analysis allowing for identifying a low dimensional non gaussian component of the whole distribution in an iterative and structure adaptive way an important step of the ngca procedure is identification of the non gaussian subspace using principle component analysis pca method this article proposes a new approach to ngca called sparse ngca which replaces the pca based procedure with a new the algorithm we refer to as convex projection