sufficient dimension reduction via squared loss mutual information estimation the goal of sufficient dimension reduction in supervised learning is to find the low dimensional subspace of input features that contains all of the information about the output values that the input features possess in this letter we propose a novel sufficient dimension reduction method using a squared loss variant of mutual information as a dependency measure we apply a density ratio estimator for approximating squared loss mutual information that is formulated as a minimum contrast estimator on parametric or nonparametric models since cross validation is available for choosing an appropriate model our method does not require any prespecified structure on the underlying distributions we elucidate the asymptotic bias of our estimator on parametric models and the asymptotic convergence rate on nonparametric models the convergence analysis utilizes the uniform tail bound of a u process and the convergence rate is characterized by the bracketing entropy of the model we then develop a natural gradient algorithm on the grassmann manifold for sufficient subspace search the analytic formula of our estimator allows us to compute the gradient efficiently numerical experiments show that the proposed method compares favorably with existing dimension reduction approaches on artificial and benchmark data sets

sufficient dimension reduction in regressions across heterogeneous subpopulations chiaromonte and co workers extended this method to regressions with qualitative predictors and developed a method partial sliced inverse regression under the assumption that the covariance matrices of the continuous predictors are constant across the levels of the qualitative predictor we extend partial sliced inverse regression by removing the restrictive homogeneous covariance condition this extension which significantly expands the applicability of the previous methodology is based on a new estimation method that makes use of a non linear least squares objective function

convergence theorems for strongly pseudo contractive and strongly accretive maps supposeeis an arbitrary real banach space andkis a nonempty closed convex and bounded subset ofe supposet k kis a uniformly continuous strong pseudo contraction with constantk 0 1 suppose an bn cn a n b n and c n are sequences in 0 1 satisfying the following conditions i an bn cn 1 a n b n c n integersn 0 ii lim bn lim b n lim c n 0 iii bn iv cn for arbitraryx0 u0 v0 k define the sequence xn n 0iteratively byxn 1 anxn bntyn cnun yn a nxn b ntxn c nvn n 0 where un vn are arbitrary sequences ink then xn converges strongly to the unique fixed point oft related results deal with the iterative solutions of nonlinear equations involving set valued strongly accretive operators

k means inverse regression li suggested the method of sliced inverse regression for dimension reduction in regressions with a univariate response in this article we extend that method to multivariate regressions by introducing a new way of performing the slicing this method applies for any number of response variables and may be particularly useful at the outset of an analysis before positing a multivariate model the emphasis is on application no new asymptotic theory is presented

aggregate inverse mean estimation for sufficient dimension reduction many well known sufficient dimension reduction methods investigate the inverse conditional moments of the predictors given the response the required linearity condition the number and ar

a validated information criterion to determine the structural dimension in dimension reduction models summary a crucial component of performing sufficient dimension reduction is to determine the structural dimension of the reduction model we propose a novel information criterion based method for this purpose a special feature of which is that when examining the goodness of fit of the current model one needs to perform model evaluation by using an enlarged candidate model althoughtheproceduredoesnotrequireestimationundertheenlargedmodelofdimensionk 1 the decision as to how well the current model of dimensionk fits relies on the validation provided by the enlarged model thus we call this procedure the validated information criterion vic k our method is different from existing information criterion based model selection methods it breaks free from dependence on the connection between dimension reduction models and their corresponding matrix eigenstructures which relies heavily on a linearity condition that we no longer assume we prove consistency of the proposed method and its finite sample performance is demonstrated numerically

on principal hessian directions for data visualization and dimension reduction another application of stein s lemma abstract modern graphical tools have enhanced our ability to learn many things from data directly with much user friendly graphical software available we are encouraged to plot a lot more often than before the benefits from direct interaction with graphics have been enormous but trailing behind these high tech advances is the issue of appropriate guidance on what to plot there are too many directions to project a high dimensional data set and unguided plotting can be time consuming and fruitless in a recent article li set up a statistical framework for study on this issue based on a notion of effective dimension reduction edr directions they are the directions to project a high dimensional input variable for the purpose of effectively viewing and studying its relationship with an output variable a methodology sliced inverse regression was introduced and shown to be useful in finding edr directions this article introduces another method for finding edr directions it begins with the observat

testing predictor contributions in sufficient dimension reduction we develop tests of the hypothesis of no effect for selected predictors in regression without assuming a model for the conditional distribution of the response given the predictors predictor effects need not be limited to the mean function and smoothing is not required the general approach is based on sufficient dimension reduction the idea being to replace the predictor vector with a lower dimensional version without loss of information on the regression methodology using sliced inverse regression is developed in detail

a note on shrinkage sliced inverse regression summary we employ lasso shrinkage within the context of sufficient dimension reduction to obtain a shrinkage sliced inverse regression estimator which provides easier interpretations and better pre diction accuracy without assuming a parametric model the shrinkage sliced inverse regression approach can be employed for both single index and multiple index models simulation studies suggest that the new estimator performs well when its tuning parameter is selected by either the bayesian information criterion or the residual information criterion

constrained inverse regression for incorporating prior information inverse regression methods facilitate dimension reduction analyses of high dimensional data by extracting a small number of factors that are linear combinations of the original predictor variables but the estimated factors may not lend themselves readily to interpretation consistent with prior information our approach to solving this problem is to first incorporate prior information via theory or data driven constraints on model parameters and then apply the proposed method constrained inverse regression cir to extract factors that satisfy the constraints we provide chi squared and t tests to assess the significance of each factor and its estimated coefficients and we also generalize cir to other inverse regression methods in situations where both dimension reduction and factor interpretation are important finally we investigate cir s small sample performance test data driven constraints and present a marketing example to illustrate its use in discovering meaningful factors that influence th

determining the dimension in sliced inverse regression and related methods abstract sliced inverse regression sir and principal hessian directions aim to reduce the dimensionality of regression problems an important step in the method is the determination of a suitable dimension although statistical tests based on the nullity eigenvalues are usually suggested this article focuses on the quality of the estimation of the effective dimension reduction edr spaces essentially the goal is to retain only sufficiently stable subspaces the goodness of the estimation is measured by the squared trace correlation between the subspaces of the edr space and their estimates asymptotic expansions are derived and estimates deduced simulations give an insight on the behavior of the criterion and indicate how it can be used in practice

sufficient dimension reduction with missing predictors in high dimensional data analysis sufficient dimension reduction sdr methods are effective in reducing the predictor dimension while retaining full regression information and imposing no parametric models however it is common in high dimensional data that a subset of predictors may have missing observations existing sdr methods resort to the complete case analysis by removing all the subjects with missingness in any of the predictors under inquiry such an approach does not make effective use of the data and is valid only when missingness is independent of both observed and unobserved quantities in this article we propose a new class of sdr estimators under a more general missingness mechanism that allows missingness to depend on the observed data we focus on a widely used sdr method sliced inverse regression and propose an augmented inverse probability weighted sliced inverse regression estimator aipw sir we show that aipw sir is doubly robust and asymptotically consistent and demonstrate

sequential sufficient dimension reduction for large p small n problems summary we propose a new and simple framework for dimension reduction in the large p small n setting the framework decomposes the data into pieces thereby enabling existing approaches for n p to be adapted to n p problems estimating a large covariance matrix which is a very difficult task is avoided we propose two separate paths to implement the framework our paths provide sufficient procedures for identifying informative variables via a sequential approach we illustrate the paths by using sufficient dimension reduction approaches but the paths are very general empirical evidence demonstrates the efficacy of our paths additional simulations and applications are given in an on line supplementary file

marginal tests with sliced average variance estimation summary we present a new computationally feasible test for the dimension of the central subspace in a regression problem based on sliced average variance estimation we also provide a marginal coordinate test under the null hypothesis both the test of dimension and the marginal coordinate test involve test statistics that asymptotically have chi squared distributions given normally distributed predictors and have a distribution that is a linear combination of chi squared distributions in general

an adaptive estimation of dimension reduction space summary searching for an effective dimension reduction space is an important problem in regression especially for high dimensional data we propose an adaptive approach based on semiparametric models which we call the conditional minimum average variance estimation mave method within quite a general setting the mave method has the following advantages most existing methods must undersmooth the nonparametric link function estimator to achieve a faster rate of consistency for the estimator of the parameters than for that of the nonparametric function in contrast a faster consistency rate can be achieved by the mave method even without undersmoothing the nonparametric link function estimator the mave method is applicable to a wide range of models with fewer restrictions on the distribution of the covariates to the extent that even time series can be included because of the faster rate of consistency for the parameter estimators it is possible for us to estimate the dimension of the space consistently the relationship of the mave method with other methods is also investigated in particular a simple outer product gradient estimator is proposed as an initial estimator in addition to theoretical results we demonstrate the efficacy of the mave method for high dimensional data sets through simulation two real data sets are analysed by using the mave approach

sliced inverse regression for dimension reduction abstract modern advances in computing power have greatly widened scientists scope in gathering and investigating information from many variables information which might have been ignored in the past yet to effectively scan a large pool of variables is not an easy task although our ability to interact with data has been much enhanced by recent innovations in dynamic graphics in this article we propose a novel data analytic tool sliced inverse regression sir for reducing the dimension of the input variable x without going through any parametric or nonparametric model fitting process this method explores the simplicity of the inverse view of regression that is instead of regressing the univariate output variable y against the multivariate x we regress x against y forward regression and inverse regression are connected by a theorem that motivates this method the theoretical properties of sir are investigated under a model of the form y f 1 x k x e where the k s are the unknown

determining the dimensionality in sliced inverse regression abstract a general regression problem is one in which a response variable can be expressed as some function of one or more different linear combinations of a set of explanatory variables as well as a random error term sliced inverse regression is a method for determining these linear combinations in this article we address the problem of determining how many linear combinations are involved procedures based on conditional means and conditional covariance matrices as well as a procedure combining the two approaches are considered in each case we develop a test that has an asymptotic chi squared distribution when the vector of explanatory variables is sampled from an elliptically symmetric distribution

on sliced inverse regression with high dimensional covariates sliced inverse regression is a promising method for the estimation of the central dimension reduction subspace cdr space in semiparametric regression models it is particularly useful in tackling cases with high dimensional covariates in this article we study the asymptotic behavior of the estimate of the cdr space with high dimensional covariates that is when the dimension of the covariates goes to infinity as the sample size goes to infinity strong and weak convergence are obtained we also suggest an estimation procedure of the bayes information criterion type to ascertain the dimension of the cdr space and derive the consistency a simulation study is conducted

on distribution weighted partial least squares with diverging number of highly correlated predictors summary because highly correlated data arise from many scientific fields we investigate parameter estimation in a semiparametric regression model with diverging number of predictors that are highly correlated for this we first develop a distribution weighted least squares estimator that can recover directions in the central subspace then use the distribution weighted least squares estimator as a seed vector and project it onto a krylov space by partial least squares to avoid computing the inverse of the covariance of predictors thus distrbution weighted partial least squares can handle the cases with high dimensional and highly correlated predictors furthermore we also suggest an iterative algorithm for obtaining a better initial value before implementing partial least squares for theoretical investigation we obtain strong consistency and asymptotic normality when the dimension p of predictors is of convergence rate o n1 2 log n and o n1 3 respectively where n is the sample size when there are no other constraints on the covariance of predictors the rates n1 2 and n1 3 are optimal we also propose a bayesian information criterion type of criterion to estimate the dimension of the krylov space in the partial least squares procedure illustrative examples with a real data set and comprehensive simulations demonstrate that the method is robust to non ellipticity and works well even in small n large p problems

convergence theorems for intermediate problems ii convergence theorems for the practical eigenvector free methods of gay and goerisch are obtained under a variety of hypotheses so that our theorems apply to both traditional boundary value problems and atomic problems in addition we prove convergence of the t t method of bazley and fox without an alignment of projections hypothesis required in previous literature

gradient based kernel dimension reduction for regression this article proposes a novel approach to linear dimension reduction for regression using nonparametric estimation with positive definite kernels or reproducing kernel hilbert spaces rkhss the purpose of the dimension reduction is to find such directions in the explanatory variables that explain the response sufficiently this is called sufficient dimension reduction the proposed method is based on an estimator for the gradient of the regression function considered for the feature vectors mapped into rkhss it is proved that the method is able to estimate the directions that achieve sufficient dimension reduction in comparison with other existing methods the proposed one has wide applicability without strong assumptions on the distributions or the type of variables and needs only eigendecomposition for estimating the projection matrix the theoretical analysis shows that the estimator is consistent with certain rate under some conditions the experimental results demonstrate that the proposed metho

diagnostics for heteroscedasticity in regression summary for the usual regression model without replication we provide a diagnostic test for heteroscedasticity based on the score statistic a graphical procedure to complement the score test is also presented some key u ords influence linear model residual score test diagnostic methods in linear regression are used to examine the appropriateness of assumptions underlying the modelling process and to locate unusual characteristics of the data that may influence conclusions the recent literature on diagnostics is dominated by studies of methods for the detection of influential observations cook weisberg 1982 provide a review diagnostics for the relevance of specific assumptions however have not received the same degree of attention even though these may be of equal importance our purpose here is to provide appropriate diagnostic techniques to aid in an assessment of the validity of the usual assumption of homoscedasticity when little or no replication is present available methods for studying this assumption include both graphical and nongraphical procedures the usual graphical procedure consists of plotting the ordinary least squares residuals against fitted values or an explanatory variable a megaphone shaped pattern is taken as evidence that the variance depends on the quantity plotted on tlle abscissa weisberg 1980 chapter 6 in 3 we suggest several ways in which this

on the interpretation of regression plots abstract a framework is developed for the interpretation of regression plots including plots of the response against selected covariates residual plots added variable plots and detrended added variable plots it is shown that many of the common interpretations associated with these plots can be misleading the framework also allows for the generalization of standard plots and the development of new plotting paradigms a paradigm for graphical exploration of regression problems is sketched

sufficient dimension reduction for spatial point processes directed by gaussian random fields summary we develop a sufficient dimension reduction paradigm for inhomogeneous spatial point processes driven by gaussian random fields specifically we introduce the notion of the kth order central intensity subspace we show that a central subspace can be defined as the combination of all central intensity subspaces for many commonly used spatial point process models we find that the central subspace is equivalent to the first order central intensity subspace to estimate the latter we propose a flexible framework under which most existing benchmark inverse regression methods can be extended to the spatial point process setting we develop novel graphical and formal testing methods to determine the structural dimension of the central subspace these methods are extremely versatile in that they do not require any specific model assumption on the correlation structures of the covariates and the spatial point process to illustrate the practical use of the methods proposed we apply them to both simulated data and two real examples

principal hessian directions revisited abstract li has suggested the method of principal hessian directions for estimating plotting directions that capture curvature in regression functions this article revisits li s proposal offering a number of suggestions for improved application

assessing the number of linear components in a general regression problem abstract this article presents a method for testing the dimension of a general regression model without assuming any specific model for the distribution of the regressors examples of application and comparisons with existing methods are also studied

dimension reduction in regressions through cumulative slicing estimation in this paper we offer a complete methodology of cumulative slicing estimation to sufficient dimension reduction in parallel to the classical slicing estimation we develop three methods that are termed respectively as cumulative mean estimation cumulative variance estimation and cumulative directional regression the strong consistency for p o n1 2 log n and the asymptotic normality for p o n1 2 are established where p is the dimension of the predictors and n is sample size such asymptotic results improve the rate p o n1 3 in many existing contexts of semiparametric modeling in addition we propose a modified bic type criterion to estimate the structural dimension of the central subspace its consistency is established when p o n1 2 extensive simulations are carried out for comparison with existing methods and a real data example is presented for illustration

sufficient dimension reduction via inverse regression a family of dimension reduction methods the inverse regression ir family is developed by minimizing a quadratic objective function an optimal member of this family the inverse regression estimator ire is proposed along with inference methods and a computational algorithm the ire has at least three desirable properties 1 its estimated basis of the central dimension reduction subspace is asymptotically efficient 2 its test statistic for dimension has an asymptotic chi squared distribution and 3 it provides a chi squared test of the conditional independence hypothesis that the response is independent of a selected subset of predictors given the remaining predictors current methods like sliced inverse regression belong to a suboptimal class of the ir family comparisons of these methods are reported through simulation studies the approach developed here also allows a relatively straightforward derivation of the asymptotic null distribution of the test statistic for dimension used in slic

graphics for regressions with a binary response abstract central dimension reduction subspaces which characterize the dependence of a response variable on one or more predictors are developed and then used to guide the construction and interpretation of graphics for regression problems with a binary response variable graphical methods requiring neither a link function nor residuals are suggested for both development and criticism of model components implied by the central dimension reduction subspace

bootstrap testing of the rank of a matrix via least squared constrained estimation to test if an unknown matrix m0 has a given rank null hypothesis noted h0 we consider a statistic that is a squared distance between an estimator and the submanifold of fixed rank matrix under h0 this statistic converges to a weighted chi squared distribution we introduce the constrained bootstrap cs bootstrap to estimate the law of this statistic under h0 an important point is that even if h0 fails the cs bootstrap reproduces the behavior of the statistic under h0 as a consequence the cs bootstrap is employed to estimate the nonasymptotic quantile for testing the rank we provide the consistency of the procedure and the simulations shed light on the accuracy of the cs bootstrap with respect to the traditional asymptotic comparison more generally the results are extended to test whether an unknown parameter belongs to a submanifold of the euclidean space finally the cs bootstrap is easy to compute it handles a large family of tests and it works under mild assumptions

improved convergence rates for intermediate problems improved convergence rate estimates are derived for a variant of aronszajn type intermediate problems that is both computationally feasible and convergent for problems with nontrivial essential spectra in a previous paper the authors obtained rate of convergence estimates for this method in terms of containment gaps between subspaces in the present work techniques for estimating relatively unbounded perturbations are refined in order to apply the kato temple inequalities this yields convergence rates for the intermediate operator eigenvalues in terms of squares of containment gaps between subspaces convergence rate estimates are also obtained for the intermediate problem eigenvectors and comparisons are made with previously known results for the method of special choice

on partial sufficient dimension reduction with applications to partially linear multi index models partial dimension reduction is a general method to seek informative convex combinations of predictors of primary interest which includes dimension reduction as its special case when the predictors in the remaining part are constants in this article we propose a novel method to conduct partial dimension reduction estimation for predictors of primary interest without assuming that the remaining predictors are categorical to this end we first take the dichotomization step such that any existing approach for partial dimension reduction estimation can be employed then we take the expectation step to integrate over all the dichotomic predictors to identify the partial central subspace as an example we use the partially linear multi index model to illustrate its applications for semiparametric modeling simulations and real data examples are given to illustrate our methodology

extending sliced inverse regression sliced inverse regression sir and an associated chi squared test for dimension have been introduced as a method for reducing the dimension of regression problems whose predictor variables are normal in this article the assumptions on the predictor distribution under which the chi squared test was proved to apply are relaxed and the result is extended a general weighted chi squared test that does not require normal regressors for the dimension of a regression is given simulations show that the weighted chi squared test is more reliable than the chi squared test when the regressor distribution digresses from normality significantly and that it compares well with the chi squared test when the regressors are normal