testing predictor contributions in sufficient dimension reduction we develop tests of the hypothesis of no effect for selected predictors in regression without assuming a model for the conditional distribution of the response given the predictors predictor effects need not be limited to the mean function and smoothing is not required the general approach is based on sufficient dimension reduction the idea being to replace the predictor vector with a lower dimensional version without loss of information on the regression methodology using sliced inverse regression is developed in detail

universal fir mmse filtering we consider the problem of causal estimation i e filtering of a real valued signal corrupted by zero mean time independent real valued additive noise under the mean squared error mse criterion we build a universal filter whose per symbol squared error for every bounded underlying signal is essentially as small as that of the best finite duration impulse response fir filter of a given order we do not assume a stochastic mechanism generating the underlying signal and assume only that the variance of the noise is known to the filter the regret of the expected mse of our scheme is shown to decay as o logn n where n is the length of the signal moreover we present a stronger concentration result which guarantees the performance of our scheme not only in expectation but also with high probability our result implies a conventional stochastic setting result i e when the underlying signal is a stationary process our filter achieves the performance of the optimal fir filter we back our theoretical findings with several experiments showcasing the potential merits of our universal filter in practice our analysis combines tools from the problems of universal filtering and competitive online regression

likelihood based sufficient dimension reduction we obtain the maximum likelihood estimator of the central subspace under conditional normality of the predictors given the response analytically and in simulations we found that our new estimator can preform much better than sliced inverse regression sliced average variance estimation and directional regression and that it seems quite robust to deviations from normality

identifying regression outliers and mixtures graphically regressions in practice can include outliers and other unknown subpopulation structures for example mixtures of regressions occur if there is an omitted categorical predictor like gender or location and different regressions occur within each category the theory of regression graphics based on central subspaces can be used to construct graphical solutions to long standing problems of this type it is argued that in practice the central subspace automatically expands to incorporate outliers and regression mixtures thus methods of estimating the central subspace can be used to identify these phenomena without specifying a model examples illustrating the power of the theory are presented

sufficient dimension reduction for longitudinally measured predictors we propose a method to combine several predictors markers that are measured repeatedly over time into a composite marker score without assuming a model and only requiring a mild condition on the predictor distribution assuming that the first and second moments of the predictors can be decomposed into a time and a marker component via a kronecker product structure that accommodates the longitudinal nature of the predictors we develop first moment sufficient dimension reduction techniques to replace the original markers with linear transformations that contain sufficient information for the regression of the predictors on the outcome these linear combinations can then be combined into a score that has better predictive performance than a score built under a general model that ignores the longitudinal structure of the data our methods can be applied to either continuous or categorical outcome measures in simulations we focus on binary outcomes and show that our method outperforms existing alternatives by using the auc the area under the receiver operator characteristics roc curve as a summary measure of the discriminatory ability of a single continuous diagnostic marker for binary disease outcomes published 2011 this article is a us government work and is in the public domain in the usa

on three term conjugate gradient algorithms for unconstrained optimization this paper presents a project for three term conjugate gradient algorithms development the search direction of the algorithms from this class has three terms and is computed as modifications of the classical conjugate gradient algorithms to satisfy both the descent and the conjugacy conditions an example of three term conjugate gradient algorithm from this class as modifications of the classical and well known hestenes and stiefel or of the cg descent by hager and zhang conjugate gradient algorithms satisfying both the descent and the conjugacy conditions is presented these properties are independent of the line search also the algorithm could be considered as a modification of the memoryless bfgs quasi newton method the new approximation of the minimum is obtained by the general wolfe line search and using by now a standard acceleration technique developed by andrei the proposed three term conjugate gradient algorithm substantially outperforms the well known hestenes and stiefel conjugate gradient algorithm as well as the more elaborated cg descent algorithm a number of 5 applications from minpack 2 test problems collection with 10 6 variables prove that the suggested three term conjugate gradient algorithm is top performer versus cg descent

linear minimax regret estimation of deterministic parameters with bounded data uncertainties we develop a new linear estimator for estimating an unknown parameter vector x in a linear model in the presence of bounded data uncertainties the estimator is designed to minimize the worst case regret over all bounded data vectors namely the worst case difference between the mean squared error mse attainable using a linear estimator that does not know the true parameters x and the optimal mse attained using a linear estimator that knows x we demonstrate through several examples that the minimax regret estimator can significantly increase the performance over the conventional least squares estimator as well as several other least squares alternatives

trace pursuit a general framework for model free variable selection abstractwe propose trace pursuit for model free variable selection under the sufficient dimension reduction paradigm two distinct algorithms are proposed stepwise trace pursuit and forward trace pursuit stepwise trace pursuit achieves selection consistency with fixed p forward trace pursuit can serve as an initial screening step to speed up the computation in the case of ultrahigh dimensionality the screening consistency property of forward trace pursuit based on sliced inverse regression is established finite sample performances of trace pursuit and other model free variable selection methods are compared through numerical studies supplementary materials for this article are available online

marginal tests with sliced average variance estimation summary we present a new computationally feasible test for the dimension of the central subspace in a regression problem based on sliced average variance estimation we also provide a marginal coordinate test under the null hypothesis both the test of dimension and the marginal coordinate test involve test statistics that asymptotically have chi squared distributions given normally distributed predictors and have a distribution that is a linear combination of chi squared distributions in general

sliced inverse regression for dimension reduction abstract modern advances in computing power have greatly widened scientists scope in gathering and investigating information from many variables information which might have been ignored in the past yet to effectively scan a large pool of variables is not an easy task although our ability to interact with data has been much enhanced by recent innovations in dynamic graphics in this article we propose a novel data analytic tool sliced inverse regression sir for reducing the dimension of the input variable x without going through any parametric or nonparametric model fitting process this method explores the simplicity of the inverse view of regression that is instead of regressing the univariate output variable y against the multivariate x we regress x against y forward regression and inverse regression are connected by a theorem that motivates this method the theoretical properties of sir are investigated under a model of the form y f 1 x k x e where the k s are the unknown

graphical tools for quadratic discriminant analysis sufficient dimension reduction methods provide effective ways to visualize discriminant analysis problems for example cook and yin showed that the dimension reduction method of sliced average variance estimation save identifies variates that are equivalent to a quadratic discriminant analysis qda solution this article makes this connection explicit to motivate the use of save variates in exploratory graphics for discriminant analysis classification can then be based on the save variates using a suitable distance measure if the chosen measure is mahalanobis distance then classification is identical to qda using the original variables just as canonical variates provide a useful way to visualize linear discriminant analysis lda so do save variates help visualize qda this would appear to be particularly useful given the lack of graphical tools for qda in current software furthermore whereas lda and qda can be sensitive to nonnormality save is more robust

universal prediction of individual binary sequences in the presence of noise the problem of predicting the next outcome of an individual binary sequence based on noisy observations of the past is considered the goal of the predictor is to perform for each individual sequence almost as well as the best in a set of experts where performance is evaluated using a general loss function a comprehensive approach to prediction in this noisy setting is presented and proven generally efficient under appropriate conditions as an illustration of the applicability of the approach suggested for concrete situations two important special cases are explicitly treated the first is the case where the data corrupting noise process is binary valued where the observed bit is the bitwise xor of the clean bit and the noise bit the second case is that of real valued additive noise it is shown that even in this more challenging situation where the information available to the predictor regarding the past sequence is incomplete a predictor can be guaranteed to successfully compete with a whole set of experts in considerably strong senses

universal linear least squares prediction upper and lower bounds we consider the problem of sequential linear prediction of real valued sequences under the square error loss function for this problem a prediction algorithm has been demonstrated whose accumulated squared prediction error for every bounded sequence is asymptotically as small as the best fixed linear predictor for that sequence taken from the class of all linear predictors of a given order p the redundancy or excess prediction error above that of the best predictor for that sequence is upper bounded by a sup 2 p ln n n where n is the data length and the sequence is assumed to be bounded by some a we provide an alternative proof of this result by connecting it with universal probability assignment we then show that this predictor is optimal in a min max sense by deriving a corresponding lower bound such that no sequential predictor can ever do better than a redundancy of a sup 2 p ln n n

sufficient dimension reduction for spatial point processes directed by gaussian random fields summary we develop a sufficient dimension reduction paradigm for inhomogeneous spatial point processes driven by gaussian random fields specifically we introduce the notion of the kth order central intensity subspace we show that a central subspace can be defined as the combination of all central intensity subspaces for many commonly used spatial point process models we find that the central subspace is equivalent to the first order central intensity subspace to estimate the latter we propose a flexible framework under which most existing benchmark inverse regression methods can be extended to the spatial point process setting we develop novel graphical and formal testing methods to determine the structural dimension of the central subspace these methods are extremely versatile in that they do not require any specific model assumption on the correlation structures of the covariates and the spatial point process to illustrate the practical use of the methods proposed we apply them to both simulated data and two real examples

on the variation of the spectra of matrices abstract several distances between the spectra of two matrices are discussed and compared optimal bounds are given which enable us to reduce certain bounds on the eigenvalue variation of matrices by a factor of about two the results of bhatia and mukherjea and of bhatia and friedland on the eigenvalue variation are derived in an elementary way using results of henrici on the spectral variation

universal filtering via hidden markov modeling the problem of discrete universal filtering in which the components of a discrete signal emitted by an unknown source and corrupted by a known discrete memoryless channel dmc are to be causally estimated is considered a family of filters are derived and are shown to be universally asymptotically optimal in the sense of achieving the optimum filtering performance when the clean signal is stationary ergodic and satisfies an additional mild positivity condition our schemes are comprised of approximating the noisy signal using a hidden markov process hmp via maximum likelihood ml estimation followed by the use of the forward recursions for hmp state estimation it is shown that as the data length increases and as the number of states in the hmp approximation increases our family of filters attains the performance of the optimal distribution dependent filter an extension to the case of channels with memory is also established

adapting to unknown smoothness via wavelet shrinkage abstract we attempt to recover a function of unknown smoothness from noisy sampled data we introduce a procedure sureshrink that suppresses noise by thresholding the empirical wavelet coefficients the thresholding is adaptive a threshold level is assigned to each dyadic resolution level by the principle of minimizing the stein unbiased estimate of risk sure for threshold estimates the computational effort of the overall procedure is order n log n as a function of the sample size n sureshrink is smoothness adaptive if the unknown function contains jumps then the reconstruction essentially does also if the unknown function has a smooth piece then the reconstruction is essentially as smooth as the mother wavelet will allow the procedure is in a sense optimally smoothness adaptive it is near minimax simultaneously over a whole interval of the besov scale the size of this interval depends on the choice of mother wavelet we know from a previous paper by the authors that traditional smoot

on post dimension reduction statistical inference the methodologies of sufficient dimension reduction have undergone extensive developments in the past three decades however there has been a lack of systematic and rigorous development of post dimension reduction inference which has seriously hindered its applications the current common practice is to treat the estimated sufficient predictors as the true predictors and use them as the starting point of the downstream statistical inference however this naive inference approach would grossly overestimate the confidence level of an interval or the power of a test leading to the distorted results in this paper we develop a general and comprehensive framework of post dimension reduction inference which can accommodate any dimension reduction method and model building method as long as their corresponding influence functions are available within this general framework we derive the influence functions and present the explicit post reduction formulas for the combinations of numerous dimension reduction and model building methods we then develop post reduction inference methods for both confidence interval and hypothesis testing we investigate the finite sample performance of our procedures by simulations and a real data analysis

asymptotic inference for eigenvectors abstract asymptotic procedures are given for testing certain hypotheses concerning eigenvectors and for constructing confidence regions for eigenvectors these asymptotic procedures are derived under fairly general conditions on the estimates of the matrix whose eigenvectors are of interest applications of the general results to principal components analysis and canonical variate analysis are given author

sufficient dimension reduction via inverse regression a family of dimension reduction methods the inverse regression ir family is developed by minimizing a quadratic objective function an optimal member of this family the inverse regression estimator ire is proposed along with inference methods and a computational algorithm the ire has at least three desirable properties 1 its estimated basis of the central dimension reduction subspace is asymptotically efficient 2 its test statistic for dimension has an asymptotic chi squared distribution and 3 it provides a chi squared test of the conditional independence hypothesis that the response is independent of a selected subset of predictors given the remaining predictors current methods like sliced inverse regression belong to a suboptimal class of the ir family comparisons of these methods are reported through simulation studies the approach developed here also allows a relatively straightforward derivation of the asymptotic null distribution of the test statistic for dimension used in slic

graphics for regressions with a binary response abstract central dimension reduction subspaces which characterize the dependence of a response variable on one or more predictors are developed and then used to guide the construction and interpretation of graphics for regression problems with a binary response variable graphical methods requiring neither a link function nor residuals are suggested for both development and criticism of model components implied by the central dimension reduction subspace

extending sliced inverse regression sliced inverse regression sir and an associated chi squared test for dimension have been introduced as a method for reducing the dimension of regression problems whose predictor variables are normal in this article the assumptions on the predictor distribution under which the chi squared test was proved to apply are relaxed and the result is extended a general weighted chi squared test that does not require normal regressors for the dimension of a regression is given simulations show that the weighted chi squared test is more reliable than the chi squared test when the regressor distribution digresses from normality significantly and that it compares well with the chi squared test when the regressors are normal

universal prediction of individual sequences the problem of predicting the next outcome of an individual binary sequence using finite memory is considered the finite state predictability of an infinite sequence is defined as the minimum fraction of prediction errors that can be made by any finite state fs predictor it is proven that this fs predictability can be achieved by universal sequential prediction schemes an efficient prediction procedure based on the incremental parsing procedure of the lempel ziv data compression algorithm is shown to achieve asymptotically the fs predictability some relations between compressibility and predictability are discussed and the predictability is proposed as an additional measure of the complexity of a sequence