a note on the unification of the akaike information criterion to measure the distance between a robust function evaluated under the true regression model and under a fitted model we propose generalized kullback leibler information using this generalization we have developed three robust model selection criteria aicr aiccr and aiccr that allow the selection of candidate models that not only fit the majority of the data but also take into account non normally distributed errors the aicr and aiccr criteria can unify most existing akaike information criteria three examples of such unification are given simulation studies are presented to illustrate the relative performance of each criterion

manifold regularization a geometric framework for learning from labeled and unlabeled examples we propose a family of learning algorithms based on a new form of regularization that allows us to exploit the geometry of the marginal distribution we focus on a semi supervised framework that incorporates labeled and unlabeled data in a general purpose learner some transductive graph learning algorithms and standard methods including support vector machines and regularized least squares can be obtained as special cases we use properties of reproducing kernel hilbert spaces to prove new representer theorems that provide theoretical basis for the algorithms as a result in contrast to purely graph based approaches we obtain a natural out of sample extension to novel examples and so are able to handle both transductive and truly semi supervised settings we present experimental evidence suggesting that our semi supervised algorithms are able to use unlabeled data effectively finally we have a brief discussion of unsupervised and fully supervised learning within our general framework

unified framework for representing and ranking in the database retrieval and nearest neighbor classification tasks the two basic problems are to represent the query and database objects and to learn the ranking scores of the database objects to the query many studies have been conducted for the representation learning and the ranking score learning problems however they are always learned independently from each other in this paper we argue that there are some inner relationships between the representation and ranking of database objects and try to investigate their relationships by learning them in a unified way to this end we proposed the unified framework for representation and ranking ur2 of objects for the database retrieval and nearest neighbor classification tasks the learning of representation parameter and the ranking scores are modeled within one single unified objective function the objective function is optimized alternately with regard to representation parameter and the ranking scores based on the optimization results iterative algorithms are developed to learn the representation parameter and the ranking scores on a unified way moreover with two different formulas of representation feature selection and subspace learning we give two versions of ur2 the proposed algorithms are tested on two challenging tasks mri image based brain tumor retrieval and nearest neighbor classification based protein identification the experiments show the advantage of the proposed unified framework over the state of the art independent representation and ranking methods highlightsobject representation and ranking are two basic problems for database retrieval they are always studies separately we proposed the first unified framework for both representation and ranking experimental results show its advantage over independent representation or ranking methods

robust linear model selection based on least angle regression in this article we consider the problem of building a linear prediction model when the number of candidate predictors is large and the data possibly contain anomalies that are difficult to visualize and clean we want to predict the nonoutlying cases therefore we need a method that is simultaneously robust and scalable we consider the stepwise least angle regression lars algorithm which is computationally very efficient but sensitive to outliers we introduce two different approaches to robustify lars the plug in approach replaces the classical correlations in lars by robust correlation estimates the cleaning approach first transforms the data set by shrinking the outliers toward the bulk of the data which we call multivariate winsorization and then applies lars to the transformed data we show that the plug in approach is time efficient and scalable and that the bootstrap can be used to stabilize its results we recommend using bootstrapped robustified lars to sequence a number of candidate pred

nonsmooth analysis and frechet differentiability of m functionals abstract a necessary requirement for existence of the frechet derivative is that the defining psi function is uniformly bounded and this naturally excludes those nonrobust estimators such as the maximum likelihood estimator in normal parametric models in this paper the methods of nonsmooth analysis described in the book by f h clarke 1983 are introduced to the theory of statistical expansions and are used here in the proofs of weak continuity and frechet differentiability of m functionals subsequently the conditions for frechet differentiability given in clarke 1983 can be relaxed to include most popular m functionals additional keywords distribution functions m estimators robustness gross error sensitivity weak continuity asymptotic expansions asymptotic normality selection functional local uniqueness author

breakdown robustness of tests abstract power and level breakdown functions of a test statistic are introduced to obtain a unified analysis of local stability as typified by the influence function and global stability as illustrated by the breakdown point the power breakdown function gives the amount of contamination of each alternative distribution that can carry the test statistic to a null value the level breakdown function gives the amount of contamination of a null distribution that can carry the test statistic to each value in the alternative space in many cases the breakdown function bears an inverse relationship to the maximum bias as a function of the contamination as the maximum bias is not invariant to one to one transformations however equivalent test statistics can have different scales for bias a distinct advantage of the inverse scale is the invariance of the breakdown functions to such transformations the resulting stability curves depend on the parameters in a way analogous to the power curve the slopes o

variable selection for 1d regression models variable selection the search for j relevant predictor variables from a group of p candidates is a standard problem in regression analysis the class of 1d regression models is a broad class that includes generalized linear models we show that existing variable selection algorithms originally meant for multiple linear regression and based on ordinary least squares and mallows s cp can also be used for 1d models graphical aids for variable selection are also provided

robust model selection criteria for robust liu estimator in linear regression analysis outliers often have large influence in the model variable selection process the aim of this study is to select the subsets of independent variables which explain dependent variables in the presence of multicollinearity outliers and possible departures from the normality assumption of the error distribution in robust regression analysis in this study to overcome this combined problem of multicollinearity and outliers we suggest to use robust selection criterion with liu and liu type m lm estimators

forward search added variable t tests and the effect of masked outliers on model selection summary monitoring the t tests for individual regression coefficients in forward search fails to identify the importance of observations to the significance of the individual regressors this failure is due to the ordering of the data by the search we introduce an added variable test which has the desired properties since the projection leading to residuals destroys the effect of the ordering an example illustrates the effect of several masked outliers on model selection comments are given on the related test for response transformations

robust inference with gmm estimators the local robustness properties of generalized method of moments gmm estimators and of a broad class of gmm based tests are investigated in a unified framework gmm statistics are shown to have bounded influence if and only if the function defining the orthogonality restrictions imposed on the underlying model is bounded since in many applications this function is unbounded it is useful to have procedures that modify the starting orthogonality conditions in order to obtain a robust version of a gmm estimator or test we show how this can be obtained when a reference model for the data distribution can be assumed we develop a flexible algorithm for constructing a robust gmm rgmm estimator leading to stable gmm test statistics the amount of robustness can be controlled by an appropriate tuning constant we relate by an explicit formula the choice of this constant to the maximal admissible bias on the level or and the power of a gmm test and the amount of contamination that one can reasonably assume given some information on the data finally we illustrate the rgmm methodology with some simulations of an application to rgmm testing for conditional heteroscedasticity in a simple linear autoregressive model in this example we find a significant instability of the size and the power of a classical gmm testing procedure under a non normal conditional error distribution on the other side the rgmm testing procedures can control the size and the power of the test under non standard conditions while maintaining a satisfactory power under an approximatively normal conditional error distribution

controlling variable selection by the addition of pseudovariables we propose a new approach to variable selection designed to control the false selection rate fsr defined as the proportion of uninformative variables included in selected models the method works by adding a known number of pseudovariables to the real dataset running a variable selection procedure and monitoring the proportion of pseudovariables falsely selected information obtained from bootstrap like replications of this process is used to estimate the proportion of falsely selected real variables and to tune the selection procedure to control the fsr

a robust version of mallows s c p abstract we present a robust version of mallows s c p for regression models it is defined by rc p w p 2 u p v p where w p i 2 i r 2 i is a weighted residual sum of squares computed from a robust fit of model p 2 is a robust and consistent estimator of 2 in the full model and u p and v p are constants depending on the weight function and the number of parameters in model p good subset models are those with rc p close to v p or smaller than v p when the weights are identically 1 w p becomes the residual sum of squares of a least squares fit and rc p reduces to mallows s c p the robust model selection procedure based on rc p allows us to choose the models that fit the majority of the data by taking into account the presence of outliers and possible departures from the normality assumption on the error distribution together with the classical c p the robust version suggests several models from which we can choose

rigidity for equivariant pseudo pretheories abstract we prove versions of the suslin and gabber rigidity theorems in the setting of equivariant pseudo pretheories of smooth schemes over a field with an action of a finite group examples include equivariant algebraic k theory presheaves with equivariant transfers equivariant suslin homology and bredon motivic cohomology

weighted likelihood equations with bootstrap root search abstract we discuss a method of weighting likelihood equations with the aim of obtaining fully efficient and robust estimators we discuss the case of continuous probability models using unimodal weighting functions these weighting functions downweight observations that are inconsistent with the assumed model at the true model therefore the proposed estimating equations behave like the ordinary likelihood equations we investigate the number of solutions of the estimating equations via a bootstrap root search the estimators obtained are consistent and asymptotically normal and have desirable robustness properties an extensive simulation study and real data examples illustrate the operating characteristics of the proposed methodology

robust inference for generalized linear models by starting from a natural class of robust estimators for generalized linear models based on the notion of quasi likelihood we define robust deviances that can be used for stepwise model selection as in the classical framework we derive the asymptotic distribution of tests based on robust deviances and we investigate the stability of their asymptotic level under contamination the binomial and poisson models are treated in detail two applications to real data and a sensitivity analysis show that the inference obtained by means of the new techniques is more reliable than that obtained by classical estimation and testing procedures

active learning based on locally linear reconstruction we consider the active learning problem which aims to select the most representative points out of many existing active learning techniques optimum experimental design oed has received considerable attention recently the typical oed criteria minimize the variance of the parameter estimates or predicted value however these methods see only global euclidean structure while the local manifold structure is ignored for example i optimal design selects those data points such that other data points can be best approximated by linear combinations of all the selected points in this paper we propose a novel active learning algorithm which takes into account the local structure of the data space that is each data point should be approximated by the linear combination of only its neighbors given the local reconstruction coefficients for every data point and the coordinates of the selected points a transductive learning algorithm called locally linear reconstruction llr is proposed to reconstruct every other point the most representative points are thus defined as those whose coordinates can be used to best reconstruct the whole data set the sequential and convex optimization schemes are also introduced to solve the optimization problem the experimental results have demonstrated the effectiveness of our proposed method

a statistical view of some chemometrics regression tools chemometrics is a field of chemistry that studies the application of statistical methods to chemical data analysis in addition to borrowing many techniques from the statistics and engineering literatures chemometrics itself has given rise to several new data analytical methods this article examines two methods commonly used in chemometrics for predictive modeling partial least squares and principal components regression from a statistical perspective the goal is to try to understand their apparent successes and in what situations they can be expected to work well and to compare them with other statistical methods intended for those situations these methods include ordinary least squares variable subset selection and ridge regression

fast robust correlation for high dimensional data the product moment covariance matrix is a cornerstone of multivariate data analysis from which one can derive correlations principal components mahalanobis distances and many other results unfo

high breakdown inference for mixed linear models mixed linear models are used to analyze data in many settings these models have a multivariate normal formulation in most cases the maximum likelihood estimator mle or the residual mle reml is usually chosen to estimate the parameters however the latter are based on the strong assumption of exact multivariate normality welsh and richardson have shown that these estimators are not robust to small deviations from multivariate normality this means that in practice a small proportion of data even only one can drive the value of the estimates on their own because the model is multivariate we propose a high breakdown robust estimator for very general mixed linear models that include for example covariates this robust estimator belongs to the class of s estimators from which we can derive asymptotic properties for inference we also use it as a diagnostic tool to detect outlying subjects we discuss the advantages of this estimator compared with other robust estimators proposed previously and i

robust estimates of location and dispersion for high dimensional datasets the computing times of high breakdown point estimates of multivariate location and scatter increase rapidly with the number of variables which makes them impractical for high dimensional datasets such as those used in data mining we propose an estimator of location and scatter based on a modified version of the gnanadesikan kettenring robust covariance estimate we compare its behavior with that of the stahel donoho sd and rousseeuw and van driessen s fast mcd fmcd estimates in simulations with contaminated multivariate normal data our estimate is almost as good as sd and clearly better than fmcd it is much faster than both especially for large dimension we give examples with real data with dimensions between 5 and 93 in which the proposed estimate is as good as or better than sd and fmcd at detecting outliers and other structures with much shorter computing times

some comments on cp i study the typical configuration of a cp plot when the number of variables in the regression problem is large and there are many weak effects i show that a particular configuration that is very commonly seen can arise in a simple way i give a formula by means of which the risk incurred by the minimum cp rule can be estimated

generalised information criteria in model selection summary the problem of evaluating the goodness of statistical models is investigated from an information theoretic point of view information criteria are proposed for evaluating models constructed by various estimation procedures when the specified family of probability distributions does not contain the distribution generating the data the proposed criteria are applied to the evaluation of models estimated by maximum likelihood robust penalised likelihood bayes procedures etc we also discuss the use of the bootstrap in model evaluation problems and present a variance reduction technique in the bootstrap simulation

higher algebraic k theory for actions of diagonalizable groups we study the k theory of actions of diagonalizable group schemes on noetherian regular separated algebraic spaces our main result shows how to reconstruct the k theory ring of such an action from the k theory rings of the loci where the stabilizers have constant dimension we apply this to the calculation of the equivariant k theory of toric varieties and give conditions under which the merkurjev spectral sequence degenerates so that the equivariant k theory ring determines the ordinary k theory ring we also prove a very refined localization theorem for actions of this type

on one step gm estimates and stability of inferences in linear regression abstract the folklore on one step estimation is that it inherits the breakdown point of the preliminary estimator and yet has the same large sample distribution as the fully iterated version as long as the preliminary estimate converges faster than n 1 4 where n is the sample size we investigate the extent to which this folklore is valid for one step gm estimators and their associated standard errors in linear regression we find that one step gm estimates based on newton raphson or scoring inherit the breakdown point of high breakdown point initial estimates such as least median of squares provided the usual weights that limit the influence of extreme points in the design space are based on location and scatter estimates with high breakdown points moreover these estimators have bounded influence functions and their standard errors can have high breakdown points the folklore concerning the large sample theory is correct assuming the regression errors are symmetrically distributed and homoscedastic

least median of squares regression abstract classical least squares regression consists of minimizing the sum of the squared residuals many authors have produced more robust versions of this estimator by replacing the square by something else such as the absolute value in this article a different approach is introduced in which the sum is replaced by the median of the squared residuals the resulting estimator can resist the effect of nearly 50 of contamination in the data in the special case of simple regression it corresponds to finding the narrowest strip covering half of the observations generalizations are possible to multivariate location orthogonal regression and hypothesis testing in linear models

a bounded influence high breakdown efficient regression estimator abstract we consider the multiple linear regression model y i x i e i i 1 2 n with random carriers and focus on the estimation of this article s main contribution is to present an estimator that is affine regression and scale equivariant has both a high breakdown point and a bounded influence function and has an asymptotic efficiency greater than 95 versus least squares under gaussian errors we give conditions under which the estimator a one step general m estimator that uses schweppe weights and is based on a high breakdown initial estimator satisfies these properties the major conditions necessary for the estimator are a it must be based on a n consistent initial estimator with a 50 breakdown point and b it must be based on a psi function that is odd bounded and strictly increasing the advantage of this estimator over previous approaches is that it does not downweight high leverage points without first considering how they fit the bulk of the data methods of computi

conditionally unbiased bounded influence estimation in general regression models with applications to generalized linear models abstract in this article robust estimation in generalized linear models for the dependence of a response y on an explanatory variable x is studied a subclass of the class of m estimators is defined by imposing the restriction that the score function must be conditionally unbiased given x within this class of conditionally fisher consistent estimators optimal bounded influence estimators of regression parameters are identified and their asymptotic properties are studied the estimators studied in this article and the efficient bounded influence estimators studied by stefanski carroll and ruppert 1986 depend on an auxiliary centering constant and nuisance matrix the centering constant can be given explicitly for the conditionally fisher consistent estimators and thus they are easier to compute than the estimators studied by stefanski et al 1986 in addition estimation of the nuisance matrix has no effect on the asymptotic distribution of the conditionally fisher consistent estimators the sam

robust indirect inference in this article we develop robust indirect inference for a variety of models in a unified framework we investigate the local robustness properties of indirect inference and derive the influence function of the indirect estimator as well as the level and power influence functions of indirect tests these tools are then used to design indirect inference procedures that are stable in the presence of small deviations from the assumed model although indirect inference was originally proposed for statistical models whose likelihood is difficult or even impossible to compute and or to maximize we use it here as a device to robustify the estimators and tests for models where this is not possible or is difficult with classical techniques such as m estimators examples from financial applications time series and spatial statistics are used for illustration

gersten conjecture for equivariant k theory and applications for a reductive group scheme g over a regular semi local ring a we prove the gersten conjecture for the equivariant k theory as a consequence we show that if f is the field of fractions of a then k g 0 a cong k g 0 f generalizing the analogous result for a dvr by serre inst hautes etudes sci publ math 34 37 52 1968 we also show the rigidity for the k theory with finite coefficients of a henselian local ring in the equivariant setting we use this rigidity theorem to compute the equivariant k theory of algebraically closed fields

fast robust model selection in large datasets large datasets are increasingly common in many research fields in particular in the linear regression context it is often the case that a huge number of potential covariates are available to explain a response variable and the first step of a reasonable statistical analysis is to reduce the number of covariates this can be done in a forward selection procedure that includes selecting the variable to enter deciding to retain it or stop the selection and estimating the augmented model least squares plus t tests can be fast but the outcome of a forward selection might be suboptimal when there are outliers in this article we propose a complete algorithm for fast robust model selection including considerations for huge sample sizes because simply replacing the classical statistical criteria with robust ones is not computationally possible we develop simplified robust estimators selection criteria and testing procedures for linear regression the robust estimator is a one step weighted m estimato

weighted wilcoxon type smoothly clipped absolute deviation method summary shrinkage type variable selection procedures have recently seen increasing applications in biomedical research however their performance can be adversely influenced by outliers in either the response or the covariate space this article proposes a weighted wilcoxon type smoothly clipped absolute deviation ww scad method which deals with robust variable selection and robust estimation simultaneously the new procedure can be conveniently implemented with the statistical software r we establish that the ww scad correctly identifies the set of zero coefficients with probability approaching one and estimates the nonzero coefficients with the rate n 1 2 moreover with appropriately chosen weights the ww scad is robust with respect to outliers in both the x and y directions the important special case with constant weights yields an oracle type estimator with high efficiency in the presence of heavier tailed random errors the robustness of the ww scad is partly justified by its asymptotic performance under local shrinking contamination we propose a bayesian information criterion type tuning parameter selector for the ww scad the performance of the ww scad is demonstrated via simulations and by an application to a study that investigates the effects of personal characteristics and dietary factors on plasma beta carotene level

outlier robust model selection in linear regression we propose a new approach to the selection of regression models based on combining a robust penalized criterion and a robust conditional expected prediction loss function that is estimated using a stratified bootstrap both components of the procedure use robust criteria i e robust functions rather than squared error loss to reduce the effects of large residuals and poor bootstrap samples a key idea is to separate estimation from model selection by choosing estimators separately from the function using the stratified bootstrap further reduces the likelihood of obtaining poor bootstrap samples we show that the model selection procedure is consistent under some conditions and works well in our simulations in particular we find that simultaneous minimization of prediction error and conditional expected prediction loss is better than separate minimization of the prediction error or the conditional expected prediction loss

robust regression for clustered data with application to binary responses summary generalized estimating equations gee can be highly influenced by the presence of unusual data points a generalization of the gee procedure which yields parameter estimates and fitted values that are resistant to influential data is introduced resistant generalized estimating equations regee include weights in the estimating equations to downweight influential observations or clusters influential observations are downweighted according to their leverage or residual in an example of correlated binary regression applied to 137 urinary incontinent elderly patients from 38 medical practices

resistant fits for some commonly used logistic models with medical application logistic regression type models are used in many applications some examples include the classical dose response experiment prospective and retrospective studies of disease incidence with and without matching and the analysis of ordinal data in most instances the model is fitted by the method of maximum likelihood which like least squares is sensitive to atypical observations an alternative to maximum likelihood is proposed and illustrated by examples