on degrees of freedom of projection estimators with applications to multivariate nonparametric regression abstract in this article we consider the nonparametric regression problem with multivariate predictors we provide a characterization of the degrees of freedom and divergence for estimators of the

derivations of the parabolic subalgebras of the general linear lie algebra over a commutative ring let r be an arbitrary commutative ring with identity gl n r the general linear lie algebra over r the aim of this paper is to give an explicit description of the derivation algebras of the parabolic subalgebras of gl n r

mappings that preserve pairs of operators with zero triple jordan product abstract let f be a field and n 3 suppose s 1 s 2 m n f contain all rank one idempotents the structure of surjections s 1 s 2 satisfying aba 0 a b a 0 is determined similar results are also obtained for a subsets of bounded operators acting on a complex or real banach space x b the space of hermitian matrices acting on n dimensional vectors over a skew field d c subsets of self adjoint bounded linear operators acting on an infinite dimensional complex hilbert space it is then illustrated that the results can be applied to characterize mappings on matrices or operators such that f aba f a b a for all a b for functions f such as the spectral norm schatten p norm numerical radius and numerical range etc

linear maps preserving idempotency of products or triple jordan products of operators abstract let b x be the algebra of all bounded linear operators on a complex banach space x we give the concrete forms of linear surjective maps on b x which preserve the nonzero idempotency of either products of two operators or triple jordan products of two operators

estimation of the mean of a multivariate normal distribution estimation of the means of independent normal random variables is considered using sum of squared errors as loss an unbiased estimate of risk is obtained for an arbitrary estimate and certain special classes of estimates are then discussed the results are applied to smoothing by use of moving averages and to trimmed analogs of the james stein estimate a suggestion is made for calculating approximate confidence sets for the mean vector centered at an arbitrary estimate

working correlation structure identification in generalized estimating equations selecting an appropriate working correlation structure is pertinent to clustered data analysis using generalized estimating equations gee because an inappropriate choice will lead to inefficient parameter estimation we investigate the well known criterion of qic for selecting a working correlation structure and have found that performance of the qic is deteriorated by a term that is theoretically independent of the correlation structures but has to be estimated with an error this leads us to propose a correlation information criterion cic that substantially improves the qic performance extensive simulation studies indicate that the cic has remarkable improvement in selecting the correct correlation structures we also illustrate our findings using a data set from the madras longitudinal schizophrenia study

model evaluation based on the sampling distribution of estimated absolute prediction error the construction of a reliable practically useful prediction rule for future responses is heavily dependent on the adequacy of the fitted regression model in this article we consider the absolute prediction error the expected value of the absolute difference between the future and predicted responses as the model evaluation criterion this prediction error is easier to interpret than the average squared error and is equivalent to the misclassification error for a binary outcome we show that the prediction error can be consistently estimated via the resubstitution and crossvalidation methods even when the fitted model is not correctly specified furthermore we show that the resulting estimators are asymptotically normal when the prediction rule is nonsmooth the variance of the above normal distribution can be estimated well with a perturbation resampling method with two real examples and an extensive simulation study we demonstrate that the interval estimates obtained from the above normal approximation for the prediction errors provide much more information about model adequacy than their point estimate counterparts

analysis of variance of cross validation estimators of the generalization error this paper brings together methods from two different disciplines statistics and machine learning we address the problem of estimating the variance of cross validation cv estimators of the generalization error in particular we approach the problem of variance estimation of the cv estimators of generalization error as a problem in approximating the moments of a statistic the approximation illustrates the role of training and test sets in the performance of the algorithm it provides a unifying approach to evaluation of various methods used in obtaining training and test sets and it takes into account the variability due to different training and test sets for the simple problem of predicting the sample mean and in the case of smooth loss functions we show that the variance of the cv estimator of the generalization error is a function of the moments of the random variables y card sj sj and y card sjc sj c where sj sj are two training sets and sjc sj c are the corresponding test sets we prove that the distribution of y and y is hypergeometric and we compare our estimator with the one proposed by nadeau and bengio 2003 we extend these results in the regression case and the case of absolute error loss and indicate how the methods can be extended to the classification case we illustrate the results through simulation

efficient computation and model selection for the support vector regression in this letter we derive an algorithm that computes the entire solution path of the support vector regression svr we also propose an unbiased estimate for the degrees of freedom of the svr model which allows convenient selection of the regularization parameter

choice of hierarchical priors admissibility in estimation of normal means in hierarchical bayesian modeling of normal means it is common to complete the prior specification by choosing a constant prior density for unmodeled hyperparameters e g variances and highest level means this common practice often results in an inadequate overall prior inadequate in the sense that estimators resulting from its use can be inadmissible under quadratic loss in this paper hierarchical priors for normal means are categorized in terms of admissibility and inadmissibility of resulting estimators for a quite general scenario the jeffreys prior for the hyper variance and a shrinkage prior for the hypermeans are recommended as admissible alternatives incidental to this analysis is presentation of the conditions under which the generally improper priors result in proper posteriors

a note on algebra automorphisms of strictly upper triangular matrices over commutative rings abstract in this note we explicitly describe the automorphism group of the r algebra of strictly upper triangular n n matrices over an arbitrary commutative ring

reduction techniques for strongly graded rings and finite representation type abstract we present reduction techniques for studying the category of lattices over strongly graded orders in particular we apply these techniques in order to reduce the problem of classifying those strongly graded orders with finite representation type to the case where the coefficient ring is a maximal order in a division ring

improving upon standard estimators in discrete exponential families with applications to poisson and negative binomial cases assume that xi xp are independent random observations having discrete exponential densities p1 0 ti x rj 1 i 1 p respectively a general technique of improving upon the uniform minimum variance unbiased estimator umvue of o1 op is developed under possibly weighted squared error loss functions it is shown that improved estimators can be constructed by solving a difference inequality typical difference inequalities of a fairly general type are presented and solved when specialized to poisson and negative binomial cases broad classes of estimators are given that dominate the umvue these results unify many known results in this rapidly diverging field and some of them are new especially those related to negative binomial distributions improved estimators are also obtained for the problems in which some of the observations are from poisson families and some from negative binomial families for sum of squared errors loss estimators which dominate the umvue in the discrete exponential families are also given explicitly

automorphisms and derivations of upper triangular matrix rings abstract kezlan proved that for a commutative ring c every c automorphism of the ring of upper triangular matrices over c is inner we generalize this result to rings in which all idempotents are central moreover we show that for a semiprime ring a and central subring c every c automorphism of the ring of upper triangular matrices over c is the composite of an inner automorphism and an automorphism induced from a c automorphism of a by the method of proof we re prove results of s p coelho and c p milies and of mathis stating that a derivation of a ring of upper triangular matrices of a c algebra n n matrices over a is a sum of an inner derivation and a derivation induced from a c derivation of a by an example we show that an extra assumption is needed for proving the above result of automorphisms of upper triangular matrices finally we consider automorphisms of subrings of n n matrices over a commutative ring c where entries over the diagonal are from c and below the diagonal are taken from a nil ideal we prove that all such automorphisms are inner

sure estimates for a heteroscedastic hierarchical model hierarchical models are extensively studied and widely used in statistics and many other scientific areas they provide an effective tool for combining information from similar resources and achieving partial pooling of inference since the seminal work by james and stein 1961 and stein 1962 shrinkage estimation has become one major focus for hierarchical models for the homoscedastic normal model it is well known that shrinkage estimators especially the james stein estimator have good risk properties the heteroscedastic model though more appropriate for practical applications is less well studied and it is unclear what types of shrinkage estimators are superior in terms of the risk we propose in this article a class of shrinkage estimators based on stein s unbiased estimate of risk sure we study asymptotic properties of various common estimators as the number of means to be estimated grows p we establish the asymptotic optimality property for the sure estimators we then extend our

the automorphism group of a structural matrix algebra abstract we characterize the automorphism group of certain subalgebras of matrix algebras with entries from a field k known as structural matrix algebras these include the algebras of upper triangular matrices we also give necessary and sufficient conditions for every k automorphism of such a subalgebra to be inner

multiplicative semigroup automorphisms of upper triangular matrices over rings abstract suppose r is a ring with 1 and c a central subring of r let tn r be the c algebra of upper triangular n n matrices over r recently several authors have shown that if r is sufficiently well behaved then every c automorphism of tn r is the composites of an inner automorphism and an automorphism induced from a c automorphism of r see 1 5 to generalize these results in this paper we prove that if n 2 and r is a semiprime ring or a ring in which all idempotents are central then f tn r tn r tn r is only regarded as a multiplicative semigroup is a multiplicative semigroup automorphism if and only if there exist a nonsingular matrix p in tn r and a ring automorphism of r such that f a p 1 a p a a ij n n n r where a aij n n

when does more regularization imply fewer degrees of freedom sufficient conditions and counterexamples regularization aims to improve prediction performance by trading an increase in training error for better agreement between training and prediction errors which is often captured through decreased degrees of freedom in this paper we give examples which show that regularization can increase the degrees of freedom in common models including the lasso and ridge regression in such situations both training error and degrees of freedom increase making the regularization inherently without merit two important scenarios are described where the expected reduction in degrees of freedom is guaranteed all symmetric linear smoothers and convex constrained linear regression models like ridge regression and the lasso when compared to unconstrained linear regression

automorphisms of the standard borel subgroup of orthogonal group o 2m r over commutative rings abstract all automorphisms of the standard borel subgroup of orthogonal group o 2 m r are determined provided that r is a commutative ring with identity 2 3 are invertible in r and m 5

the estimation of prediction error having constructed a data based estimation rule perhaps a logistic regression or a classification tree the statistician would like to know its performance as a predictor of future cases there are two main theories concerning prediction error 1 penalty methods such as cp akaike s information criterion and stein s unbiased risk estimate that depend on the covariance between data points and their corresponding predictions and 2 cross validation and related nonparametric bootstrap techniques this article concerns the connection between the two theories a rao blackwell type of relation is derived in which nonparametric methods such as cross validation are seen to be randomized versions of their covariance penalty counterparts the model based penalty methods offer substantially better accuracy assuming that the model is believable

the little bootstrap and other methods for dimensionality selection in regression x fixed prediction error abstract when a regression problem contains many predictor variables it is rarely wise to try to fit the data by means of a least squares regression on all of the predictor variables usually a regression equation based on a few variables will be more accurate and certainly simpler there are various methods for picking good subsets of variables and programs that do such procedures are part of every widely used statistical package the most common methods are based on stepwise addition or deletion of variables and on best subsets the latter refers to a search method that given the number of variables to be in the equation say five locates that regression equation based on five variables that has the lowest residual sum of squares among all five variable equations all of these procedures generate a sequence of regression equations the first based on one variable the next based on two variables and so on each member of this sequence is called a submodel and the number of variables in the e

improvements on cross validation the 632 bootstrap method abstract a training set of data has been used to construct a rule for predicting future responses what is the error rate of this rule this is an important question both for comparing models and for assessing a final selected model the traditional answer to this question is given by cross validation the cross validation estimate of prediction error is nearly unbiased but can be highly variable here we discuss bootstrap estimates of prediction error which can be thought of as smoothed versions of cross validation we show that a particular bootstrap method the 632 rule substantially outperforms cross validation in a catalog of 24 simulation experiments besides providing point estimates we also consider estimating the variability of an error rate estimate all of the results here are nonparametric and apply to any possible prediction rule however we study only classification problems with 0 1 loss in detail our simulations include smooth prediction rules like fisher s linear discriminant fun

spatial shrinkage estimation of diffusion tensors on diffusion weighted imaging data diffusion tensor imaging dti based on the diffusion weighted imaging dwi data acquired from magnetic resonance experiments has been widely used to analyze the physical structure of white matter fibers in the human brain in vivo the raw dwi data however carry noise this contaminates the diffusion tensor dt estimates and introduces systematic bias into the induced eigenvalues these bias components affect the effectiveness of fiber tracking algorithms in this article we propose a two stage spatial shrinkage estimation spske procedure to accommodate the spatial information carried in dwi data in dt estimation and to reduce the bias components in the corresponding derived eigenvalues to this end in the framework of the heteroscedastic linear model spske incorporates l 1 type penalization and the locally weighted least square function the theoretical properties of spske are explored the effectiveness of spske is further illustrated by simulation and real data examples supplementary materi

discussion of from fixed x to random x regression bias variance decompositions covariance penalties and prediction error estimation rosset and tibshirani are to be congratulated for addressing an important yet under studied topic of modern statistics so called random x prediction in regression analysis in statistics random x

quantile regression in reproducing kernel hilbert spaces in this article we consider quantile regression in reproducing kernel hilbert spaces which we call kernel quantile regression kqr we make three contributions 1 we propose an efficient algorithm that computes the entire solution path of the kqr with essentially the same computational cost as fitting one kqr model 2 we derive a simple formula for the effective dimension of the kqr model which allows convenient selection of the regularization parameter and 3 we develop an asymptotic theory for the kqr model

linear maps preserving products of positive or hermitian matrices abstract in this note we consider linear maps on the complex n n matrix algebra m n preserving products of either two positive matrices a positive matrix and a hermitian matrix or two hermitian matrices we give characterizations of all those injections

optimal shrinkage estimation of mean parameters in family of distributions with quadratic variance this paper discusses the simultaneous inference of mean parameters in a family of distributions with quadratic variance function we first introduce a class of semi parametric parametric shrinkage estimators and establish their asymptotic optimality properties two specific cases the location scale family and the natural exponential family with quadratic variance function are then studied in detail we conduct a comprehensive simulation study to compare the performance of the proposed methods with existing shrinkage estimators we also apply the method to real data and obtain encouraging results

automorphisms of the lie algebra of strictly upper triangular matrices over a commutative ring let r be an arbitrary commutative ring with identity denote by n r the lie algebra over r consisting of all strictly upper triangular n 1 n 1 matrices over r with n 3 in addition for n 3 assume that the annihilator of 2 in r is zero the aim of this paper is to describe the automorphism group of n r we show that any automorphism of n r can be expressed as where and are graph extremal central and inner automorphisms respectively of n r

maps preserving the local spectrum of skew product of operators abstract let h and k be infinite dimensional complex hilbert spaces and let b h be the algebra of all bounded linear operators on h let t h denote the local spectrum of an operator t b h at any vector h h and fix two nonzero vectors h 0 h and k 0 k we show that if a map b h b k has a range containing all operators of rank at most two and satisfies t s h 0 t s k 0 for all t s b h then there exist two unitary operators u and v in b h k such that u h 0 k 0 for some nonzero c and t u t v for all t b h we also described maps b h b k satisfying t s t h 0 t s t k 0 for all t s b h and with the range containing all operators of rank at most four

parametric empirical bayes inference theory and applications abstract this article reviews the state of multiparameter shrinkage estimators with emphasis on the empirical bayes viewpoint particularly in the case of parametric prior distributions some successful applications of major importance are considered recent results concerning estimates of error and confidence intervals are described and illustrated with data

jump information criterion for statistical inference in estimating discontinuous curves nonparametric regression analysis when the regression function is discontinuous has many applications existing methods for estimating a discontinuous regression curve usually assume that the number of jumps in the regression curve is known beforehand which is unrealistic in some situations although there has been research on estimation of a discontinuous regression curve when the number of jumps is unknown the problem remains mostly open because such research often requires assumptions on other related quantities such as a known minimum jump size in this paper we propose a jump information criterion which consists of a term measuring the fidelity of the estimated regression curve to the observed data and a penalty related to the number of jumps and the jump sizes the number of jumps can then be determined by minimizing our criterion theoretical and numerical studies show that our method works well

evaluating prediction rules for t year survivors with censored regression models suppose that we are interested in establishing simple but reliable rules for predicting future t year survivors through censored regression models in this article we present inference procedures for evaluating such binary classification rules based on various prediction precision measures quantified by the overall misclassification rate sensitivity and specificity and positive and negative predictive values specifically under various working models we derive consistent estimators for the above measures through substitution and cross validation estimation procedures furthermore we provide large sample approximations to the distributions of these nonsmooth estimators without assuming that the working model is correctly specified confidence intervals for example for the difference of the precision measures between two competing rules can then be constructed all of the proposals are illustrated with real examples and their finite sample properties are evaluated through a simulation study

convex optimization shape constraints compound decisions and empirical bayes rules estimation of mixture densities for the classical gaussian compound decision problem and their associated empirical bayes rules is considered from two new perspectives the first motivated by brown and greenshtein introduces a nonparametric maximum likelihood estimator of the mixture density subject to a monotonicity constraint on the resulting bayes rule the second motivated by jiang and zhang proposes a new approach to computing the kiefer wolfowitz nonparametric maximum likelihood estimator for mixtures in contrast to prior methods for these problems our new approaches are cast as convex optimization problems that can be efficiently solved by modern interior point methods in particular we show that the reformulation of the kiefer wolfowitz estimator as a convex optimization problem reduces the computational effort by several orders of magnitude for typical problems by comparison to prior em algorithm based methods and thus greatly expands the practical applicability of the resulting method

the covariance inflation criterion for adaptive model selection we propose a new criterion for model selection in prediction problems the covariance inflation criterion adjusts the training error by the average covariance of the predictions and responses when the prediction rule is applied to permuted versions of the data set this criterion can be applied to general prediction problems e g regression or classification and to general prediction rules e g stepwise regression tree based models and neural nets as a by product we obtain a measure of the effective number of parameters used by an adaptive procedure we relate the covariance inflation criterion to other model selection procedures and illustrate its use in some regression and classification problems we also revisit the conditional bootstrap approach to model selection

multiplicative semigroup automorphisms of the strictly upper triangular matrices over a commutative ring let r be an arbitrary commutative ring with identity and nn r b e the multiplicative semigroup consisting of all n n strictly upper triangular matrices over r t he aim of this paper is to give an explicit description of any automorphism of the semigroup nn r

penalized model based clustering with application to variable selection variable selection in clustering analysis is both challenging and important in the context of model based clustering analysis with a common diagonal covariance matrix which is especially suitable for high dimension low sample size settings we propose a penalized likelihood approach with an l1 penalty function automatically realizing variable selection via thresholding and delivering a sparse solution we derive an em algorithm to fit our proposed model and propose a modified bic as a model selection criterion to choose the number of components and the penalization parameter a simulation study and an application to gene function prediction with gene expression profiles demonstrate the utility of our method

on the degrees of freedom of the lasso we study the effective degrees of freedom of the lasso in the framework of stein s unbiased risk estimation sure we show that the number of nonzero coefficients is an unbiased estimate forthe degrees of freedom of the lasso a conclusion that requires no special assumption on the predictors in addition the unbiased estimator is shown to be asymptotically consistent with these results on hand various model selection criteria cp aic and bic are available which along with the lars algorithm provide a principled and efficient approach to obtaining the optimal lasso fit with the computational effort of a single ordinary least squares fit

from stein s unbiased risk estimates to the method of generalized cross validation this paper concerns the method of generalized cross validation gcv a promising way of choosing between linear estimates based on stein estimates and the associated unbiased risk estimates stein 1981 a new approach to gcv is developed many consistency results are obtained for the cross validated steinized estimates in the contexts of nearest neighbor nonparametric regression model selection ridge regression and smoothing splines moreover the associated stein s unbiased risk estimate is shown to be uniformly consistent in assessing the true loss not the risk consistency properties are examined as well when the sampling error is unknown finally we propose a variant of gcv to handle the case that the dimension of the raw data is known to be greater than that of their expected values

empirical bayes prediction for the multivariate newsvendor loss function motivated by an application in inventory management we consider the multi product newsvendor problem of nding the optimal stocking levels that minimize the total backorder and lost sales costs we focus on a setting where we have a large number of products and observe only noisy estimates of the underlying demand we develop an empirical bayes methodology for predicting stocking levels using data adaptive linear shrinkage strategies which are constructed by minimizing uniformly ecient asymptotic risk estimates in calculating the magnitude and direction of shrinkage our proposed predictive rules incorporate the asymmetric nature of the piecewise linear newsvendor loss function and are shown to be asymptotically optimal using simulated data we study the non asymptotic performance of our method and obtain encouraging results the asymptotic risk estimation method and the proof techniques developed here can also be used to formulate optimal empirical bayes predictive strategies for general piecewise linear and related asymmetric loss functions 1 introduction the newsvendor problem of determining optimal stocking levels is one of the classical problems in the literature on inventory management arrow harris and marschak 1951 choi 2012 we study the traditional newsvendor problem scarf 1959 in the modern big data regime and consider the inventory management problem of a vendor who sells a large number of products the demand distribution of each product is unknown and must be estimated from data our problem is motivated by modern day online retailers who carry hundreds to millions of products the historical sales and projected demand varies greatly across dierent products and the company must decide the stocking quantity of each product we consider a one period setting where based on the observed demand in the previous period the rm must determine the stocking quantity of each product in the next period the observed demand in the previous period provides a basis for estimating the demand in the next period the rm has to balance the tradeos between stocking too much and incurring high inventory cost versus stocking too little and suering lost sales our objective is to develop a data driven policy that minimizes the total inventory and lost sales costs across all products

the group of automorphisms of certain subalgebras of matrix algebras the purpose of this note is to find and characterize the full group of automorphisms of certain prime algebras which are subalgebras of matrix algebras over commutative rings here we extend the methods which we used in 6 to characterize automorphisms of certain upper triangular matrix rings for a central simple k algebra a the group of k automorphisms of a is extensively treated in the literature 5 theorem 6 21 other cases where the automorphism group of a ring is known are the automorphism group of a polynomial ring in two indeterminates over a field and the free algebra in two variables over a field 4 chap 61 finally if a is a prime p z algebra with center c then every c automorphism of a is x inner 7 example 3 91

how biased is the apparent error rate of a prediction rule abstract a regression model is fitted to an observed set of data how accurate is the model for predicting future observations the apparent error rate tends to underestimate the true error rate because the data have been used twice both to fit the model and to check its accuracy we provide simple estimates for the downward bias of the apparent error rate the theory applies to general exponential family linear models and general measures of prediction error special attention is given to the case of logistic regression on binary data with error rates measured by the proportion of misclassified cases several connected ideas are compared mallows s cp cross validation generalized cross validation the bootstrap and akaike s information criterion

discussion local rademacher complexities and oracle inequalities in risk minimization in this magnificent paper professor koltchinskii offers general and powerful performance bounds for empirical risk minimization a fundamental principle of statistical learning theory since the elegant pioneering work of vapnik and chervonenkis in the early 1970s various such bounds have been known that relate the performance of empirical risk minimizers to combinatorial and geometrical features of the class over which the minimization is performed this area of research has been a rich source of motivation and a major field of applications of empirical process theory the appearance of advanced concentration inequalities in the 1990s primarily thanks to talagrand s influential work provoked major advances in both empirical process theory and statistical learning theory and led to a much deeper understanding of some of the basic phenomena in the discussed paper professor koltchinskii develops a powerful new methodology iterative localization which with the help of concentration inequalities is able to explain most of the recent results and go significantly beyond them in many cases the main motivation behind professor koltchinskii s paper is based on classical problems of statistical learning theory such as binary classification and regression in which given a sample xi y i i 1 n of independent and identically distributed pairs of random variables where the xi take their values in some feature space x and the yi are say real valued the goal is to find a function f x r whose risk defined in terms of the expected value of an appropriately chosen loss function is as small as possible in the remaining part of this discussion we point out how the performance bounds of professor koltchinskii s paper can be used to study a seemingly different model motivated by nonparametric ranking problems which has received increasing attention both in the statistical and machine learning literature indeed in several applications such as the search engine problem or credit risk screening the goal is to learn how to rank or to score observations rather than just classify them in this case performance measures involve pairs of observations as can be seen for instance with the auc area under an roc curve criterion in this

three automorphism theorems for triangular matrix algebras abstract let t n a be the algebra of upper triangular n n matrices with entries from an associative k algebra a where k is a commutative ring recently several authors barker coelho jondrup kezlan have shown that if a is sufficiently well behaved then every k algebra automorphism of t n a decomposes into a product of an inner automorphism and an automorphism defined by an automorphism of a in this paper we find new sufficient conditions for a that guarantee such decompositions and we give comparisons with previous results as an intermediate step we show that a single automorphism admits such a decomposition if and only if it respects the subspace of strict upper triangular matrices we also consider the case of infinite matrices

on measuring and correcting the effects of data mining and model selection abstract in the theory of linear models the concept of degrees of freedom plays an important role this concept is often used for measurement of model complexity for obtaining an unbiased estimate of the error variance and for comparison of different models i have developed a concept of generalized degrees of freedom gdf that is applicable to complex modeling procedures the definition is based on the sum of the sensitivity of each fitted value to perturbation in the corresponding observed value the concept is nonasymptotic in nature and does not require analytic knowledge of the modeling procedures the concept of gdf offers a unified framework under which complex and highly irregular modeling procedures can be analyzed in the same way as classical linear models by using this framework many difficult problems can be solved easily for example one can now measure the number of observations used in a variable selection process different modeling procedures such as a tree based regression and a

estimating the error rate of a prediction rule improvement on cross validation abstract we construct a prediction rule on the basis of some data and then wish to estimate the error rate of this rule in classifying future observations cross validation provides a nearly unbiased estimate using only the original data cross validation turns out to be related closely to the bootstrap estimate of the error rate this article has two purposes to understand better the theoretical basis of the prediction problem and to investigate some related estimators which seem to offer considerably improved estimation in small samples

epimorphisms of infinite triangular and unitriangular matrices we discuss the form of all epimorphisms of t f the group of all infinite upper triangular matrices over a field into itself we show that every such epimorphism is a composition of some standard maps moreover we present an analogous result for ut f we use the obtained results to describe the groups of automorphisms of these groups

pointwise influence matrices for functional response regression we extend the notion of an influence or hat matrix to regression with functional responses and scalar predictors for responses depending linearly on a set of predictors our definition is shown to reduce to the conventional influence matrix for linear models the pointwise degrees of freedom the trace of the pointwise influence matrix are shown to have an adaptivity property that motivates a two step bivariate smoother for modeling nonlinear dependence on a single predictor this procedure adapts to varying complexity of the nonlinear model at different locations along the function and thereby achieves better performance than competing tensor product smoothers in an analysis of the development of white matter microstructure in the brain

excess optimism how biased is the apparent error of an estimator tuned by sure abstractnearly all estimators in statistical prediction come with an associated tuning parameter in one way or another common practice given data is to choose the tuning parameter value that minimizes a constructed estimate of the prediction error of the estimator we focus on stein s unbiased risk estimator or sure which forms an unbiased estimate of the prediction error by augmenting the observed training error with an estimate of the degrees of freedom of the estimator parameter tuning via sure minimization has been advocated by many authors in a wide variety of problem settings and in general it is natural to ask what is the prediction error of the sure tuned estimator an obvious strategy would be simply use the apparent error estimate as reported by sure that is the value of the sure criterion at its minimum to estimate the prediction error of the sure tuned estimator but this is no longer unbiased in fact we would expect that the minimum of the sure criterion is systematically bias

general maximum likelihood empirical bayes estimation of normal means we propose a general maximum likelihood empirical bayes gmleb method for the estimation of a mean vector based on observations with i i d normal errors we prove that under mild moment conditions on the unknown means the average mean squared error mse of the gmleb is within an infinitesimal fraction of the minimum average mse among all separable estimators which use a single deterministic estimating function on individual observations provided that the risk is of greater order than log n 5 n we also prove that the gmleb is uniformly approximately minimax in regular and weak l p balls when the order of the length normalized norm of the unknown means is between log n k1 n 1 p 2 and n log n k2 simulation experiments demonstrate that the gmleb outperforms the james stein and several state of the art threshold estimators in a wide range of settings without much down side

large margin semi supervised learning in classification semi supervised learning occurs when a large amount of unlabeled data is available with only a small number of labeled data in such a situation how to enhance predictability of classification through unlabeled data is the focus in this article we introduce a novel large margin semi supervised learning methodology using grouping information from unlabeled data together with the concept of margins in a form of regularization controlling the interplay between labeled and unlabeled data based on this methodology we develop two specific machines involving support vector machines and learning denoted as ssvm and spsi through difference convex programming in addition we estimate the generalization error using both labeled and unlabeled data for tuning regularizers finally our theoretical and numerical analyses indicate that the proposed methodology achieves the desired objective of delivering high performance in generalization particularly against some strong performers

differential geometric least angle regression a differential geometric approach to sparse generalized linear models sparsity is an essential feature of many contemporary data problems remote sensing various forms of automated screening and other high throughput measurement devices collect a large amount of information typically about few independent statistical subjects or units in certain cases it is reasonable to assume that the underlying process generating the data is itself sparse in the sense that only a few of the measured variables are involved in the process we propose an explicit method of monotonically decreasing sparsity for outcomes that can be modelled by an exponential family in our approach we generalize the equiangular condition in a generalized linear model although the geometry involves the fisher information in a way that is not obvious in the simple regression setting the equiangular condition turns out to be equivalent with an intuitive condition imposed on the rao score test statistics in certain special cases the method can be tweaked to obtain l1 penalized generalized linear model solution paths but the method itself defines sparsity more directly although the computation of the solution paths is not trivial the method compares favourably with other path following algorithms

grouping pursuit through a regularization solution surface extracting grouping structure or identifying homogenous subgroups of predictors in regression is crucial for high dimensional data analysis a low dimensional structure in particular grouping when captured in a regression model enables to enhance predictive performance and to facilitate a model s interpretability grouping pursuit extracts homogenous subgroups of predictors most responsible for outcomes of a response this is the case in gene network analysis where grouping reveals gene functionalities with regard to progression of a disease to address challenges in grouping pursuit we introduce a novel homotopy method for computing an entire solution surface through regularization involving a piecewise linear penalty this nonconvex and overcomplete penalty permits adaptive grouping and nearly unbiased estimation which is treated with a novel concept of grouped subdifferentials and difference convex programming for efficient computation finally the proposed method not only achieves high performanc

simultaneous pursuit of sparseness and rank structures for matrix decomposition in multi response regression pursuit of two different types of structures is essential to battle the curse of dimensionality in this paper we seek a sparsest decomposition representation of a parameter matrix in terms of a sum of sparse and low rank matrices among many overcomplete decompositions on this basis we propose a constrained method subject to two nonconvex constraints respectively for sparseness and low rank properties computationally obtaining an exact global optimizer is rather challenging to overcome the difficulty we use an alternating directions method solving a low rank subproblem and a sparseness subproblem alternatively where we derive an exact solution to the low rank subproblem as well as an exact solution in a special case and an approximated solution generally through a surrogate of the l0 constraint and difference convex programming for the sparse subproblem theoretically we establish convergence rates of a global minimizer in the hellinger distance providing an insight into why pursuit of two different types of decomposed structures is expected to deliver higher estimation accuracy than its counterparts based on either sparseness alone or low rank approximation alone numerical examples are given to illustrate these aspects in addition to an application to facial imagine recognition and multiple time series analysis

optimal model assessment selection and combination central to statistical theory and application is statistical modeling which typically involves choosing a single model or combining a number of models of different sizes and from different sources whereas model selection seeks a single best modeling procedure model combination combines the strength of different modeling procedures in this article we look at several key issues and argue that model assessment is the key to model selection and combination most important we introduce a general technique of optimal model assessment based on data perturbation thus yielding optimal selection in particular model selection and combination from a frequentist perspective we advocate model combination over a selected subset of modeling procedures because it controls bias while reducing variability hence yielding better performance in terms of the accuracy of estimation and prediction to realize the potential of model combination we develop methodologies for determining the optimal tuning parameter such

on the automorphisms of incidence algebras abstract given a locally finite partially ordered set x a ring with identity r and an automorphism of the incidence algebra of x over r it is determined when is the composite of an inner automorphism an automorphism of x and an induced automorphism of r

linear model selection when covariates contain errors prediction precision is arguably the most relevant criterion of a model in practice and is often a sought after property a common difficulty with covariates measured with errors is the impossibility of performing prediction evaluation on the data even if a model is completely given without any unknown parameters we bypass this inherent difficulty by using special properties on moment relations in linear regression models with measurement errors the end product is a model selection procedure that achieves the same optimality properties that are achieved in classical linear regression models without covariate measurement error asymptotically the procedure selects the model with the minimum prediction error in general and selects the smallest correct model if the regression relation is indeed linear our model selection procedure is useful in prediction when future covariates without measurement error become available e g due to improved technology or better management and design of data collection procedures

maps preserving the idempotency of products of operators abstract let b x be the algebra of all bounded linear operators on the banach space x and let n x be the set of nilpotent operators in b x suppose b x b x is a surjective map such that a b b x satisfy ab n x if and only if a b n x if x is infinite dimensional then there exists a map f b x c 0 such that one of the following holds a there is a bijective bounded linear or conjugate linear operator s x x such that has the form a s f a a s 1 b the space x is reflexive and there exists a bijective bounded linear or conjugate linear operator s x x such that has the form a s f a a s 1 if x has dimension n with 3 n b x is identified with the algebra m n of n n complex matrices then there exist a map f m n c 0 a field automorphism c c and an invertible s m n such that has one of the following forms a a ij f a s a ij s 1 or a a ij f a s a ij t s 1 where a t denotes the transpose of a the results are extended to the product of more than two operators and to other types of products on b x including the jordan triple product a b aba furthermore the results in the finite dimensional case are used to characterize surjective maps on matrices preserving the spectral radius of products of matrices

a proximal iteration for deconvolving poisson noisy images using sparse representations we propose an image deconvolution algorithm when the data is contaminated by poisson noise the image to restore is assumed to be sparsely represented in a dictionary of waveforms such as the wavelet or curvelet transforms our key contributions are as follows first we handle the poisson noise properly by using the anscombe variance stabilizing transform leading to a nonlinear degradation equation with additive gaussian noise second the deconvolution problem is formulated as the minimization of a convex functional with a data fidelity term reflecting the noise properties and a nonsmooth sparsity promoting penalty over the image representation coefficients e g lscr1 norm an additional term is also included in the functional to ensure positivity of the restored image third a fast iterative forward backward splitting algorithm is proposed to solve the minimization problem we derive existence and uniqueness conditions of the solution and establish convergence of the iterative algorithm finally a gcv based model selection procedure is proposed to objectively select the regularization parameter experimental results are carried out to show the striking benefits gained from taking into account the poisson statistics of the noise these results also suggest that using sparse domain regularization may be tractable in many deconvolution applications with poisson noise such as astronomy and microscopy

model selection for generalized estimating equations accommodating dropout missingness the generalized estimating equation gee has been a popular tool for marginal regression analysis with longitudinal data and its extension the weighted gee approach can further accommodate data that are missing at random mar model selection methodologies for gee however have not been systematically developed to allow for missing data we propose the missing longitudinal information criterion mlic for selection of the mean model and the mlic for correlation mlicc for selection of the correlation structure in gee when the outcome data are subject to dropout monotone missingness and are mar our simulation results reveal that the mlic and mlicc are effective for variable selection in the mean model and selecting the correlation structure respectively we also demonstrate the remarkable drawbacks of naively treating incomplete data as if they were complete and applying the existing gee model selection method the utility of proposed method is further illustrated by two real applications involving missing longitudinal outcome data

spatial weights matrix selection and model averaging for spatial autoregressive models abstract spatial econometrics relies on the spatial weights matrix to specify the cross sectional dependence however the candidate spatial weights matrices might not be unique this paper proposes a model selection procedure to choose a weights matrix from several candidates by using a mallows type criterion we prove that when the true weights matrix is not in the candidates the procedure is asymptotically optimal in the sense of minimizing the squared loss otherwise the procedure can select the true weights matrix consistently we then propose a model averaging procedure to reduce the squared loss we also provide procedures for the spatial model with heteroscedasticity and endogenous regressors and the model with both spatial lag and spatial error monte carlo experiments show that proposed procedures have satisfactory finite sample performances we apply the model selection and model averaging procedures to study the market integration in china using historical rice prices

a new solution path algorithm in support vector regression in this paper regularization path algorithms were proposed as a novel approach to the model selection problem by exploring the path of possibly all solutions with respect to some regularization hyperparameter in an efficient way this approach was later extended to a support vector regression svr model called epsiv svr however the method requires that the error parameter epsiv be set a priori this is only possible if the desired accuracy of the approximation can be specified in advance in this paper we analyze the solution space for epsiv svr and propose a new solution path algorithm called epsiv path algorithm which traces the solution path with respect to the hyperparameter epsiv rather than lambda although both two solution path algorithms possess the desirable piecewise linearity property our epsiv path algorithm overcomes some limitations of the original lambda path algorithm and has more advantages it is thus more appealing for practical use

adaptive model selection most model selection procedures use a fixed penalty penalizing an increase in the size of a model these nonadaptive selection procedures perform well only in one type of situation for instance bayesian information criterion bic with a large penalty performs well for small models and poorly for large models and akaike s information criterion aic does just the opposite this article proposes an adaptive model selection procedure that uses a data adaptive complexity penalty based on a concept of generalized degrees of freedom the proposed procedure combining the benefit of a class of nonadaptive procedures approximates the best performance of this class of procedures across a variety of different situations this class includes many well known procedures such as aic bic mallows s cp and risk inflation criterion ric the proposed procedure is applied to wavelet thresholding in nonparametric regression and variable selection in least squares regression simulation results and an asymptotic

on the degrees of freedom of reduced rank estimators in multivariate regression we study the effective degrees of freedom of a general class of reduced rank estimators for multivariate regression in the framework of stein s unbiased risk estimation a finite sample exact unbiased estimator is derived that admits a closed form expression in terms of the thresholded singular values of the least squares solution and hence is readily computable the results continue to hold in the high dimensional setting where both the predictor and the response dimensions may be larger than the sample size the derived analytical form facilitates the investigation of theoretical properties and provides new insights into the empirical behaviour of the degrees of freedom in particular we examine the differences and connections between the proposed estimator and a commonly used naive estimator the use of the proposed estimator leads to efficient and accurate prediction risk estimation and model selection as demonstrated by simulation studies and a data example

nonparametric empirical bayes and compound decision approaches to estimation of a high dimensional vector of normal means we consider the classical problem of estimating a vector 1 n based on independent observations y i n i 1 i 1 n suppose i i 1 n are independent realizations from a completely unknown g we suggest an easily computed estimator such that the ratio of its risk e 2 with that of the bayes procedure approaches 1 a related compound decision result is also obtained our asymptotics is of a triangular array that is we allow the distribution g to depend on n thus our theoretical asymptotic results are also meaningful in situations where the vector is sparse and the proportion of zero coordinates approaches 1 we demonstrate the performance of our estimator in simulations emphasizing sparse setups in moderately sparse situations our procedure performs very well compared to known procedures tailored for sparse setups it also adapts well to nonsparse situations

asymptotic optimality for c_p c_l cross validation and generalized cross validation discrete index set c p c l cross validation and generalized cross validation are useful data driven techniques for selecting a good estimate from a proposed class of linear estimates the asymptotic behaviors of these procedures are studied some easily interpretable conditions are derived to demonstrate the asymptotic optimality it is argued that cross validation and generalized cross validation can be viewed as some special ways of applying c l applications in nearest neighbor nonparametric regression and in model selection are discussed in detail

more comments on c p we discuss the interpretation of c p plots and show how they can be calibrated in several ways we comment on the practice of using the display as a basis for formal selection of a subset regression model and extend the range of application of the device to encompass arbitrary linear estimates of the regression coefficients for example ridge estimates

on the predictive risk in misspecified quantile regression abstract in the present paper we investigate the predictive risk of possibly misspecified quantile regression functions the in sample risk is well known to be an overly optimistic estimate of the predictive risk and we provide two relatively simple asymptotic characterizations of the associated bias also called expected optimism we propose estimates for the expected optimism and the predictive risk and establish their uniform consistency under mild conditions our results hold for models of moderately growing size and allow the quantile function to be incorrectly specified empirical evidence from our estimates is encouraging as it compares favorably with cross validation