nonparametric estimates of regression quantiles and their local bahadur representation let x y be a random vector such that x is d dimensional y is real valued and y theta x varepsilon where x and varepsilon are independent and the alpha th quantile of varepsilon is 0 alpha is fixed such that 0 0 and set r p m 2p d where m is a nonnegative integer smaller than p let t theta denote a derivative of theta of order m it is proved that there exists a pointwise estimate hat t n of t theta based on a set of i i d observations x 1 y 1 cdots s n y n that achieves the optimal nonparametric rate of convergence n r under appropriate regularity conditions further a local bahadur type representation is shown to hold for the estimate hat t n and this is used to obtain some useful asymptotic results

estimation of the mean of a multivariate normal distribution estimation of the means of independent normal random variables is considered using sum of squared errors as loss an unbiased estimate of risk is obtained for an arbitrary estimate and certain special classes of estimates are then discussed the results are applied to smoothing by use of moving averages and to trimmed analogs of the james stein estimate a suggestion is made for calculating approximate confidence sets for the mean vector centered at an arbitrary estimate

identifying risk factors for severe childhood malnutrition by boosting additive quantile regression we investigated the risk factors for childhood malnutrition in india based on the 2005 2006 demographic and health survey by applying a novel estimation technique for additive quantile regression ordinary linear and generalized linear regression models relate the mean of a response variable to a linear combination of covariate effects and as a consequence focus on average properties of the response the use of such a regression model for analyzing childhood malnutrition in developing or transition countries implies that the estimated effects describe the average nutritional status however it is of even greater interest to analyze quantiles of the response distribution such as the 5 or 10 quantile which relate to the risk of extreme malnutrition our investigation is based on a semiparametric extension of quantile regression models where different types of nonlinear effects are included in the model equation leading to additive quantile regression we addressed the variable selection and model c

bi level path following for cross validated solution of kernel quantile regression we show how to follow the path of cross validated solutions to families of regularized optimization problems defined by a combination of a parameterized loss function and a regularization term a primary example is kernel quantile regression where the parameter of the loss function is the quantile being estimated even though the bi level optimization problem we encounter for every quantile is non convex the manner in which the optimal cross validated solution evolves with the parameter of the loss function allows tracking of this solution we prove this property construct the resulting algorithm and demonstrate it on real and artificial data this algorithm allows us to efficiently solve the whole family of bi level problems we show how it can be extended to cover other modeling problems like support vector regression and alternative in sample model selection approaches

the entire regularization path for the support vector machine the support vector machine svm is a widely used tool for classification many efficient implementations exist for fitting a two class svm model the user has to supply values for the tuning parameters the regularization cost parameter and the kernel parameters it seems a common practice is to use a default value for the cost parameter often leading to the least restrictive model in this paper we argue that the choice of the cost parameter can be critical we then derive an algorithm that can fit the entire path of svm solutions for every value of the cost parameter with essentially the same computational cost as fitting one svm model we illustrate our algorithm on some examples and use our representation to give further insight into the range of svm solutions

quantile smoothing splines summary although nonparametric regression has traditionally focused on the estimation of conditional mean functions nonparametric estimation of conditional quantile functions is often of substantial practical interest we explore a class of quantile smoothing splines defined as solutions to

the estimation of prediction error having constructed a data based estimation rule perhaps a logistic regression or a classification tree the statistician would like to know its performance as a predictor of future cases there are two main theories concerning prediction error 1 penalty methods such as cp akaike s information criterion and stein s unbiased risk estimate that depend on the covariance between data points and their corresponding predictions and 2 cross validation and related nonparametric bootstrap techniques this article concerns the connection between the two theories a rao blackwell type of relation is derived in which nonparametric methods such as cross validation are seen to be randomized versions of their covariance penalty counterparts the model based penalty methods offer substantially better accuracy assuming that the model is believable

convergence rate of sieve estimates in this paper we develop a general theory for the convergence rate of sieve estimates maximum likelihood estimates mle s and related estimates obtained by optimizing certain empirical criteria in general parameter spaces in many cases especially when the parameter space is infinite dimensional maximization over the whole parameter space is undesirable in such cases one has to perform maximization over an approximating space sieve of the original parameter space and allow the size of the approximating space to grow as the sample size increases this method is called the method of sieves in the case of the maximum likelihood estimation an mle based on a sieve is called a sieve mle we found that the convergence rate of a sieve estimate is governed by a the local expected values variances and l 2 entropy of the criterion differences and b the approximation error of the sieve a robust nonparametric regression problem a mixture problem and a nonparametric regression problem are discussed as illustrations of the theory we also found that when the underlying space is too large the estimate based on optimizing over the whole parameter space may not achieve the best possible rates of convergence whereas the sieve estimate typically does not suffer from this difficulty

quantile regression in reproducing kernel hilbert spaces in this article we consider quantile regression in reproducing kernel hilbert spaces which we call kernel quantile regression kqr we make three contributions 1 we propose an efficient algorithm that computes the entire solution path of the kqr with essentially the same computational cost as fitting one kqr model 2 we derive a simple formula for the effective dimension of the kqr model which allows convenient selection of the regularization parameter and 3 we develop an asymptotic theory for the kqr model

on the degrees of freedom of the lasso we study the effective degrees of freedom of the lasso in the framework of stein s unbiased risk estimation sure we show that the number of nonzero coefficients is an unbiased estimate forthe degrees of freedom of the lasso a conclusion that requires no special assumption on the predictors in addition the unbiased estimator is shown to be asymptotically consistent with these results on hand various model selection criteria cp aic and bic are available which along with the lars algorithm provide a principled and efficient approach to obtaining the optimal lasso fit with the computational effort of a single ordinary least squares fit

segmented model selection in quantile regression using the minimum description length principle this article proposes new model fitting techniques for quantiles of an observed data sequence including methods for data segmentation and variable selection the main contribution however is in providing a means to perform these two tasks simultaneously this is achieved by matching the data with the best fitting piecewise quantile regression model where the fit is determined by a penalization derived from the minimum description length principle the resulting optimization problem is solved with the use of genetic algorithms the proposed fully automatic procedures are unlike traditional break point procedures not based on repeated hypothesis tests and do not require unlike most variable selection procedures the specification of a tuning parameter theoretical large sample properties are derived empirical comparisons with existing break point and variable selection methods for quantiles indicate that the new procedures work well in practice

how biased is the apparent error rate of a prediction rule abstract a regression model is fitted to an observed set of data how accurate is the model for predicting future observations the apparent error rate tends to underestimate the true error rate because the data have been used twice both to fit the model and to check its accuracy we provide simple estimates for the downward bias of the apparent error rate the theory applies to general exponential family linear models and general measures of prediction error special attention is given to the case of logistic regression on binary data with error rates measured by the proportion of misclassified cases several connected ideas are compared mallows s cp cross validation generalized cross validation the bootstrap and akaike s information criterion

freeness of partial words the paper approaches the classical combinatorial problem of freeness of words in the more general case of partial words first we propose an algorithm that tests efficiently whether a partial word is k free or not for a given k then we show that there exist arbitrarily many k free infinite partial words over a binary alphabet containing an infinite number of holes for k 3 moreover we present an efficient algorithm for the construction of a cube free partial word with a given number of holes over a binary alphabet in the final section of the paper we show that there exists an infinite word over a four symbol alphabet in which we can substitute randomly one symbol with a hole and still obtain a cube free word we show that such a word does not exist for alphabets with fewer symbols further we prove that in this word we can replace arbitrarily many symbols with holes such that each two consecutive holes are separated by at least two symbols and obtain a cube free partial word this result seems interesting because any partial word containing two holes with less than two symbols between them is not cube free finally we modify the previously presented algorithm to construct over a four symbol alphabet a cube free partial word with exactly n holes having minimal length among all the possible cube free partial words with at least n holes

the reconstruction approach from interpolation to regression this article introduces an interpolation based method called the reconstruction approach for nonparametric regression based on the fact that interpolation usually has negligible errors compared

on measuring and correcting the effects of data mining and model selection abstract in the theory of linear models the concept of degrees of freedom plays an important role this concept is often used for measurement of model complexity for obtaining an unbiased estimate of the error variance and for comparison of different models i have developed a concept of generalized degrees of freedom gdf that is applicable to complex modeling procedures the definition is based on the sum of the sensitivity of each fitted value to perturbation in the corresponding observed value the concept is nonasymptotic in nature and does not require analytic knowledge of the modeling procedures the concept of gdf offers a unified framework under which complex and highly irregular modeling procedures can be analyzed in the same way as classical linear models by using this framework many difficult problems can be solved easily for example one can now measure the number of observations used in a variable selection process different modeling procedures such as a tree based regression and a

local linear quantile regression abstract in this article we study nonparametric regression quantile estimation by kernel weighted local linear fitting two such estimators are considered one is based on localizing the characterization of a regression quantile as the minimizer of e pp y a x x where p is the appropriate check function the other follows by inverting a local linear conditional distribution estimator and involves two smoothing parameters rather than one our aim is to present fully operational versions of both approaches and to show that each works quite well although either might be used in practice we have a particular preference for the second our automatic smoothing parameter selection method is novel the main regression quantile smoothing parameters are chosen by rule of thumb adaptations of state of the art methods for smoothing parameter selection for regression mean estimation the techniques are illustrated by application to two datasets and compared in simulations

censored median regression using weighted empirical survival and hazard functions abstract for median regression models that regress the median of the survival time or a transform thereof on the covariates some semi parametric estimators that include the intercept component are introduced when the survival time may be censored these new median regression estimators do not require estimating the censoring distributions they can be viewed as an extension of the sample median to the censored regression model these estimators are based on some weighted empirical survival and hazard functions and are shown to be consistent and asymptotically normal they performed very well in various numerical studies the proposed procedures are illustrated in some real data examples

nonparametric conditional density estimation using piecewise linear solution path of kernel quantile regression the goal of regression analysis is to describe the stochastic relationship between an input vector x and a scalar output y this can be achieved by estimating the entire conditional density p y x in this letter we present a new approach for nonparametric conditional density estimation we develop a piecewise linear path following method for kernel based quantile regression it enables us to estimate the cumulative distribution function of p y x in piecewise linear form for all x in the input domain theoretical analyses and experimental results are presented to show the effectiveness of the approach

nonadaptive group testing based on sparse pooling graphs an information theoretical analysis of nonadaptive group testing schemes based on sparse pooling graphs is presented a pooling graph is a bipartite graph for which the adjacency matrix is a pooling matrix the binary status of the objects to be tested are modeled by independent identically distributed bernoulli random variables with probability p an l r n regular pooling graph is a bipartite graph in which the left nodes have degree l the right nodes have degree r and n is the number of left nodes the main contribution of this paper is a direct coding theorem that gives the conditions for the existence of an estimator that can achieve an arbitrarily small probability of error an estimator is a function that uses observations to infer the state of an object the direct coding theorem is proved by averaging the upper bound on the probability of the estimation error of the typical set estimator over an l r n regular pooling graph ensemble numerical results indicate sharp threshold behaviors in the asymptotic regime these results can provide a concrete benchmark for nonadaptive group testing of existing and emerging detection algorithms over a noiseless system

bivariate quantile smoothing splines it has long been recognized that the mean provides an inadequate summary whereas the set of quantiles can supply a more complete description of a sample we introduce bivariate quantile smoothing splines which belong to the space of bilinear tensor product splines as nonparametric estimators for the conditional quantile functions in a two dimensional design space the estimators can be computed by using standard linear programming techniques and can further be used as building blocks for conditional quantile estimations in higher dimensions for moderately large data sets we recommend penalized bivariate b splines as approximate solutions we use real and simulated data to illustrate the methodology proposed

hierarchical spline models for conditional quantiles and the demand for electricity abstract methods for estimating nonparametric models for conditional quantiles are suggested based on the regression quantile methods of koenker and bassett spline parameterizations of the conditional quantile functions are used the methods are illustrated by estimating hierarchical models for household electricity demand using data from the chicago metropolitan area the empirical results show that lower quantiles of demand base load vary only slightly across residential households this variability is difficult to explain using household characteristics however upper quantiles of the demand distribution vary considerably and are systematically related to household characteristics and appliance ownership the implications of analyzing mean demand behavior rather than various quantiles of the distribution of demand are also discussed

on the degrees of freedom in shape restricted regression for the problem of estimating a regression function say subject to shape constraints like monotonicity or convexity it is argued that the divergence of the maximum likelihood estimator provides a useful measure of the effective dimension of the model inequalities are derived for the expected mean squared error of the maximum likelihood estimator and the expected residual sum of squares these generalize equalities from the case of linear regression as an application it is shown that the maximum likelihood estimator of the error variance 2 is asymptotically normal with mean 2 and variance 2 2 n for monotone regression it is shown that the maximum likelihood estimator of attains the optimal rate of convergence and a bias correction to the maximum likelihood estimator of 2 is derived

on quantile regression in reproducing kernel hilbert spaces with the data sparsity constraint for spline regressions it is well known that the choice of knots is crucial for the performance of the estimator as a general learning framework covering the smoothing splines learning in a reproducing kernel hilbert space rkhs has a similar issue however the selection of training data points for kernel functions in the rkhs representation has not been carefully studied in the literature in this paper we study quantile regression as an example of learning in a rkhs in this case the regular squared norm penalty does not perform training data selection we propose a data sparsity constraint that imposes thresholding on the kernel function coefficients to achieve a sparse kernel function representation we demonstrate that the proposed data sparsity method can have competitive prediction performance for certain situations and have comparable performance in other cases compared to that of the traditional squared norm penalty therefore the data sparsity method can serve as a competitive alternative to the squared norm penalty method some theoretical properties of our proposed method using the data sparsity constraint are obtained both simulated and real data sets are used to demonstrate the usefulness of our data sparsity constraint