nonparametric least squares estimation of a multivariate convex regression function this paper deals with the consistency of the least squares estimator of a convex regression function when the predictor is multidimensional we characterize and discuss the computation of such an estimator via the solution of certain quadratic and linear programs mild sufficient conditions for the consistency of this estimator and its subdifferentials in fixed and stochastic design regression settings are provided we also consider a regression function which is known to be convex and componentwise nonincreasing and discuss the characterization computation and consistency of its least squares estimator

global risk bounds and adaptation in univariate convex regression we consider the problem of nonparametric estimation of a convex regression function phi 0 we study the risk of the least squares estimator lse under the natural squared error loss we show that the risk is always bounded from above by n 4 5 modulo logarithmic factors while being much smaller when phi 0 is well approximable by a piecewise affine convex function with not too many affine pieces in which case the risk is at most 1 n up to logarithmic factors on the other hand when phi 0 has curvature we show that no estimator can have risk smaller than a constant multiple of n 4 5 in a very strong sense by proving a local minimax lower bound we also study the case of model misspecification where we show that the lse exhibits the same global behavior provided the loss is measured from the closest convex projection of the true regression function in the process of deriving our risk bounds we prove new results for the metric entropy of local neighborhoods of the space of univariate convex functions these results which may be of independent interest demonstrate the non uniform nature of the space of univariate convex functions in sharp contrast to classical function spaces based on smoothness constraints

testing and estimating shape constrained nonparametric density and regression in the presence of measurement error in many applications we can expect that or are interested to know if a density function or a regression curve satisfies some specific shape constraints for example when the explanatory variable x represents the value taken by a treatment or dosage the conditional mean of the response y is often anticipated to be a monotone function of x indeed if this regression mean is not monotone in the appropriate direction then the medical or commercial value of the treatment is likely to be significantly curtailed at least for values of x that lie beyond the point at which monotonicity fails in the case of a density common shape constraints include log concavity and unimodality if we can correctly guess the shape of a curve then nonparametric estimators can be improved by taking this information into account addressing such problems requires a method for testing the hypothesis that the curve of interest satisfies a shape constraint and if the conclusion of the test is positive a technique for e

semiconvex regression for metamodeling based optimization stochastic search involves finding a set of controllable parameters that minimizes an unknown objective function using a set of noisy observations we consider the case when the unknown function is convex and a metamodel is used as a surrogate objective function often the data are non i i d and include an observable state variable such as applicant information in a loan rate decision problem state information is difficult to incorporate into convex models we propose a new semiconvex regression method that is used to produce a convex metamodel in the presence of a state variable we show consistency for this method we demonstrate its effectiveness for metamodeling on a set of synthetic inventory management problems and a large real life auto loan dataset

unified lasso estimation by least squares approximation we propose a method of least squares approximation lsa for unified yet simple lasso estimation our general theoretical framework includes ordinary least squares generalized linear models quantile regression and many others as special cases specifically lsa can transfer many different types of lasso objective functions into their asymptotically equivalent least squares problems thereafter the standard asymptotic theory can be established and the lars algorithm can be applied in particular if the adaptive lasso penalty and a bayes information criterion type tuning parameter selector are used the resulting lsa estimator can be as efficient as the oracle extensive numerical studies confirm our theory

visual error criteria for qualitative smoothing abstract an important gap between the classical mathematical theory and the practice and implementation of nonparametric curve estimation is due to the fact that the usual norms on function spaces measure something different from what the eye can see visually in a graphical presentation mathematical error criteria that more closely follow visual impression are developed and analyzed from both graphical and mathematical viewpoints examples from wavelet regression and kernel density estimation are considered

a marginalized zero inflated poisson regression model with overall exposure effects the zero inflated poisson zip regression model is often employed in public health research to examine the relationships between exposures of interest and a count outcome exhibiting many zeros in excess of the amount expected under sampling from a poisson distribution the regression coefficients of the zip model have latent class interpretations which correspond to a susceptible subpopulation at risk for the condition with counts generated from a poisson distribution and a non susceptible subpopulation that provides the extra or excess zeros the zip model parameters however are not well suited for inference targeted at marginal means specifically in quantifying the effect of an explanatory variable in the overall mixture population we develop a marginalized zip model approach for independent responses to model the population mean count directly allowing straightforward inference for overall exposure effects and empirical robust variance estimation for overall log incidence density ratios through simulation studies the performance of maximum likelihood estimation of the marginalized zip model is assessed and compared with other methods of estimating overall exposure effects the marginalized zip model is applied to a recent study of a motivational interviewing based safer sex counseling intervention designed to reduce unprotected sexual act counts

yield curve estimation by kernel smoothing methods we introduce a new method for the estimation of discount functions yield curves and forward curves from government issued coupon bonds our approach is nonparametric and does not assume a particular functional form for the discount function although we do show how to impose various restrictions in the estimation our method is based on kernel smoothing and is defined as the minimum of some localized population moment condition the solution to the sample problem is not explicit and our estimation procedure is iterative rather like the backfitting method of estimating additive nonparametric models we establish the asymptotic normality of our methods using the asymptotic representation of our estimator as an infinite series with declining coefficients the rate of convergence is standard for one dimensional nonparametric regression we investigate the finite sample performance of our method in comparison with other well established methods in a small simulation experiment

zero inflated poisson regression with an application to defects in manufacturing zero inflated poisson zip regression is a model for count data with excess zeros it assumes that with probability p the only possible observation is 0 and with probability 1 p a poisson random variable is observed for example when manufacturing equipment is properly aligned defects may be nearly impossible but when it is misaligned defects may occur according to a poisson distribution both the probability p of the perfect zero defect state and the mean number of defects in the imperfect state may depend on covariates sometimes p and are unrelated other times p is a simple function of such as p l 1 t for an unknown constant t in either case zip regression models are easy to fit the maximum likelihood estimates mle s are approximately normal in large samples and confidence intervals can be constructed by inverting likelihood ratio tests or using the approximate normality of the mle s simulations suggest that the confidence intervals based on likelihood ratio test

a canonical process for estimation of convex functions the invelope of integrated brownian motion t4 a process associated with integrated brownian motion is introduced that characterizes the limit behavior of nonparametric least squares and maximum likelihood estimators of convex functions and convex densities respectively we call this process the invelope and show that it is an almost surely uniquely defined function of integrated brownian motion its role is comparable to the role of the greatest convex minorant of brownian motion plus a parabolic drift in the problem of estimating monotone functions an iterative cubic spline algorithm is introduced that solves the constrained least squares problem in the limit situation and some results obtained by applying this algorithm are shown to illustrate the theory

improving point and interval estimators of monotone functions by rearrangement suppose that a target function is monotonic and an available original estimate of this target function is not monotonic rearrangements univariate and multivariate transform the original estimate to a monotonic estimate that always lies closer in common metrics to the target function furthermore suppose an original confidence interval which covers the target function with probability at least 1 is defined by an upper and lower endpoint functions that are not monotonic then the rearranged confidence interval defined by the rearranged upper and lower endpoint functions is monotonic shorter in length in common norms than the original interval and covers the target function with probability at least 1 we illustrate the results with a growth chart example

new goodness of fit tests and their application to nonparametric confidence sets suppose one observes a process v on the unit interval where dv f 0 dw with an unknown parameter f 0 l 1 0 1 and standard brownian motion w we propose a particular test of one point hypotheses about f 0 which is based on suitably standardized increments of v this test is shown to have desirable consistency properties if for instance f 0 is restricted to various holder classes of functions the test is mimicked in the context of nonparametric density estimation nonparametric regression and interval censored data under shape restrictions on the parameter such as monotonicity or convexity we obtain confidence sets for f 0 adapting to its unknown smoothness

nonparametric regression confidence regions and regularization in this paper we offer a unified approach to the problem of nonparametric regression on the unit interval it is based on a universal honest and nonasymptotic confidence region a n which is defined by a set of linear inequalities involving the values of the functions at the design points interest will typically center on certain simplest functions in a n where simplicity can be defined in terms of shape number of local extremes intervals of convexity concavity or smoothness bounds on derivatives or a combination of both once some form of regularization has been decided upon the confidence region can be used to provide honest nonasymptotic confidence bounds which are less informative but conceptually much simpler

smooth discrimination analysis ties f and g can be based on the estimation of the set g x fx gx we consider applications where it is appropriate to assume that the region g has a smooth boundary or belongs to another nonparametric class of sets in particular this assumption makes sense if discrimination is used as a data analytic tool decision rules based on minimization of empirical risk over the whole class of sets and over sieves are considered their rates of convergence are obtained we show that these rules achieve optimal rates for estimation of g and optimal rates of convergence for bayes risks an interesting conclusion is that the optimal rates for bayes risks can be very fast in particular faster than the parametric root n rate these fast rates cannot be guaranteed for plug in rules

semi nonparametric estimation of the call option price surface under strike and time to expiry no arbitrage constraints abstract we suggest a semi nonparametric estimator for the call option price surface the estimator is a bivariate tensor product b spline to enforce no arbitrage constraints across strikes and expiry dates we establish sufficient no arbitrage conditions on the control net of the b spline surface the conditions are linear and therefore allow for an implementation of the estimator by means of standard quadratic programming techniques the consistency of the estimator is proved by means of simulations we explore the statistical efficiency benefits that are associated with estimating option price surfaces and state price densities under the full set of no arbitrage constraints we estimate a call option price surface families of first order strike derivatives and state price densities for s p 500 option data

locally adaptive regression splines in this paper least squares penalized regression estimates withtotal variation penalities are considered it is shown that theseestimators are least squares splines with locally data adaptive placed knotpoints algorithms and asymptotic properties are discussed

group regularization for zero inflated negative binomial regression models with an application to health care demand in germany in many biomedical applications covariates are naturally grouped with variables in the same group being systematically related or statistically correlated under such settings variable selection must be conducted at both group and individual variable levels motivated by the widespread availability of zero inflated count outcomes and grouped covariates in many practical applications we consider group regularization for zero inflated negative binomial regression models using a least squares approximation of the mixture likelihood and a variety of group wise penalties on the coefficients we propose a unified algorithm gooogle group regularization for zero inflated count regression models to efficiently compute the entire regularization path of the estimators we investigate the finite sample performance of these methods through extensive simulation experiments and the analysis of a german health care demand dataset finally we derive theoretical properties of these methods under reasonable assumptions which further provides deeper insight into the asymptotic behavior of these approaches the open source software implementation of this method is publicly available at https github com himelmallick gooogle

piecewise convex function estimation pilot estimators given noisy data function estimation is considered when the unknown function is known a priori to be either convex or concave on each of a small number of regions where the function when the number of regions is unknown the model selection problem is to determine the number of convexity change points for kernel estimates in gaussian noise the number of false change points is evaluated as a function of the smoothing parameter to insure that the number of false convexity change points tends to zero the smoothing level must be larger than is generically optimal for minimizing the mean integrated square error mise a two stage estimator is proposed and shown to achieve the optimal rate of convergence of the mise in the first stage the number and location of the change points is estimated using strong smoothing in the second stage a constrained smoothing spline fit is performed with the smoothing level chosen to minimize the mise the imposed constraint is that a single change point occur in a region about each empirical change point from the first stage estimate this constraint is equivalent to the requirement that the third derivative of the second stage estimate has a single sign in a small neighborhood about each first stage change point the change points from the second stage are first stage change points but need not be at the identical locations

local asymptotics for quantile smoothing splines quantile smoothing splines were introduced by koenker ng and portnoy as natural and appealing estimates of conditional quantiles of response variables the natural setting for the problem considers minimization of a weighted combination of a fit penalty and a roughness penalty over the space of functions whose derivatives have bounded variation although this space is not traditional shen has shown recently that the quantile smoothing splines do indeed converge at the usual optimal rate n 2 5 in various norms here local asymptotic results are obtained by establishing bahadur representations for local parameters of the splines these are used to obtain local rates of convergence to establish uniform convergence rates to provide local distribution theory for quantile b splines and to expand the fit measure in order to analyze an information criterion for determining the smoothing parameter examples of using derivatives of the smoothing splines for estimating jump functions are also presented

variational solution of penalized likelihood problems and smooth curve estimation usual nonparametric regression estimators often show many little wiggles which do not appear to be necessary for a good description of the data the new wp smoother is a maximum penalized likelihood mpl estimate with a novel roughness penalty it penalizes a relative change of curvature this leads to disjoint classes of functions each with a given number n w of inflection points for a wp estimate f x x w 1 x w nw exp h f x which is semiparametric with parameters w j and nonparametric part h f the main mathematical result is a convenient form of the characterizing differential equation for a very general class of mpl estimators

optimal rates of convergence for convex set estimation from support functions nitely many noisy measurements of its support function the solution is based on appropriate regularizations of the least squares estimator both xed and random designs are considered 1 introduction we study the nonparametric estimation problem of estimating a compact convex set in euclidean space from noisy support function measurements the support function hk of a compact convex subset k of r d d 2 is dened for u in the unit sphere s d 1 fx2 r d x 2 x 2 1g by

unitary integration with operator splitting for weakly dissipative systems unitary integration is a numerical method that preserves the structure of the quantum liouville equation by evolving the density via unitary transformations unitary integrators preserve the kinematic invariants cj tr j j 1 n to all orders in the time step here we extend unitary integration to weakly dissipative systems we apply the technique of operator splitting using a unitary integrator for the hamiltonian evolution and a conventional integrator for the dissipative piece in this way we guarantee that all dissipation and decoherence variation of the cj is due to the new non hamiltonian terms and not to any numerical artifacts we illustrate the method with examples

signs of divided differences yield least squares data fitting with constrained monotonicity or convexity methods are presented for least squares data smoothing by using the signs of divided differences of the smoothed values professor m j d powell initiated the subject in the early 1980s and since then theory algorithms and fortran software make it applicable to several disciplines in various ways let us consider n data measurements of a univariate function which have been altered by random errors then it is usual for the divided differences of the measurements to show sign alterations which are probably due to data errors we make the least sum of squares change to the measurements by requiring the sequence of divided differences of order m to have at most q sign changes for some prescribed integer q the positions of the sign changes are integer variables of the optimization calculation which implies a combinatorial problem whose solution can require about o nq quadratic programming calculations in n variables and n m constraints suitable methods have been developed for the following cases it has been found that a dynamic programming procedure can calculate the global minimum for the important cases of piecewise monotonicity m 1 q 1 and piecewise convexity concavity m 2 q 1 of the smoothed values the complexity of the procedure in the case of m 1 is o n2 qn log2 n computer operations while it is reduced to only o n when q 0 monotonicity and q 1 increasing decreasing monotonicity the case m 2 q 1 requires o qn2 computer operations and n2 quadratic programming calculations which is reduced to one and n 2 quadratic programming calculations when m 2 q 0 i e convexity and m 2 q 1 i e convexity concavity respectively unfortunately the technique that receives this efficiency cannot generalize for the highly nonlinear case m 3 q 2 however the case m 3 q 0 is solved by a special strictly convex quadratic programming calculation and the case m 3 q 1 can be solved by at most 2 n m applications of the previous algorithm also as m gets larger large sets of active constraints usually occur at the optimal approximation which makes the calculation for higher values of q less expensive than what it seems to further attention is given to the sensitivity of the solution with respect to changes in the constraints and the data the smoothing technique is an active research topic and there seems to be room for further developments one strong reason for studying methods that make use of divided differences for data smoothing is that whereas only the data are provided the achieved accuracy goes much beyond the accuracy of the data at an order determined automatically by the chosen divided differences

a group bridge approach for variable selection in multiple regression problems when covariates can be naturally grouped it is important to carry out feature selection at the group and within group individual variable levels simultaneously the existing methods including the lasso and group lasso are designed for either variable selection or group selection but not for both we propose a group bridge approach that is capable of simultaneous selection at both the group and within group individual variable levels the proposed approach is a penalized regularization method that uses a specially designed group bridge penalty it has the oracle group selection property in that it can correctly select important groups with probability converging to one in contrast the group lasso and group least angle regression methods in general do not possess such an oracle property in group selection simulation studies indicate that the group bridge has superior performance in group and individual variable selection relative to several existing methods

quantile and probability curves without crossing this paper proposes a method to address the longstanding problem of lack of monotonicity in estimation of conditional and structural quantile functions also known as the quantile crossing problem bassett and koenker 1982 the method consists in sorting or monotone rearranging the original estimated non monotone curve into a monotone rearranged curve we show that the rearranged curve is closer to the true quantile curve than the original curve in finite samples establish a functional delta method for rearrangement related operators and derive functional limit theory for the entire rearranged curve and its functionals we also establish validity of the bootstrap for estimating the limit law of the entire rearranged curve and its functionals our limit results are generic in that they apply to every estimator of a monotone function provided that the estimator satisfies a functional central limit theorem and the function satisfies some smoothness conditions consequently our results apply to estimation of other econometric functions with monotonicity restrictions such as demand production distribution and structural distribution functions we illustrate the results with an application to estimation of structural distribution and quantile functions using data on vietnam veteran status and earnings

estimation of a convex function characterizations and asymptotic theory we study nonparametric estimation of convexregression and density functions by methods of least squares in the regression and density cases and maximum likelihood in the density estimation case we provide characterizations of these estimators prove that they are consistent and establish their asymptotic distributions at a fixed point of positive curvature of the functions estimated the asymptotic distribution theory relies on the existence of an invelope function for integrated two sided brownian motion t 4 which is established in a companion paper by groeneboom jongbloed and wellner

minimax lower bound and optimal estimation of convex functions in the sup norm estimation of convex functions finds broad applications in science and engineering however the convex shape constraint complicates the asymptotic performance analysis of such estimators this technical note is devoted to the minimax optimal estimation of univariate convex functions in a given holder class particularly a minimax lower bound in the supremum norm or simply sup norm is established by constructing a novel family of piecewise quadratic convex functions in the holder class this result along with a recent result on the minimax upper bound gives rise to the optimal rate of convergence for the minimax sup norm risk of convex functions with the holder order between one and two the present technical note provides the first rigorous justification of the optimal minimax risk for convex estimation on the entire interval of interest in the sup norm

a penalized method for multivariate concave least squares with application to productivity analysis abstract we propose a penalized method for the least squares estimator of a multivariate concave regression function this estimator is formulated as a quadratic programing qp problem with o n2 constraints where n is the number of observations computing such an estimator is a very time consuming task and the computational burden rises dramatically as the number of observations increases by introducing a quadratic penalty function we reformulate the concave least squares estimator as a qp with only non negativity constraints this reformulation can be adapted for estimating variants of shape restricted least squares i e the monotonic concave convex least squares the experimental results and an empirical study show that the reformulated problem and its dual are solved significantly faster than the original problem the matlab and r codes for implementing the penalized problems are provided in the paper

adaptive estimation of planar convex sets in this paper we consider adaptive estimation of an unknown planar compact convex set from noisy measurements of its support function on a uniform grid both the problem of estimating the support function at a point and that of estimating the convex set are studied data driven adaptive estimators are proposed and their optimality properties are established for pointwise estimation it is shown that the estimator optimally adapts to every compact convex set instead of a collection of large parameter spaces as in the conventional minimax theory in nonparametric estimation literature for set estimation the estimators adaptively achieve the optimal rate of convergence in both these problems our analysis makes no smoothness assumptions on the unknown sets

some asymptotics for multimodality tests based on kernel density estimates a test due to b w silverman for modality of a probability density is based on counting modes of a kernel density estimator and the idea of critical smoothing an asymptotic formula is given for the expected number of modes this together with other methods establishes the rate of convergence of the critically smoothed bandwidth these ideas are extended to provide insight concerning the behaviour of the test based on bootstrap critical values

bayesian multivariate isotonic regression splines in many applications interest focuses on assessing the relationship between a predictor and a multivariate outcome variable and there may be prior knowledge about the shape of the regression curves for example regression functions that relate dose of a possible risk factor to different adverse outcomes can often be assumed to be nondecreasing in such cases interest focuses on 1 assessing evidence of an overall adverse effect 2 determining which outcomes are most affected and 3 estimating outcome specific regression curves this article proposes a bayesian approach for addressing this problem motivated by multisite tumor data from carcinogenicity experiments a multivariate smoothing spline model is specified that accommodates dependency in the multiple curves through a hierarchical markov random field prior for the basis coefficients while also allowing for residual correlation a gibbs sampler is proposed for posterior computation and the approach is applied to data on body weight and t

integrated brownian motion conditioned to be positive we study the two dimensional process of integrated brownian motion and brownian motion where integrated brownian motion is conditioned to be positive the transition density of this process is derived from the asymptotic behavior of hitting times of the unconditioned process explicit expressions for the transition density in terms of confluent hypergeometric functions are derived and it is shown how our results on the hitting time distributions imply previous results of isozaki watanabe and goldman the conditioned process is characterized by a system of stochastic differential equations sdes for which we prove an existence and unicity result some sample path properties are derived from the sdes and it is shown that t to t 9 10 is a critical curve for the conditioned process in the sense that the expected time that the integral part of the conditioned process spends below any curve t to t alpha is finite for alpha 9 10 and infinite for alpha geq 9 10