on the estimation of a probability density function by the maximum penalized likelihood method abstract a class of probability density estimates can be obtained by penalizing the likelihood by a functional which depends on the roughness of the logarithm of the density the limiting case of the estimates as the amount of smoothing increasing has a natural form which makes the method attractive for data analysis and which provides a rationale for a particular choice of roughness penalty the estimates are shown to be the solution of an unconstrained convex optimization problem and mild natural conditions are given for them to exist rates of consistency in various norms and conditions for asymptotic normality and approximation by a gaussian process are given thus breaking new ground in the theory of maximum penalized likelihood density estimation author

marginal longitudinal nonparametric regression locality and efficiency of spline and kernel methods we consider nonparametric regression in a longitudinal marginal model of generalized estimating equation gee type with a time varying covariate in the situation where the number of observations per subject is finite and the number of subjects is large in such models the basic shape of the regression function is affected only by the covariate values and not otherwise by the ordering of the observations two methods of estimating the nonparametric function can be considered kernel methods and spline methods recently surprising evidence has emerged suggesting that for kernel methods previously proposed in the literature it is generally asymptotically preferable to ignore the correlation structure in our marginal model and instead assume that the data are independent that is working independence in the gee jargon as seen through equivalent kernel results in univariate independent data problems splines and kernels have similar behavior smoothing splines are equivalent to kernel regression with a s

a fast and efficient cross validation method for smoothing parameter choice in spline regression abstract the spline smoothing approach to nonparametric regression is considered with particular reference to the problem of choosing how much to smooth both computational and statistical aspects of the method of cross validation are discussed an approximate cross validation method is proposed that requires a small linear amount of computer time a simulation study indicates that the method has very good statistical properties some mathematical justification and motivation are also provided

convolutional autoregressive models for functional time series abstract functional data analysis has became an increasingly popular class of problems in statistical research however functional data observed over time with serial dependence remains a less studied area motivated by bosq 2000 who first introduced the functional autoregressive models we propose a convolutional functional autoregressive model where the function at time t is a result of the sum of convolutions of the past functions and a set of convolution functions plus a noise process mimicking the vector autoregressive process it provides an intuitive and direct interpretation of the dynamics of a stochastic process instead of principal component analysis commonly used in functional data analysis we adopt a sieve estimation procedure based on b spline approximation of the convolution functions we establish convergence rate of the proposed estimator and investigate its theoretical properties the model building model validation and prediction procedures are also developed both simulated and real data examples are presented

smoothing splines regression derivatives and deconvolution abstract the statistical properties of a cubic smoothing spline and its derivative are analyzed it is shown that unless unnatural boundary conditions hold the integrated squared bias is dominated by local effects near the boundary similar effects are shown to occur in the regularized solution of a translation kernel integral equation these results are derived by developing a fourier representation for a smoothing spline author

a unified framework for spline estimators this dissertation develops a uni ed framework to study the asymptotic properties of all periodic spline based estimators that is of regression penalized and smoothing splines the explicit form of the periodic demmler reinsch basis of general degree in terms of exponential splines allows to derive the exact expression for the equivalent kernel of all spline estimators simultaneously the corresponding bandwidth which drives the asymptotic behavior of spline estimators is shown to be a function of both the number of knots and the smoothing parameter a strategy for the optimal bandwidth selection is discussed

spline smoothing the equivalent variable kernel method on considere une approche spline pour la regression non parametrique et l estimation de courbe on montre que dans un certain sens le lissage spline correspond approximativement au lissage par une methode du noyau avec une largeur de bande dependant de la densite locale des points du plan de regression

inverse passive learning of an input output map through update spline smoothing this paper presents a robust method of learning passively a one dimensional input output map when receiving only indirect information about the correct input output map e g only the sign of the deviation between the actual estimated output value and the correct output value is obtained this information is obtained for only one input output combination per updating cycle the approach is to increment or decrement step by step the output values of the actually stored map and then to apply global or local cubic spline smoothing in order to avoid adaptation holes at points which are never updated or less frequently updated than other points this method works with noisy measurements as well as slowly time varying systems even discontinuous changes of the desired input output relation do not result in instability problems of convergence and stability are treated and design rules are given

a class of grouped brunk estimators and penalized spline estimators for monotone regression we study a class of monotone univariate regression estimators we use b splines to approximate an underlying regression function and estimate spline coefficients based on grouped data we investigate asymptotic properties of two monotone estimators a grouped brunk estimator and a penalized monotone estimator these estimators are consistent at the boundary and their mean square errors achieve optimal convergence rates under suitable assumptions of the true regression function asymptotic distributions are developed and are shown to be independent of spline degrees and the number of knots simulation results and car data illustrate performance of the proposed estimators

linear smoothers and additive models we study linear smoothers and their use in building nonparametric regression models in the first part of this paper we examine certain aspects of linear smoothers for scatterplots examples of these are the running mean and running line kernel and cubic spline smoothers the eigenvalue and singular value decompositions of the corresponding smoother matrix are used to describe qualitatively a smoother and several other topics such as the number of degrees of freedom of a smoother are discussed in the second part of the paper we describe how linear smoothers can be used to estimate the additive model a powerful nonparametric regression model using the backfitting algorithm we show that backfitting is the gauss seidel iterative method for solving a set of normal equations associated with the additive model we provide conditions for consistency and nondegeneracy and prove convergence for the backfitting and related algorithms for a class of smoothers that includes cubic spline smoothers

automated smoothing of image and other regularly spaced data this paper is primarily motivated by the problem of automatically removing unwanted noise from high dimensional remote sensing imagery the initial step involves the transformation of the data to a space of intrinsically lower dimensionality and the smoothing of images in the new space different images require different amounts of smoothing the signal assumed to be mostly smooth with relatively few discontinuities is estimated from the data using the method of generalized cross validation it is shown how the generalized cross validated thin plate smoothing spline with observations on a regular grid in d dimensions is easily approximated and computed in the fourier domain space domain approximations are also investigated the technique is applied to some remote sensing data

piecewise convex function estimation pilot estimators given noisy data function estimation is considered when the unknown function is known a priori to be either convex or concave on each of a small number of regions where the function when the number of regions is unknown the model selection problem is to determine the number of convexity change points for kernel estimates in gaussian noise the number of false change points is evaluated as a function of the smoothing parameter to insure that the number of false convexity change points tends to zero the smoothing level must be larger than is generically optimal for minimizing the mean integrated square error mise a two stage estimator is proposed and shown to achieve the optimal rate of convergence of the mise in the first stage the number and location of the change points is estimated using strong smoothing in the second stage a constrained smoothing spline fit is performed with the smoothing level chosen to minimize the mise the imposed constraint is that a single change point occur in a region about each empirical change point from the first stage estimate this constraint is equivalent to the requirement that the third derivative of the second stage estimate has a single sign in a small neighborhood about each first stage change point the change points from the second stage are first stage change points but need not be at the identical locations

asymptotically exact nonparametric hypothesis testing in sup norm and at a fixed point for the signal in gaussian white noise model we consider the problem of testing the hypothesis h 0 f 0 the signal f is zero against the nonparametric alternative h 1 f where is a set of functions on r 1 of the form f f f c here is a holder or sobolev class of functions f is either the sup norm of f or the value of f at a fixed point c 0 is a constant is the minimax rate of testing and 0 is the asymptotic parameter of the model we find exact separation constants c 0 such that a test with the given summarized asymptotic errors of first and second type is possible for c c and is not possible for c c we propose asymptotically minimax test statistics

estimation in partially linear models with missing covariates the partially linear model y xt z has been studied extensively when data are completely observed in this article we consider the case where the covariate x is sometimes missing with missingness probability depending on y z new methods are developed for estimating and our methods are shown to outperform asymptotically methods based only on the complete data asymptotic efficiency is discussed and the semiparametric efficient score function is derived justification of the use of the nonparametric bootstrap in this context is sketched the proposed estimators are extended to a working independence analysis of longitudinal clustered data and applied to analyze an aids clinical trial dataset the results of a simulation experiment are also given to illustrate our approach

empirical functionals and efficient smoothing parameter selection a striking feature of curve estimation is that the smoothing parameter h 0 which minimizes the squared error of a kernel or smoothing spline estimator is very difficult to estimate this is manifest both in slow rates of convergence and in high variability of standard methods such as cross validation we quantify this difficulty by describing nonparametric information bounds and exhibit asymptotically efficient estimators of h 0 that attain the bounds the efficient estimators are substantially less variable than cross validation and other current procedures and simulations suggest that they may offer improvements at moderate sample sizes at least in terms of minimizing the squared error

maximum entropy autoregressive conditional heteroskedasticity model abstract in many applications it has been found that the autoregressive conditional heteroskedasticity arch model under the conditional normal or student s t distributions are not general enough to account for the excess kurtosis in the data moreover asymmetry in the financial data is rarely modeled in a systematic way in this paper we suggest a general density function based on the maximum entropy me approach that takes account of asymmetry excess kurtosis and also of high peakedness the me principle is based on the efficient use of available information and as is well known many of the standard family of distributions can be derived from the me approach we demonstrate how we can extract information functional from the data in the form of moment functions we also propose a test procedure for selecting appropriate moment functions our procedure is illustrated with an application to the nyse stock returns the empirical results reveal that the me approach with a fewer moment functions leads to a model that captures the stylized facts quite effectively

on different facets of regularization theory this review provides a comprehensive understanding of regularization theory from different perspectives emphasizing smoothness and simplicity principles using the tools of operator theory and fourier analysis it is shown that the solution of the classical tikhonov regularization problem can be derived from the regularized functional defined by a linear differential integral operator in the spatial fourier domain state of the art research relevant to the regularization theory is reviewed covering occam s razor minimum length description bayesian theory pruning algorithms informational entropy theory statistical learning theory and equivalent regularization the universal principle of regularization in terms of kolmogorov complexity is discussed finally some prospective studies on regularization theory and beyond are suggested

biometrika centenary nonparametrics summary contributions in biometrika to nonparametric statistics are surveyed with particular emphasis on more recent work it is argued that even in the journal s very early years it had interests in smoothing methods for density estimation and in techniques for nonparametric regression in addition to its more obvious focus on classical nonparametric statistics such as ranking methods biometrika s early involvement with permutation testing has links to its contemporary work on bootstrap methods emphases on different types of nonparametric statistics have evolved over time however a rapidly increasing proportion of the journal s pages is devoted to papers that include some use of nonparametric ideas or methods

on the asymptotics of penalized splines summavry we study the asymptotic behaviour of penalized spline estimators in the univariate case we use b splines and a penalty is placed on mth order differences of the coefficients the number of knots is assumed to converge to infinity as the sample size increases we show that penalized splines behave similarly to nadaraya watson kernel estimators with equivalent kernels depending upon m the equivalent kernels we obtain for penalized splines are the same as those found by silverman for smoothing splines the asymptotic distribution of the penalized spline estimator is gaussian and we give simple expressions for the asymptotic mean and variance provided that it is fast enough the rate at which the number of knots converges to infinity does not affect the asymptotic distribution the optimal rate of convergence of the penalty parameter is given penalized splines are not design adaptive

conditional minimum volume predictive regions for stochastic processes abstract motivated by interval region prediction in nonlinear time series we propose a minimum volume mv predictor for a strictly stationary process the mv predictor varies with respect to the current position in the state space and has the minimum lebesgue measure among all regions with the nominal coverage probability we have established consistency convergence rates and asymptotic normality for both coverage probability and lebesgue measure of the estimated mv predictor under the assumption that the observations are from a strong mixing process applications with both real and simulated datasets illustrate the proposed methods

bayesian confidence intervals for smoothing splines abstract the frequency properties of wahba s bayesian confidence intervals for smoothing splines are investigated by a large sample approximation and by a simulation study when the coverage probabilities for these pointwise confidence intervals are averaged across the observation points the average coverage probability acp should be close to the nominal level from a frequency point of view this agreement occurs because the average posterior variance for the spline is similar to a consistent estimate of the average squared error and because the average squared bias is a modest fraction of the total average squared error these properties are independent of the bayesian assumptions used to derive this confidence procedure and they explain why the acp is accurate for functions that are much smoother than the sample paths prescribed by the prior this analysis accounts for the choice of the smoothing parameter bandwidth using cross validation in the case of natural splines an adaptive method for avo

kernel smoothing of data with correlated errors abstract kernel smoothing is a common method of estimating the mean function in the nonparametric regression model y f x e where f x is a smooth deterministic mean function and e is an error process with mean zero in this article the mean squared error of kernel estimators is computed for processes with correlated errors and the estimators are shown to be consistent when the sequence of error processes converges to a mixing sequence the standard techniques for bandwidth selection such as cross validation and generalized cross validation are shown to perform very badly when the errors are correlated standard selection techniques are shown to favor undersmoothing when the correlations are predominantly positive and oversmoothing when negative the selection criteria can however be adjusted to correct for the effect of correlation in simulations the standard selection criteria are shown to behave as predicted the corrected criteria are shown to be very effective when the correlation functi

on degenerations of modules abstract we generalize a result of zwara concerning the degeneration of modules over artinian algebras to that over general algebras in fact let r be any algebra over a field and let m and n be finitely generated left r modules then we show that m degenerates to n if and only if there is a short exact sequence of finitely generated left r modules 0 z m z n 0 such that the endomorphism on z is nilpotent we give several applications of this theorem to commutative ring theory

on bandwidth variation in kernel estimates a square root law we consider kernel estimation of a smooth density f at a point but depart from the usual approach in admitting an adaptive dependence of the sharpness of the kernels on the underlying density proportionally varying the bandwidths like f 1 2 at the contributing readings lowers the bias to a vanishing fraction of the usual value and makes for performance seen in well known estimators that force moment conditions on the kernel and so sacrifice positivity of the curve estimate issues of equivariance and variance stabilitization are treated

choosing a range for the amount of smoothing in nonparametric regression abstract most nonparametric regression estimates smooth the observed data in the process of generating a suitable estimate the resulting curve estimate is sensitive to the amount of smoothing introduced and it is important to examine the estimate over a range of smoothing choices an objective way to specify such a range is to estimate the optimal value of the smoothing parameter bandwidth with respect to some loss function and then report a confidence interval for this parameter estimate this interval suggests examining three regression estimates an undersmoothed curve evaluated at the lower endpoint of the confidence interval an oversmoothed curve evaluated at the upper endpoint and also an optimal curve chosen by the data based method this article describes two strategies for constructing such confidence intervals using asymptotic approximations and simulation techniques suppose that data are observed from the model yk f tk ek 1 k n where f is a function that is twice differentia

non parametric identification of non linear oscillating systems the problem of system identification from a time series of measurements is solved by using non parametric additive models having only few structural information about the system a non parametric approach may be more appropriate than a parametric one for which detailed prior knowledge is needed based on non parametric regression the functions in the additive models are estimated by a penalized least squares approach using backfitting the optimal smoothing parameters are determined via generalized cross validation making this approach completely adaptive to the data the procedure is applied to identify the non linear restoring force of vibrationally excited helical wire rope isolators