automatic bandwidth choice and confidence intervals in nonparametric regression in the present paper we combine the issues of bandwidth choice and construction of confidence intervals in nonparametric regression main emphasis is put on fully data driven methods we modify the n consistent bandwidth selector of hahrdle hall and marron such that it is appropriate for heteroscedastic data and we show how one can optimally choose the bandwidth g of the pilot estimator m g then we consider classical confidence intervals based on kernel estimators with data driven bandwidths and compare their coverage accuracy we propose a method to put undersmoothing with a data driven bandwidth into practice and show that this procedure outperforms explicit bias correction

adapting for heteroscedasticity in linear models in a heteroscedastic linear model it is known that if the variances are a parametric function of the design then one can construct an estimate of the regression parameter which is asymptotically equivalent to the weighted least squares estimate with known variances we show that the same is true when the only thing known about the variances is that they are determined by an unknown but smooth function of the design or the mean response

adaptive estimation over anisotropic functional classes via oracle approach we address the problem of adaptive minimax estimation in white gaussian noise model under lp loss 1 p 1 on the anisotropic nikolskii classes we present the es timation procedure based on a new data driven selection scheme from the family of kernel estimators with varying bandwidths for proposed estimator we establish so called lp norm oracle inequality and use it for deriving minimax adaptive results we prove the existence of rate adaptive estimators and fully characterize behavior of the minimax risk for different relationships between regularity parameters and norm indexes in definitions of the functional class and of the risk in particular some new asymptotics of the minimax risk are discovered including necessary and sufficient conditions for existencea uniformly consistent estimator we provide also with detailed overview of existing methods and results and formulate open problems in adaptive minimax estimation

ideal spatial adaptation by wavelet shrinkage summary with ideal spatial adaptation an oracle furnishes information about how best to adapt a spatially variable estimator whether piecewise constant piecewise polynomial variable knot spline or variable bandwidth kernel to the unknown function estimation with the aid of an oracle offers dramatic advantages over traditional linear estimation by nonadaptive kernels however it is a priori unclear whether such performance can be obtained by a procedure relying on the data alone we describe a new principle for spatially adaptive estimation selective wavelet reconstruction we show that variable knot spline fits and piecewise polynomial fits when equipped with an oracle to select the knots are not dramatically more powerful than selective wavelet reconstruction with an oracle we develop a practical spatially adaptive method riskshrink which works by shrinkage of empirical wavelet coefficients riskshrink mimics the performance of an oracle for selective wavelet reconstruction as well as it is possible to do so a new inequality in multivariate normal decision theory which we call the oracle inequality shows that attained performance differs from ideal performance by at most a factor of approximately 2 log n where n is the sample size moreover no estimator can give a better guarantee than this within the class of spatially adaptive procedures riskshrink is essentially optimal relying only on the data it comes within a factor log 2 n of the performance of piecewise polynomial and variableknot spline methods equipped with an oracle in contrast it is unknown how or if piecewise polynomial methods could be made to function this well when denied access to an oracle and forced to rely on data alone

a stable indirect adaptive control scheme for first order plants with no prior knowledge on the parameters an indirect adaptive control scheme for a first order linear time invariant plant with unknown parameters is presented and analyzed the scheme requires a priori knowledge of the sign or bounds of the plant parameters a discontinuous control law involving the use of probing is utilized to avoid any singularities caused by the lack of stabilizability of the estimated model the scheme guarantees signal boundedness and zero residual tracking error

global tunability of one dimensional siso systems it is shown by example that with a suitably defined certainty equivalence controller sigma sub r it is possible to make a closed loop parameterized system sigma tunable on its entire parameter space p even though p may contain points at which the parameterized sigma sub d upon which sigma sub r s definition is based is not stabilizable the implications of this discovery are briefly discussed

on the early history of the singular value decomposition this paper surveys the contributions of five mathematicians eugenio beltrami 1835 1899 camille jordan 1838 1921 james joseph sylvester 1814 1897 erhard schmidt 1876 1959 and hermann weyl 1885 1955 who were responsible for establishing the existence of the singular value decomposition and developing its theory

variable bandwidth kernel estimators of regression curves on montre qu en termes de l erreur en moyenne quadratique integree asymptotique les estimateurs du noyau avec un certain choix de largeur de bande local sont superieurs aux estimateurs ordinaires du noyau avec un choix de largeur de bande global si on utilise des largeurs de bandes optimales

nonparametric regression optimal local bandwidth choice summary kernel estimators of a regression function are investigated the bandwidths are locally chosen by a data driven method based on the minimization of a local cross validation criterion this method is shown to be asymptotically optimal with respect to local quadratic measures of errors monte carlo experiments are presented and finally the method is applied to some data of medical interest

multivariate adaptive regression splines a new method is presented for flexible regression modeling of high dimensional data the model takes the form of an expansion in product spline basis functions where the number of basis functions as well as the parameters associated with each one product degree and knot locations are automatically determined by the data this procedure is motivated by the recursive partitioning approach to regression and shares its attractive properties unlike recursive partitioning however this method produces continuous models with continuous derivatives it has more power and flexibility to model relationships that are nearly additive or involve interactions in at most a few variables in addition the model can be represented in a form that separately identifies the additive contributions and those associated with the different multivariable interactions

locally adaptive bandwidth choice for kernel regression estimators abstract kernel estimators with a global bandwidth are commonly used to estimate regression functions on the other hand it is obvious that the choice of a local bandwidth can lead to better results because a larger class of kernel estimators is available evidently this may in turn affect variability the optimal bandwidths depend essentially on the regression function itself and on the residual variance and it is desirable to estimate them from the data in this article a local bandwidth estimator is studied a comparison with its global bandwidth equivalent is performed both in theory and in simulations as the main result it is shown that the possible gain in mean integrated squared error of the resulting regression estimator must be paid for by a larger variability of the estimator this may lead to worse results if the sample size is small an algorithm has been devised that puts special weight on stability aspects our simulation study shows that improvements over a global bandwidth estimator

a constrained risk inequality with applications to nonparametric functional estimation a general constrained minimum risk inequality is derived given two densities f and f 0 we find a lower bound for the risk at the point given an upper bound for the risk at the point 0 the inequality sheds new light on superefficient estimators in the normal location problem and also on an adaptive estimation problem arising in nonparametric functional estimation

optimal data based kernel estimation of evolutionary spectra complex demodulation of evolutionary spectra is formulated as a two dimensional kernel smoother in the time frequency domain first a tapered fourier transform y sub v f t is calculated then the log spectral estimate is smoothed as the characteristic widths of the kernel smoother increase the bias from the temporal and frequency averaging increases while the variance decreases the demodulation parameters such as the order length and bandwidth of spectral taper and the kernel smoother are determined by minimizing the expected error for well resolved evolutionary spectra the optimal taper length is a small fraction of the optimal kernel halfwidth the optimal frequency bandwidth w for the spectral window scales as w sup 2 approximately lambda tau where tau is the characteristic time and lambda sub f is the characteristic frequency scalelength in contrast the optimal halfwidths for the second stage kernel smoother scales as h approximately 1 tau lambda sub f sup 1 p 2 where p is the order of the kernel smoother the ratio of the optimal frequency halfwidth to the optimal time halfwidth is determined

small confidence sets for the mean of a spherically symmetric distribution summary suppose that x has a k variate spherically symmetric distribution with mean vector 0 and identity covariance matrix wepresent two spherical confidence sets for 0 both centred at a positive part stein estimator t x in the first we obtain the radius by approximating the upper a point of the sampling disiribution of iits x 9112 by the first two non zero terms of its taylor series about the origin we can analyse some of the properties of this confidence set and see that it performs well in terms of coverage probability volume and conditional behaviour in the second method we find the radius by using a parametric bootstrap procedure here even greater improvement in terms of volume over the usual confidence set is possible at the expense of having a less explicit radius function a real data example is provided and extensions to the unknown covariance matrix and elliptically symmetric cases are discussed

nonlinear black box models in system identification mathematical foundations we discuss several aspects of the mathematical foundations of the nonlinear black box identification problem we shall see that the quality of the identification procedure is always a result of a certain trade off between the expressive power of the model we try to identify the larger the number of parameters used to describe the model the more flexible is the approximation and the stochastic error which is proportional to the number of parameters a consequence of this trade off is the simple fact that a good approximation technique can be the basis of a good identification algorithm from this point of view we consider different approximation methods and pay special attention to spatially adaptive approximants we introduce wavelet and neuron approximations and show that they are spatially adaptive then we apply the acquired approximation experience to estimation problems finally we consider some implications of these theoretical developments for the practically implemented versions of the spatially adaptive algorithms

nonparametric regression analysis of growth curves in recent years nonparametric curve estimates have been extensively explored in theoretical work there has however been a certain lack of convincing applications in particular involving comparisons with parametric techniques the present investigation deals with the analysis of human height growth where longitudinal measurements were collected for a sample of boys and a sample of girls evidence is presented that kernel estimates of acceleration and velocity of height and of height itself might offer advantages over a parametric fitting via functional models recently introduced for the specific problem considered both approaches are biased but the parametric one shows qualitative and quantitative distortion which both are not easily predictable data analytic problems involved with kernel estimation concern the choice of kernels the choice of the smoothing parameter and also whether the smoothing parameter should be chosen uniformly for all subjects or individually

better subset regression using the nonnegative garrote a new method called the nonnegative nn garrote is proposed for doing subset regression it both shrinks and zeroes coefficients in tests on real and simulated data it produces lower prediction error than ordinary subset selection it is also compared to ridge regression if the regression equations generated by a procedure do not change drastically with small changes in the data the procedure is called stable subset selection is unstable ridge is very stable and the nn garrote is intermediate simulation results illustrate the effects of instability on prediction error

approximation dans les espaces metriques et theorie de l estimation we investigate the relations between the speed of estimation and the metric structure of the parameter space especially in the case when its metric dimension is infinite given some distance d on generally hellinger distance in the case of n i i d variables we consider the minimax risk for n observations r n q mathop inf sup limits t n theta in theta text ie theta d q theta t n t n being any estimate of we shall look for functions r such that for positive constants c 1 q and c 2 q c 1 r q n r n q c2 r q n r n is the speed of estimation and we shall show under fairly general conditions including i i d variables and regular cases of markov chains and stationnary gaussian processes that r n is determined up to multiplicative constants by the metric structure of we shall also give a construction for some sort of universal estimates the risk of which is bounded by c 2 r q n in all cases where the preceding theory applies

the grenader estimator a nonasymptotic approach on examine certaines proprietes non asymtotiques de l estimateur de grenander d une densite decroissante f cet estimation est defini comme la pente du plus petit majorant concave de la fonction de repartition empirique on montre que son risque mesure avec perte l 1 est borne par une certaine fonctionnelle qui depend de f et du nombre d observations n le comportement asymptotique de la fonctionnelle quand n tend vers l infini est en accord avec les performances asymptotiques connues de l estimation de grenander

kernel and probit estimates in quantal bioassay abstract a kernel method for the estimation of quantal dose response curves is considered in contrast to parametric modeling this local smoothing method does not require any assumptions beyond smoothness of the dose response curve and in this sense is nonparametric in finite sample situations the kernel estimate of the dose response curve is not necessarily monotone and therefore if the additional assumption of monotonicity of the dose response curve is made a monotonized version is discussed bias variance asymptotic normality and uniform consistency including rates of convergence of kernel estimates are derived and applied to establish consistency and limiting distribution of kernel estimates of the ed here ed is the effective dose at level that is the dose where 100 of the subjects show a response the properties of the kernel estimated ed are compared with the corresponding properties of the maximum likelihood estimator assuming the probit model practical application of the k

local maximum likelihood estimation and inference local maximum likelihood estimation is a nonparametric counterpart of the widely used parametric maximum likelihood technique it extends the scope of the parametric maximum likelihood method to a much wider class of parametric spaces associated with this nonparametric estimation scheme is the issue of bandwidth selection and bias and variance assessment this paper provides a unified approach to selecting a bandwidth and constructing confidence intervals in local maximum likelihood estimation the approach is then applied to least squares nonparametric regression and to nonparametric logistic regression our experiences in these two settings show that the general idea outlined here is powerful and encouraging

data driven bandwidth selection in local polynomial fitting variable bandwidth and spatial adaptation when estimating a mean regression function and its derivatives locally weighted least squares regression has proven to be a very attractive technique the present paper focuses on the important issue of how to select the smoothing parameter or bandwidth in the case of estimating curves with a complicated structure a variable bandwidth is desirable furthermore the bandwidth should be indicated by the data themselves recent developments in nonparametric smoothing techniques inspired us to propose such a data driven bandwidth selection procedure which can be used to select both constant and variable bandwidths the idea is based on a residual squares criterion along with a good approximation of the bias and variance of the estimator the procedure can be applied to select bandwidths not only for estimating the regression curve but also for estimating its derivatives the resulting estimation procedure has the necessary flexibility for capturing complicated shapes of curves this is illustrated via a large variety of testing examples including examples with a large spatial variability the results are also compared with wavelet thresholding techniques and it seems that our results are at least comparable i e local polynomial regression using our data driven variable bandwidth has spatial adaptation properties that are similar to wavelets

flexible parsimonious smoothing and additive modeling a simple method is presented for fitting regression models that are nonlinear in the explanatory variables despite its simplicity or perhaps because of it the method has some powerful characteristics that cause it to be competitive with and often superior to more sophisticated techniques especially for small data sets in the presence of high noise

confidence intervals for nonparametric curve estimates numerous nonparametric regression methods exist that yield consistent estimators of function curves often one is also interested in constructing confidence intervals for the unknown function when a function estimate is based on a single global smoothing parameter the resulting confidence intervals may hold their desired confidence level 1 on average but because bias in nonparametric estimation is not uniform they do not hold the desired level uniformly at all design points most research in this area has focused on mean squared error properties of the estimator for example mise itself a global measure in addition measures like mise are one step removed from the practical issue of coverage probability recent work that focuses on coverage probability has considered only coverage in an average sense ignoring the important issue of uniformity of coverage across the design space to deal with this problem a new estimator is developed which uses a local cross validation criterion lcv to determi

uniform consistency of automatic and location adaptive delta sequence estimators the class of delta sequence estimators for a probability density includes the kernel histogram and orthogonal series types because each can be characterized as a collection of averages of some function that is indexed by a smoothing parameter there are two important extensions of this class the first allows a random smoothing parameter for example that specified by a cross validation method the second allows the smoothing parameter to be a function of location for example an estimator based on nearest neighbor distance in this paper a general method is presented which establishes uniform consistency for all of these estimators

statistical estimation and optimal recovery new formulas are given for the minimax linear risk in estimating a linear functional of an unknown object from indirect data contaminated with random gaussian noise the formulas cover a variety of loss functions and do not require the symmetry of the convex a priori class it is shown that affine minimax rules are within a few percent of minimax even among nonlinear rules for a variety of loss functions it is also shown that difficulty of estimation is measured by the modulus of continuity of the functional to be estimated the method of proof exposes a correspondence between minimax affine estimates in the statistical estimation problem and optimal algorithms in the theory of optimal recovery

interpolation of besov spaces we investigate besov spaces and their connection with dyadic spline approximation in lp q 0 p oo our main results are the determination of the interpolation spaces between a pair of besov spaces an atomic decomposition for functions in a besov space the characterization of the class of functions which have certain prescribed degree of approximation by dyadic splines

the kernel estimate of a regression function in likelihood based models abstract smoothing splines have a penalized likelihood motivation good and gaskins 1971 allowing direct application to nonparametric regression in likelihood based models the notion of a weighted likelihood for the nonparametric kernel estimation of a regression function is proposed generalizing the local likelihood theory of tibshirani and hastie 1987 let the data be of the form xi yi i 1 n where xi 0 1 d are lattice points and the yi are independent random variables from a family of distributions with parameter i g xi with g having continuous partial derivatives of order k 2 the goal is to arrive at a nonparametric estimate o of o g xo for a fixed point xo 0 1 d we consider the estimator o that maximizes the weighted likelihood function w n i 1 w xo xi b log f yi with f the density of yi w a symmetric kernel with compact support and b the bandwidth that controls the degree of smoothing sufficient conditions for consistency and asympt

control chart tests based on geometric moving averages a geometrical moving average gives the most recent observation the greatest weight and all previous observations weights decreasing in geometric progression from the most recent back to the first a graphical procedure for generating geometric moving averages is described in which the most recent observation is assigned a weight r the properties of control chart tests based on geometric moving averages are compared to tests based on ordinary moving averages

applications of hysteresis switching in parameter adaptive control the hysteresis switching algorithm of r h middleton et al ibid vol 33 no 1 p 50 8 jan 1988 is reexamined in a broader context to demonstrate its utility the algorithm is applied to various families of identifier based parameterized controllers of both the direct and indirect control types application to the direct control type results in a model reference adaptive controller capable of stabilizing without excitation any siso process which can be modeled by a minimum phase linear system whose transfer function has relative and mcmillan degrees not exceeding prescribing integers m and n respectively it is shown that such processes can also be adaptively stabilized with indirect adaptive controllers and hysteresis switching a simple numerical example involving a non minimum phase process model is used to illustrate how hysteresis switching might be applied to implicitly tuned parameterized controllers to realize an adaptive controller with capabilities which might prove very difficult if not impossible to achieve without hysteresis switching or some other form of discontinuous control

smoothing splines with varying smoothing parameter this paper considers the development of spatially adaptive smoothing splines for the estimation of a regression function with nonhomogeneous smoothness across the domain two challenging issues arising in this context are the evaluation of the equivalent kernel and the determination of a local penalty the penalty is a function of the design points in order to accommodate local behaviour of the regression function we show that the spatially adaptive smoothing spline estimator is approximately a kernel estimator and that the equivalent kernel is spatially dependent the equivalent kernels for traditional smoothing splines are a special case of this general solution with the aid of the green s function for a two point boundary value problem explicit forms of the asymptotic mean and variance are obtained for any interior point thus the optimal roughness penalty function is obtained by approximately minimizing the asymptotic integrated mean squared error simulation results and an application illustrate the performance of the proposed estimator copyright 2013 oxford university press

minimax risk over hyperrectangles and implications consider estimating the mean of a standard gaussian shift when that mean is known to lie in an orthosymmetric quadratically convex set in l 2 such sets include ellipsoids hyperrectangles and l p bodies with p 2 the minimax risk among linear estimates is within 25 of the minimax risk among all estimates the minimax risk among truncated series estimates is within a factor 4 44 of the minimax risk this implies that the difficulty of estimation a statistical quantity is measured fairly precisely by the n widths a geometric quantity if the set is not quadratically convex as in the case of l p bodies with p 2 things change appreciably minimax linear estimators may be out performed arbitrarily by nonlinear estimates the ordinary kolmogorov n widths still determine the difficulty of linear estimation but the difficulty of nonlinear estimation is tied to the inner bernstein n widths which can be far smaller essential use is made of a new heuristic that the difficulty of the hardest rectangular subproblem is equal to the difficulty of the full problem

bandwidth choice for nonparametric regression this paper is concerned with the problem of choosing a bandwidth parameter for nonparametric regression we analyze a tapered fourier series estimate and discuss the relationship of this estimate to a kernel estimate we first consider a method based on an unbiased estimate of mean square error and show that the bandwidth thus chosen is asymptotically optimal other methods are examined as well and are shown to be asymptotically equivalent a small simulation shows however that for small or moderate sample size the methods perform quite differently

estimation of heteroscedasticity in regression analysis consider the regression model y i g t i varepsilon i 1 leq i leq n with nonrandom design variables t i and measurements y i for the unknown regression function g cdot we assume that the data are heteroscedastic i e e varepsilon 2 i sigma 2 i not equiv operatorname const and investigate how to estimate sigma 2 i if sigma 2 i sigma 2 t i with a smooth function sigma 2 cdot initial estimators tilde sigma 2 i can be improved by kernel smoothers and the resulting class of estimators is shown to be uniformly consistent these estimates can be used to improve the estimation of the regression function g itself in parametric and nonparametric models further applications are suggested

kernel estimation of the instantaneous frequency considers kernel estimators of the instantaneous frequency of a slowly evolving sinusoid in white noise the expected estimation error consists of two terms the systematic bias error grows as the kernel halfwidth increases while the random error decreases for a nonmodulated signal g t the kernel halfwidth that minimizes the expected error is proportional to h spl sim spl sigma sup 2 n spl part sub t sup 2 g 2 sup 1 5 where spl sigma sup 2 is the noise variance and n is the number of measurements per unit time the author shows that estimating the instantaneous frequency corresponds to estimating the first derivative of a modulated signal a t exp i spl phi t for instantaneous frequency estimation the halfwidth which minimizes the expected error is larger h sub 1 3 spl sim spl sigma sup 2 a sup 2 n spl part sub t sup 3 e sup i spl phi t sup 2 sup 1 u7 since the optimal halfwidths depend on derivatives of the unknown function the authors initially estimate these derivatives prior to estimating the actual signal

variable bandwidth and local linear regression smoothers in this paper we introduce an appealing nonparametric method for estimating the mean regression function the proposed method combines the ideas of local linear smoothers and variable bandwidth hence it also inherits the advantages of both approaches we give expressions for the conditional mse and mise of the estimator minimization of the mise leads to an explicit formula for an optimal choice of the variable bandwidth moreover the merits of considering a variable bandwidth are discussed in addition we show that the estimator does not have boundary effects and hence does not require modifications at the boundary the performance of a corresponding plug in estimator is investigated simulations illustrate the proposed estimation method

translation invariant denoising using multiwavelets translation invariant ti single wavelet denoising was developed by coifman and donoho 1994 and they show that ti is better than non ti single wavelet denoising on the other other hand strela et al 1994 have found that non ti multiwavelet denoising gives better results than non ti single wavelets we extend coifman and donoho s ti single wavelet denoising scheme to multiwavelets experimental results show that ti multiwavelet denoising is better than the single case for soft thresholding

piecewise linear regularized solution paths we consider the generic regularized optimization problem argmin l y x j efron hastie johnstone and tibshirani ann statist 32 2004 407 499 have shown that for the lasso that is if l is squared error loss and j 1 is the l 1 norm of the optimal coefficient path is piecewise linear that is is piecewise constant we derive a general characterization of the properties of loss l penalty j pairs which give piecewise linear coefficient paths such pairs allow for efficient generation of the full regularized coefficient paths we investigate the nature of efficient path following algorithms which arise we use our results to suggest robust versions of the lasso for regression and classification and to develop new efficient algorithms for existing problems in the literature including mammen and van de geer s locally adaptive regression splines

outlier robust corner preserving methods for reconstructing noisy images the ability to remove a large amount of noise and the ability to preserve most structure are desirable properties of an image smoother unfortunately they usually seem to be at odds with each other one can only improve one property at the cost of the other by combining m smoothing and least squares trimming the tm smoother is introduced as a means to unify corner preserving properties and outlier robustness to identify edge and corner preserving properties a new theory based on differential geometry is developed further robustness concepts are transferred to image processing in two examples the tm smoother outperforms other corner preserving smoothers a software package containing both the tm and the m smoother can be downloaded from the internet

a unifying approach to nonparametric regression estimation abstract there exists a variety of nonparametric regression estimators with cubic smoothing splines k nearest neighbor k nn estimators and various types of kernel estimators among the most popular a class of kernel estimators with local bandwidth depending on the density of the design points is introduced where the degree of design adaptation may be expressed by a single parameter in 0 1 the adaptation to the design is such that the bandwidth is made larger to a degree depending on where the design is thin special values of this parameter correspond approximately to the ordinary fixed width kernel estimator the smoothing spline and the k nn estimator hence this method offers a synthesis of some classical methods the same method allows estimation of derivatives by using appropriate kernels the influence of this degree of design adaptation on the integrated mean squared error is investigated there is no uniformly optimal solution the optimal solution depends in a complex way on de

spectral decomposition of real circulant matrices abstract this paper presents spectral decompositions i e eigendecompositions and singular value decompositions of four types of real circulant matrices right and left circulants whose elements topple from right to left or from left to right respectively as well as skew right and skew left circulants whose elements change their sign when toppling are analyzed the inherent periodicity of circulant matrices means that they are closely related to fourier analysis and group theory this relationship is utilized in the spectral decompositions of this paper

adaptive bandwidth choice for kernel regression abstract a data based procedure is introduced for local bandwidth selection for kernel estimation of a regression function at a point the estimated bandwidth is shown to be consistent and asymptotically normal as an estimator of the asymptotic optimal value for minimum mean square estimation simulation studies indicate satisfactory behavior of the new bandwidth estimator in finite samples the findings are improvements over a global bandwidth estimator the same methodology works for local linear regression and extends easily to weighted local polynomial fits

adaptive smoothing of the log spectrum with multiple tapering a hybrid estimator of the log spectral density of a stationary time series is proposed first a multiple taper estimated is performed followed by kernel smoothing the log multiple taper estimate this procedure reduces the expected mean square error by spl pi sup 2 4 sup 4 5 over simply smoothing the log tapered periodogram a data adaptive implementation of a variable bandwidth kernel smoother is given

minimax estimation via wavelet shrinkage we attempt to recover an unknown function from noisy sampled data using orthonormal bases of compactly supported wavelets we develop a nonlinear method which works in the wavelet domain by simple nonlinear shrinkage of the empirical wavelet coefficients the shrinkage can be tuned to be nearly minimax over any member of a wide range of triebel and besov type smoothness constraints and asymptotically minimax over besov bodies with p q linear estimates cannot achieve even the minimax rates over triebel and besov classes with p 2 so the method can significantly outperform every linear method e g kernel smoothing spline sieve in a minimax sense variants of our method based on simple threshold nonlinear estimators are nearly minimax our method possesses the interpretation of spatial adaptivity it reconstructs using a kernel which may vary in shape and bandwidth from point to point depending on the data least favorable distributions for certain of the triebel and besov scales generate objects with sparse wavelet transforms many real objects have similarly sparse transforms which suggests that these minimax results are relevant for practical problems sequels to this paper which was first drafted in november 1990 discuss practical implementation spatial adaptation properties universal near minimaxity and applications to inverse problems

adapting to unknown smoothness via wavelet shrinkage abstract we attempt to recover a function of unknown smoothness from noisy sampled data we introduce a procedure sureshrink that suppresses noise by thresholding the empirical wavelet coefficients the thresholding is adaptive a threshold level is assigned to each dyadic resolution level by the principle of minimizing the stein unbiased estimate of risk sure for threshold estimates the computational effort of the overall procedure is order n log n as a function of the sample size n sureshrink is smoothness adaptive if the unknown function contains jumps then the reconstruction essentially does also if the unknown function has a smooth piece then the reconstruction is essentially as smooth as the mother wavelet will allow the procedure is in a sense optimally smoothness adaptive it is near minimax simultaneously over a whole interval of the besov scale the size of this interval depends on the choice of mother wavelet we know from a previous paper by the authors that traditional smoot

from multiple gaussian sequences to functional data and beyond a stein estimation approach summary we expand the notion of gaussian sequence models to n experiments and propose a stein estimation strategy which relies on pooling information across experiments an oracle inequality is established to assess conditional risks given the underlying effects based on which we can quantify the size of relative error and obtain a tuning free recovery strategy that is easy to compute produces model parsimony and extends to unknown variance we show that the simultaneous recovery is adaptive to an oracle strategy which also enjoys a robustness guarantee in a minimax sense a connection to functional data is established via le cam theory for fixed and random designs under general regularity settings we further extend the model projection to general bases with mild conditions on correlation structure and conclude with potential application to other statistical problems simulated and real data examples are provided to lend empirical support to the methodology proposed and to illustrate the potential for substantial computational savings

efficient denoising algorithms for large experimental datasets and their applications in fourier transform ion cyclotron resonance mass spectrometry modern scientific research produces datasets of increasing size and complexity that require dedicated numerical methods to be processed in many cases the analysis of spectroscopic data involves the denoising of raw data before any further processing current efficient denoising algorithms require the singular value decomposition of a matrix with a size that scales up as the square of the data length preventing their use on very large datasets taking advantage of recent progress on random projection and probabilistic algorithms we developed a simple and efficient method for the denoising of very large datasets based on the qr decomposition of a matrix randomly sampled from the data this approach allows a gain of nearly three orders of magnitude in processing time compared with classical singular value decomposition denoising this procedure called urqrd uncoiled random qr denoising strongly reduces the computer memory footprint and allows the denoising algorithm to be applied to virtually unlimited data size the efficiency of these numerical tools is demonstrated on experimental data from high resolution broadband fourier transform ion cyclotron resonance mass spectrometry which has applications in proteomics and metabolomics we show that robust denoising is achieved in 2d spectra whose interpretation is severely impaired by scintillation noise these denoising procedures can be adapted to many other data analysis domains where the size and or the processing time are crucial

uniform in bandwidth consistency of kernel type function estimators we introduce a general method to prove uniform in bandwidth consistency of kernel type function estimators examples include the kernel density estimator the nadaraya watson regression estimator and the conditional empirical process our results may be useful to establish uniform consistency of data driven bandwidth kernel type function estimators

pairwise curve synchronization for functional data summary data collected by scientists are increasingly in the form of trajectories or curves often these can be viewed as realizations of a composite process driven by both amplitude and time variation we consider the situation in which functional variation is dominated by time variation and develop a curve synchronization method that uses every trajectory in the sample as a reference to obtain pairwise warping functions in the first step these initial pairwise warping functions are then used to create improved estimators of the underlying individual warping functions in the second step a truncated averaging process is used to obtain robust estimation of individual warping functions the method compares well with other available time synchronization approaches and is illustrated with berkeley growth data and gene expression data for multiple sclerosis

wavelet shrinkage asymptopia much recent effort has sought asymptotically minimax methods for recovering infinite dimensional objects curves densities spectral densities images from noisy data a now rich and complex body of work develops nearly or exactly minimax estimators for an array of interesting problems unfortunately the results have rarely moved into practice for a variety of reasons among them being similarity to known methods computational intractability and lack of spatial adaptivity we discuss a method for curve estimation based on n noisy data translate the empirical wavelet coefficients towards the origin by an amount 2 log n n the proposal differs from those in current use is computationally practical and is spatially adaptive it thus avoids several of the previous objections further the method is nearly minimax both for a wide variety of loss functions pointwise error global error measured in l p norms pointwise and global error in estimation of derivatives and for a wide range of smoothness classes including standard holder and sobolev classes and bounded variation this is a much broader near optimality than anything previously proposed we draw loose parallels with near optimality in robustness and also with the broad near eigenfunction properties of wavelets themselves finally the theory underlying the method is interesting as it exploits a correspondence between statistical questions and questions of optimal recovery and information based complexity

nonparametric matching and efficient estimators of homothetically separable functions for vectors z and w and scalar v let r v z w be a function that can be non parametrically estimated consistently and asymptotically normally such as a distribution density or conditional mean regression function we provide consistent asymptotically normal nonparametric estimators for the functions g and h where r v z w h vg z w and some related models this framework encompasses homothetic and homothetically separable functions and transformed partly additive models r v z w h v g z w for unknown functions g and h such models reduce the curse of dimensionality provide a natural generalization of linear index models and are widely used in utility production and cost function applications we also provide an estimator of g that is oracle efficient achieving the same performance as an estimator based on local least squares when h is known

on convergence rates equivalency and sampling strategies in functional deconvolution models using the asymptotical minimax framework we examine convergence rates equivalency between a continuous functional deconvolution model and its real life discrete counterpart over a wide range of besov balls and for thel 2 risk for this purpose all possible models are divided into three groups for the models in the

nonparametric estimation of a regression function consider the regression model y i g x i e i i 1 2 n where x i s denote unordered design variables and g is an unknown function defined on the interval 0 1 assume e i are iid random variables with zero mean and finite variance priestley and chao 1972 and clark 1977 proposed estimators g 2n and g 3n respectively for g in this paper the asymptotic behavior of g 2n and g 3n is studied utilizing the properties of the new estimator g 1n it is shown that g 1n g 2n g 3n are asymptotically equivalent in various senses moreover consistency results are established and rates of uniform convergence obtained for example if e e 3 if g is lipschitz of order 1 and if n is any sequence of constants tending to as n then for all 0 a leqq b 1 n 1 mathord left vphantom 1 3 right kern nulldelimiterspace 3 mathord left vphantom n 1 mathord left vphantom 1 3 right kern nulldelimiterspace 3 beta n log n mathop sup limits a leqq x leqq b g 1n x g x xrightarrow w p 1 0 right kern nulldelimiterspace beta n log n mathop sup limits a leqq x leqq b g 1n x g x xrightarrow w p 1 0 as n finally when g is monotone a strong consistent isotonic estimator g n is considered

denoising scheme based on singular value decomposition for one dimensional spectra and its application in precision storage ring mass spectrometry this work concerns noise reduction for one dimensional spectra in the case that the signal is corrupted by an additive white noise the proposed method starts with mapping the noisy spectrum to a partial circulant matrix in virtue of singular value decomposition of the matrix components belonging to the signal are determined by inspecting the total variations of left singular vectors afterwards a smoothed spectrum is reconstructed from the low rank approximation of the matrix consisting of the signal components only the denoising effect of the proposed method is shown to be highly competitive among other existing nonparametric methods including moving average wavelet shrinkage and total variation furthermore its applicable scenarios in precision storage ring mass spectrometry are demonstrated to be rather diverse and appealing

optimal spatial adaptation to inhomogeneous smoothness an approach based on kernel estimates with variable bandwidth selectors a new v ria e 1 s e 0r estimation is proposed the application of this bandwidth selector leads to kernel estimates that achieve optimal rates of convergence over b 3 sses this implies that the procedure adapts to spatially inhomogeneous s noothness in particu lar the estimates share optimality properties with wavelet estimates based on thresholding of empirical wavelet coefficients 1 introduction in nonparametric curve estimation the statistical anal ysis may focus on the inference of the qualitative structure of the analyzed curve often interesting features of the curve are connected with spatially inhomogeneous smoothness in this case curve estimates that are spatially adaptlve ar e appropriaie a variety of such procedures have been proposed in the literature in breiman friedman olshen and stone 1983 piecewise constant least squares estimates are considered with a data adaptive choice of the pieces cart more generally friedman and silverman 1989 friedman 1991 and luo and wahba 1995 use variable knot splines mars knot points are added removed and allocated recursively using cross validation techniques these methods have shown good performance in simulations and real data examples however no asymptotic theory is available mammen and van de geer 1997 discuss penalized least squares curve estimation for spatially inhomogeneous curves they propose penalty terms which allow more spatial inhomogeneity than the usual l2 norms of deriva tives of the curve the estimates turn out to be variable knot splines see also mammen 1991 results on rates of convergence and a pointwise asymptotic distribution theory are given miiller and stadtmiiller 1987 staniswalis 1989 and brockmann gasser and hermann 1993 propose kernel estimation with locally variable band width selectors the calculation of local bandwidths is based on pilot estima tion of local smoothness characteristics an asymptotic analysis is available

automatic bayesian curve fitting summary a method of estimating a variety of curves by a sequence of piecewise polynomials is proposed motivated by a bayesian model and an appropriate summary of the resulting posterior distribution a joint distribution is set up over both the number and the position of the knots defining the piecewise polynomials throughout we use reversible jump markov chain monte carlo methods to compute the posteriors the methodology has been successful in giving good estimates for smooth functions i e continuous and differentiable as well as functions which are not differentiable and perhaps not even continuous at a finite number of points the methodology is extended to deal with generalized additive models

speed of estimation in positron emission tomography and related inverse problems we study a continuous idealization of the image reconstruction problem in positron emission tomography considered as an example of bivariate density estimation based on indirect observations given a large sample of indirect observations we consider the size of the equivalent sample of observations whose original exact positions would allow equally accurate estimation of the image of interest both for indirect and for direct observations we establish exact minimax rates of convergence of estimation for all possible estimators over suitable smoothness classes of functions the approach of the paper is applicable to a wide class of linear inverse problems

local bandwidth selection for kernel estimates abstract a kernel estimate of a curve that uses an adaptive procedure for local selection of the bandwidth is considered here a two step procedure is proposed for estimating the local bandwidth that minimizes the mean squared error mse of a kernel estimator for nonparametric regression first a consistent estimate of the exact mse is constructed then the bandwidth that minimizes the estimate of the mse is calculated sufficient conditions under which this bandwidth is asymptotically optimal and normally distributed are given the local bandwidth selection procedure was implemented on some simulated data and compared to a global bandwidth selection procedure proposed by rice 1984b a 68 91 reduction in the average mse of a kernel estimator was realized with the local bandwidth selection procedure such a scheme was also studied by muller 1985 and termed a direct pilot estimator approach muller derived sufficient conditions similar to those presented here under which the direct pilot estimator

on bandwidth variation in kernel estimates a square root law we consider kernel estimation of a smooth density f at a point but depart from the usual approach in admitting an adaptive dependence of the sharpness of the kernels on the underlying density proportionally varying the bandwidths like f 1 2 at the contributing readings lowers the bias to a vanishing fraction of the usual value and makes for performance seen in well known estimators that force moment conditions on the kernel and so sacrifice positivity of the curve estimate issues of equivariance and variance stabilitization are treated

optimal pointwise adaptive methods in nonparametric estimation the problem of optimal adaptive estimation of a function at a given point from noisy data is considered two procedures are proved to be asymptotically optimal for different settings first we study the problem of bandwidth selection for nonparametric pointwise kernel estimation with a given kernel we propose a bandwidth selection procedure and prove its optimality in the asymptotic sense moreover this optimality is stated not only among kernel estimators with a variable bandwidth the resulting estimator is asymptotically optimal among all feasible estimators the important feature of this procedure is that it is fully adaptive and it works for a very wide class of functions obeying a mild regularity restriction with it the attainable accuracy of estimation depends on the function itself and is expressed in terms of the ideal adaptive bandwidth corresponding to this function and a given kernel the second procedure can be considered as a specialization of the first one under the qualitative assumption that the function to be estimated belongs to some holder class l with unknown parameters l this assumption allows us to choose a family of kernels in an optimal way and the resulting procedure appears to be asymptotically optimal in the adaptive sense in any range of adaptation with 2

a cyclic switching strategy for parameter adaptive control this paper introduces a new strategy called cyclic switching to deal with the well known certainty equivalence control synthesis problem which arises in the design of identifier based adaptive controllers because of the existence of points in parameter space where the design model spl sigma sub d upon which certainty equivalence synthesis is based loses stabilizability unlike most previously suggested methods for handling this problem the technique proposed here can be employed with or without process excitation for the technique to work it is not necessary for there to be a mechanism for moving tuned parameters away from values at which spl sigma sub d loses stabilizability and no such mechanism is used

towards a unified theory of parameter adaptive control ii certainty equivalence and implicit tuning for pt i see ibid vol 35 p 1002 12 1990 it is shown that for a large class of linear multivariable process models a properly designed certainty equivalence controller results in a tunable closed loop parameterized system the importance of this result to both the analysis and synthesis of parameter adaptive control systems is discussed it is demonstrated by means of examples how the connection between certainty equivalence and tunability together with the concept of tunability itself can be used to markedly simplify the stability analysis of adaptive control systems a new family of indirect parameterized controllers is introduced which has capabilities comparable to those of the well known direct parameterized controllers which have served as basic building blocks for adaptive control systems of all types the concept of implicit tuning is formalized and its potential importance to adaptive control is briefly discussed it is explained how the idea of tunability can be used to analyze an implicitly tuned system

asymptotic minimax risk for sup norm loss solution via optimal recovery we study the problem of estimating an unknown function on the unit interval or itsk th derivative with supremum norm loss when the function is observed in gaussian white noise and the unknown function is known only to obey lipschitz smoothness k 0 we discuss an optimization problem associated with the theory ofoptimal recovery although optimal recovery is concerned with deterministic noise chosen by a clever opponent the solution of this problem furnishes the kernel of the minimax linear estimate for gaussian white noise moreover this minimax linear estimator is asymptotically minimax among all estimates we sketch also applications to higher dimensions and to indirect measurement e g deconvolution problems

multivariate functional coefficient regression models for nonlinear vector time series data summary vector time series data are widely found in practice in this paper we propose a multivariate functional coefficient regression model with heteroscedasticity for modeling such data the local linear smoother is employed to estimate the unknown coefficient matrices asymptotic normality 10 of the proposed estimators is established and bandwidth selection is considered to deal with cointegration widely observed in financial markets we propose an error corrected multivariate functional coefficient model simulations show that the proposed estimation procedures capture nonlinear structures of coefficients well analysis of united states interest rates illustrates the proposed methodology 15

global convergence of the recursive kernel regression estimates with applications in classification and nonlinear system estimation an improved exponential bound on the l sub 1 error for the recursive kernel regression estimates is derived it is shown using the martingale device that weak strong and complete l sub 1 consistencies are equivalent consequently the conditions on a certain smoothing sequence are necessary and sufficient for strong l sub 1 consistency of the recursive kernel regression estimate the rates of global convergence are also given obtained results are applied to recursive classification rules and to nonlinear time series estimation

minimum bias multiple taper spectral estimation two families of orthonormal tapers are proposed for multitaper spectral analysis minimum bias tapers and sinusoidal tapers spl upsisup k where spl upsisub nsup k spl radic 2 n 1 sin spl pi kn n 1 and n is the number of points the resulting sinusoidal multitaper spectral estimate is s spl circ f 1 2k n 1 spl sigmasub j 1sup k y f j 2n 2 y f j 2n 2 sup 2 where y f is the fourier transform of the stationary time series s f is the spectral density and k is the number of tapers for fixed j the sinusoidal tapers converge to the minimum bias tapers like 1 n since the sinusoidal tapers have analytic expressions no numerical eigenvalue decomposition is necessary both the minimum bias and sinusoidal tapers have no additional parameter for the spectral bandwidth the bandwidth of the jth taper is simply 1 n centered about the frequencies spl plusmn j 2n 2 thus the bandwidth of the multitaper spectral estimate can be adjusted locally by simply adding or deleting tapers the band limited spectral concentration spl intsub wsup w v f sup 2 df of both the minimum bias and sinusoidal tapers is very close to the optimal concentration achieved by the slepian 1978 tapers in contrast the slepian tapers can have the local bias spl intsub 1 2 sup 1 2 f sup 2 v f sup 2 df much larger than of the minimum bias tapers and the sinusoidal tapers

a theory for multiresolution signal decomposition the wavelet representation multiresolution representations are effective for analyzing the information content of images the properties of the operator which approximates a signal at a given resolution were studied it is shown that the difference of information between the approximation of a signal at the resolutions 2 sup j 1 and 2 sup j where j is an integer can be extracted by decomposing this signal on a wavelet orthonormal basis of l sup 2 r sup n the vector space of measurable square integrable n dimensional functions in l sup 2 r a wavelet orthonormal basis is a family of functions which is built by dilating and translating a unique function psi x this decomposition defines an orthogonal multiresolution representation called a wavelet representation it is computed with a pyramidal algorithm based on convolutions with quadrature mirror filters wavelet representation lies between the spatial and fourier domains for images the wavelet representation differentiates several spatial orientations the application of this representation to data compression in image coding texture discrimination and fractal analysis is discussed