improved estimators in nonparametric regression problems abstract linear estimators of multivariate means are considered generalizations of some well known theorems about admissibility of linear estimators are given the results then are applied to show that commonly used kernel type estimators in nonparametric regression problems can be constructively improved in a simple way an asymptotic result is described that gives a quantitative measure of the maximum improvement to be gained in certain situations a theoretical bound shows that gains are achievable in the relative risk of up to 58 6 rectangular kernel or 29 2 epanechnikov kernel some examples of smaller sample size are also investigated and these show relative risk gains ranging up to 18 in realistic settings

an essay on statistical decision theory 2000 an essay on statistical decision theory journal of the american statistical association vol 95 no 452 pp 1277 1281

p cyclic matrices and the symmetric successive overrelaxation method abstract in this paper the new functional equation 1 2 p 1 p 2 2 2 p p which connects the eigenvalues of a particular weakly cyclic of index p jacobi matrix b to the eigenvalues of its associated symmetric successive overrelaxation ssor matrix s is derived this functional equation is then applied to the problem of determining bounds for the intervals of convergence and divergence of the ssor iterative method for classes of h matrices

a note on the ssor convergence domain due to neumaier and varga in a recent paper a neumaier and r s varga show that if a is an n ncomplex h matrix with 12 ja 1 then sa 1 for all 0 gw 2 1 2 ja 1 where sa and ja are respectively the symmetric successive overrelaxation ssor iteration matrix and the jacobi iteration matrix associated with a we show that sagw 1 this allows us to also sharpen further results on the ssor method for h matrices due to varga niethammer and cai

minimax estimation of a constrained poisson vector and minimax risk p t we give analytical and numerical results for small intervals when p 1 usually however approximations are needed if t is rectangulary convex at 0 there exist linear estimators with risk at most 1 26p t for general t p t 2 p2 p a fq where a q is the principal eigenvalue of the laplace operator on the polydisc transform q ql t a domain in twice p dimensional space the bound is asymptotically sharp p mt p a l m o m explicit forms are given for t a simplex or a hyperrectangle we explore the curious parallel of the results for t with those for a gaussian vector of double the dimension lying in fl

symmetric block sor methods for rank deficient least squares problems in this article we develop symmetric block successive overrelaxation s block sor methods for finding the solution of the rank deficient least squares problems we propose an s2 block sor and an s3 block sor method for solving such problems and the convergence of these two methods is studied the comparisons between the s2 block and the s3 block methods are presented with some numerical examples

superefficiency in nonparametric function estimation fixed parameter asymptotic statements are often used in the context of nonparametric curve estimation problems e g nonparametric density or regression estimation in this context several forms of superefficiency can occur in contrast to what can happen in regular parametric problems here every parameter point e g unknown density or regression function can be a point of superefficiency we begin with an example which shows how fixed parameter asymptotic statements have often appeared in the study of adaptive kernel estimators and how superefficiency can occur in this context we then carry out a more systematic study of such fixed parameter statements it is shown in four general settings how the degree of superefficiency attainable depends on the structural assumptions in each case

bounds on the bayes and minimax risk for signal parameter estimation in estimating the parameter from a parametrized signal problem with 0l ll observed through gaussian white noise four useful and computable lower bounds for the bayes risk are developed for problems with different l and different signal to noise ratios some bounds are superior to others the lower bound obtained from taking the maximum of the four serves not only as a good lower bound for the bayes risk but also as a good lower bound for the minimax risks threshold behavior of the bayes risk is also evident as is shown in the lower bound

estimating a bounded normal mean the problem of estimating a normal mean has received much attention in recent years if one assumes however that the true mean lies in a bounded interval the problem changes drastically in this paper we show that if the interval is small approximately two standard deviations wide then the bayes rule against a two point prior is the unique minimax estimator under squared error loss for somewhat wider intervals we also derive sufficient conditions for minimaxity of the bayes rule against a three point prior

renormalizing upper and lower bounds for integrated risk in the white noise model where wt is brownian motion many results which might be difficult to prove in the density estimation or nonparametric regression context take on a more transparent form in this white noise model a sample size of n in the density estimation and nonparametric regression problems corresponds to o 0 o xn in 1 when ois suitably chosen in particular the tools of rescaling developed in low 1992 and donoho and low 1992 and the hardest one dimensional subfamily arguments of donoho and liu 1987 1991 have yielded a fairly complete picture of how to estimate both bounded and unbounded linear functionals on the basis of observations generated by 1 a separate literature is developing to show how to replace density estimation and regression problems by the corresponding white noise problems see for example low 1992 brown and low 1990 and donoho and low 1990 in this paper we focus attention on estimating the entire function f on the basis of the observation scheme given by 1 using integrated squared error as a measure of loss in particular we shall let r f o denote the minimax risk under this loss function that is

optimal adaptive estimation of a quadratic functional minimax mean squared error estimates of quadratic functionals of smooth functions have been constructed for a variety of smoothness classes in contrast to many nonparametric function estimation problems there are both regular and irregular cases in the regular cases the minimax mean squared error converges at a rate proportional to the inverse of the sample size whereas in the irregular case much slower rates are the rule we investigate the problem of adaptive estimation of a quadratic functional of a smooth function when the degree of smoothness of the underlying function is not known it is shown that estimators cannot achieve the minimax rates of convergence simultaneously over two parameter spaces when at least one of these spaces corresponds to the irregular case a lower bound for the mean squared error is given which shows that any adaptive estimator which is rate optimal for the regular case must lose a logarithmic factor in the irregular case on the other hand we give a rather simple adaptive estimator which is sharp for the regular case and attains this lower bound in the irregular case moreover we explicitly describe a subset of functions where our adaptive estimator loses the logarithmic factor and show that this subset is relatively small

some recent results on the modified sor theory abstract for the solution of the linear system ax i t x c 1 where t is a weakly cyclic of index p 2 matrix of a special structure the block sor method with different relaxation factors associated with the row blocks of a is considered first a well known relationship connecting the eigenvalue spectra of the block jacobi matrix t of a and its associated modified sor msor matrix is proved for all p 3 via an approach due to varga niethammer and cai next it is shown that the matrix analogue of the eigenvalue relationship holds at least for p 3 this together with the facts that the matrix analogue holds true for p 2 and also for any p 3 provided all relaxation factors coincide suggests a more general validity of the aforementioned matrix identity then based on the matrix relationship an equivalence is established between the msor method and a stationary p parameter p step iterative one for the solution of 1 so one can study convergence properties and determine optimal parameters of both methods by studying the simpler of the two that is the p step one finally some applications of the theory developed when p 2 are presented and a brief discussion concerning comparisons of the optimum msor and aor methods in a specific case is given

a data driven block thresholding approach to wavelet estimation a data driven block thresholding procedure for wavelet regression is proposed and its theoretical and numerical properties are investigated the procedure empirically chooses the block size and threshold level at each resolution level by minimizing stein s unbiased risk estimate the estimator is sharp adaptive over a class of besov bodies and achieves simultaneously within a small constant factor of the minimax risk over a wide collection of besov bodies including both the dense and sparse cases the procedure is easy to implement numerical results show that it has superior finite sample performance in comparison to the other leading wavelet thresholding estimators

minimax risk over hyperrectangles and implications consider estimating the mean of a standard gaussian shift when that mean is known to lie in an orthosymmetric quadratically convex set in l 2 such sets include ellipsoids hyperrectangles and l p bodies with p 2 the minimax risk among linear estimates is within 25 of the minimax risk among all estimates the minimax risk among truncated series estimates is within a factor 4 44 of the minimax risk this implies that the difficulty of estimation a statistical quantity is measured fairly precisely by the n widths a geometric quantity if the set is not quadratically convex as in the case of l p bodies with p 2 things change appreciably minimax linear estimators may be out performed arbitrarily by nonlinear estimates the ordinary kolmogorov n widths still determine the difficulty of linear estimation but the difficulty of nonlinear estimation is tied to the inner bernstein n widths which can be far smaller essential use is made of a new heuristic that the difficulty of the hardest rectangular subproblem is equal to the difficulty of the full problem

exact convergence and divergence domains for the symmetric successive overrelaxation iterative ssor method applied to h matrices abstract in this paper exact convergence and divergence domains for the ssor iterative method as applied to the class of h matrices are obtained the theory of regular splittings and the recent results of varga niethammer and cai are used as tools in establishing these convergence and divergence domains

variance estimation in nonparametric regression via the difference sequence method consider a gaussian nonparametric regression problem having both an unknown mean function and unknown variance function this article presents a class of difference based kernel estimators for the variance function optimal convergence rates that are uniform over broad functional classes and bandwidths are fully characterized and asymptotic normality is also established we also show that for suitable asymptotic formulations our estimators achieve the minimax rate

minimax risk over l p balls for l p error consider estimating the mean vector from datan n 2 i withl q norm loss q 1 when is known to lie in ann dimensionall p ball p 0 for largen the ratio of minimaxlinear risk to minimax risk can bearbitrarily large ifp q obvious exceptions aside the limiting ratio equals 1 only ifp q 2 our arguments are mostly indirect involving a reduction to a univariate bayes minimax problem whenp q simple non linear co ordinatewise threshold rules are asymptotically minimax at small signal to noise ratios and within a bounded factor of asymptotic minimaxity in general we also give asymptotic evaluations of the minimax linear risk our results are basic to a theory of estimation in besov spaces using wavelet bases to appear elsewhere

minimax estimation via wavelet shrinkage we attempt to recover an unknown function from noisy sampled data using orthonormal bases of compactly supported wavelets we develop a nonlinear method which works in the wavelet domain by simple nonlinear shrinkage of the empirical wavelet coefficients the shrinkage can be tuned to be nearly minimax over any member of a wide range of triebel and besov type smoothness constraints and asymptotically minimax over besov bodies with p q linear estimates cannot achieve even the minimax rates over triebel and besov classes with p 2 so the method can significantly outperform every linear method e g kernel smoothing spline sieve in a minimax sense variants of our method based on simple threshold nonlinear estimators are nearly minimax our method possesses the interpretation of spatial adaptivity it reconstructs using a kernel which may vary in shape and bandwidth from point to point depending on the data least favorable distributions for certain of the triebel and besov scales generate objects with sparse wavelet transforms many real objects have similarly sparse transforms which suggests that these minimax results are relevant for practical problems sequels to this paper which was first drafted in november 1990 discuss practical implementation spatial adaptation properties universal near minimaxity and applications to inverse problems

recovering edges in ill posed inverse problems optimality of curvelet frames we consider a model problem of recovering a function f x 1 x 2 from noisy radon data the function f to be recovered is assumed smooth apart from a discontinuity along a c 2 curve that is an edge we use the continuum white noise model with noise level varepsilon traditional linear methods for solving such inverse problems behave poorly in the presence of edges qualitatively the reconstructions are blurred near the edges quantitatively they give in our model mean squared errors mses that tend to zero with noise level varepsilon only as o varepsilon 1 2 as varepsilon to 0 a recent innovation nonlinear shrinkage in the wavelet domain visually improves edge sharpness and improves mse convergence to o varepsilon 2 3 however as we show here this rate is not optimal in fact essentially optimal performance is obtained by deploying the recently introduced tight frames of curvelets in this setting curvelets are smooth highly anisotropic elements ideally suited for detecting and synthesizing curved edges to deploy them in the radon setting we construct a curvelet based biorthogonal decomposition of the radon operator and build curvelet shrinkage estimators based on thresholding of the noisy curvelet coefficients in effect the estimator detects edges at certain locations and orientations in the radon domain and automatically synthesizes edges at corresponding locations and directions in the original domain we prove that the curvelet shrinkage can be tuned so that the estimator will attain within logarithmic factors the mse o varepsilon 4 5 as noise level varepsilon to 0 this rate of convergence holds uniformly over a class of functions which are c 2 except for discontinuities along c 2 curves and except for log terms is the minimax rate for that class our approach is an instance of a general strategy which should apply in other inverse problems we sketch a deconvolution example

from multiple gaussian sequences to functional data and beyond a stein estimation approach summary we expand the notion of gaussian sequence models to n experiments and propose a stein estimation strategy which relies on pooling information across experiments an oracle inequality is established to assess conditional risks given the underlying effects based on which we can quantify the size of relative error and obtain a tuning free recovery strategy that is easy to compute produces model parsimony and extends to unknown variance we show that the simultaneous recovery is adaptive to an oracle strategy which also enjoys a robustness guarantee in a minimax sense a connection to functional data is established via le cam theory for fixed and random designs under general regularity settings we further extend the model projection to general bases with mild conditions on correlation structure and conclude with potential application to other statistical problems simulated and real data examples are provided to lend empirical support to the methodology proposed and to illustrate the potential for substantial computational savings

wavelet shrinkage asymptopia much recent effort has sought asymptotically minimax methods for recovering infinite dimensional objects curves densities spectral densities images from noisy data a now rich and complex body of work develops nearly or exactly minimax estimators for an array of interesting problems unfortunately the results have rarely moved into practice for a variety of reasons among them being similarity to known methods computational intractability and lack of spatial adaptivity we discuss a method for curve estimation based on n noisy data translate the empirical wavelet coefficients towards the origin by an amount 2 log n n the proposal differs from those in current use is computationally practical and is spatially adaptive it thus avoids several of the previous objections further the method is nearly minimax both for a wide variety of loss functions pointwise error global error measured in l p norms pointwise and global error in estimation of derivatives and for a wide range of smoothness classes including standard holder and sobolev classes and bounded variation this is a much broader near optimality than anything previously proposed we draw loose parallels with near optimality in robustness and also with the broad near eigenfunction properties of wavelets themselves finally the theory underlying the method is interesting as it exploits a correspondence between statistical questions and questions of optimal recovery and information based complexity

optimal bounds for aggregation of affine estimators we study the problem of aggregation of estimators when the estimators are not independent of the data used for aggregation and no sample splitting is allowed if the estimators are deterministic vectors it is well known that the minimax rate of aggregation is of order log m where m is the number of estimators to aggregate it is proved that for affine estimators the minimax rate of aggregation is unchanged it is possible to handle the linear dependence between the affine estimators and the data used for aggregation at no extra cost the minimax rate is not impacted either by the variance of the affine estimators or any other measure of their statistical complexity the minimax rate is attained with a penalized procedure over the convex hull of the estimators for a penalty that is inspired from the q aggregation procedure the results follow from the interplay between the penalty strong convexity and concentration

on bifurcating time periodic flow of a navier stokes liquid past a cylinder we provide general sufficient conditions for the existence and uniqueness of branching out of a time periodic family of solutions from steady state solutions to the two dimensional navier stokes equations in the exterior of a cylinder by separating the time independent averaged component of the velocity field from its oscillatory one we show that the problem can be formulated as a coupled elliptic parabolic nonlinear system in appropriate and distinct function spaces with the property that the relevant linearized operators become fredholm of index 0 in this functional setting the notorious difficulty of 0 being in the essential spectrum entirely disappears and in fact it is even meaningless our approach is different and we believe more natural and simpler than those proposed by previous authors discussing similar questions moreover the latter all fail when applied to the problem studied here

minimax estimation of the mean of a normal distribution when the parameter space is restricted if x is a n theta 1 random variable let rho m be the minimax risk for estimation with quadratic loss subject to theta leq m then rho m 1 pi 2 m 2 o m 2 we exhibit estimates which are asymptotically minimax to this order as well as approximations to the least favorable prior distributions the approximate least favorable distributions correct to order m 2 have density m 1 cos 2 big frac pi 2m s big s leq m rather than the naively expected uniform density on lbrack m m rbrack we also show how our results extend to estimation of a vector mean and give some explicit solutions

multiscale likelihood analysis and complexity penalized estimation we describe here a framework for a certain class of multiscale likelihood factorizations wherein in analogy to a wavelet decomposition of an l 2 function a given likelihood function has an alternative representation as a product of conditional densities reflecting information in both the data and the parameter vector localized in position and scale the framework is developed as a set of sufficient conditions for the existence of such factorizations formulated in analogy to those underlying a standard multiresolution analysis for wavelets and hence can be viewed as a multiresolution analysis for likelihoods we then consider the use of these factorizations in the task of nonparametric complexity penalized likelihood estimation we study the risk properties of certain thresholding and partitioning estimators and demonstrate their adaptivity and near optimality in a minimax sense over a broad range of function spaces based on squared hellinger distance as a loss function in particular our results provide an illustration of how properties of classical wavelet based estimators can be obtained in a single unified framework that includes models for continuous count and categorical data types

tensor product space anova models to deal with the curse of dimensionality in high dimensional nonparametric problems we consider using tensor product space anova models which extend the popular additive models and are able to capture interactions of any order the multivariate function is given an anova decomposition that is it is expressed as a constant plus the sum of functions of one variable main effects plus the sum of functions of two variables two factor interactions and so on we assume the interactions to be in tensor product spaces we show in both regression and white noise settings the optimal rate of convergence for the tps anova model is within a log factor of the one dimensional optimal rate and that the penalized likelihood estimator in tps anova achieves this rate of convergence the quick optimal rate of the tps anova model makes it very preferable in high dimensional function estimation many properties of the tensor product space of sobolev hilbert spaces are also given

on minimax estimation of a sparse normal mean vector mallows has conjectured that among distributions which are gaussian but for occasional contamination by additive noise the one having least fisher information has two sided geometric contamination a very similar problem arises in estimation of a non negative vector parameter in gaussian white noise when it is known also that most i e 1i components are zero we provide a partial asymptotic expansion of the minimax risk as 0 while the conjecture seems unlikely to be exactly true for flnite we verify it asymptotically up to the accuracy of the expansion numerical work suggests the expansion is accurate for as large as 05 the best 1 estimation rule is flrst but not second order minimax the results bear on an earlier study of maximum entropy estimation and various questions in robustness and function estimation using wavelet bases

geometrizing rates of convergence ii consider estimating a functional t f of an unknown distribution f in mathbf f from data x 1 cdots x n i i d f let omega varepsilon denote the modulus of continuity of the functional t over mathbf f computed with respect to hellinger distance for well behaved loss functions l t we show that inf t n sup mathbf f e fl t n t f is equivalent to l omega n 1 2 to within constants whenever t is linear and mathbf f is convex the same conclusion holds in three nonlinear cases estimating the rate of decay of a density estimating the mode and robust nonparametric regression we study the difficulty of testing between the composite infinite dimensional hypotheses h 0 t f leq t and h 1 t f geq t delta our results hold in the cases studied because the difficulty of the full infinite dimensional composite testing problem is comparable to the difficulty of the hardest simple two point testing subproblem