choice of kernel order in density estimation the selection of the order i e number of vanishing moments of the kernel in a kernel density estimator is considered from two points of view first theoretical properties are investigated by a mean integrated squared error analysis of the problem second and perhaps more importantly cross validation is proposed as a practical method of choice and theoretical backing for this is provided through an asymptotic optimality result

extrapolation algorithms for discrete signals with application in spectral estimation in this paper we present some new algorithms useful for extrapolation and spectral estimation of band limited sequences in one and two dimensions first we show that many of the existing extrapolation algorithms for noiseless observations are unified under the criterion of minimum norm least squares mnls extrapolation for example the iterative algorithms proposed in 2 and 8 10 are shown to be special cases of a one step gradient algorithm which has linear convergence convergence and other numerical properties are improved by going to a conjugate gradient algorithm for noisy observations these algorithms could be extended by considering a mean square extrapolation criterion which gives rise to a mean square extrapolation filter and also to a recursive extrapolation filter examples and application of these methods are given extension of these algorithms is made for problems where the signal is known to be periodic a new set of functions called the periodic discrete prolate spheroidal sequences p dpss analogous to dpss 21 22 are introduced and their properties are studied finally several of these algorithms are generalized to two dimensions and the relevant equations are given

restoration of a discrete time signal segment by interpolation based on the left sided and right sided autoregressive parameters this paper presents an algorithm for the interpolation of a missing signal segment on the assumption that the signal can be modeled as an autoregressive ar process unlike previous algorithms the presented algorithm does not model the signal of the missing segment and the neighboring signal portions by a single ar parameter vector instead two separate vectors are used so that stationarity need no longer be assumed to extend beyond both sides of the missing segment the relaxation of this stationarity assumption is essential when the duration of the missing segment is on the order of the short time stationarity duration of the signal the algorithm provides the optimal solution to the problem of interpolating a missing segment based on the left sided and right sided ar parameter vectors the solution is optimal in the sense of a least squares residual the algorithm is applied to speech and music signals and is compared with other restoration techniques

information theoretic lower bounds on the oracle complexity of stochastic convex optimization relative to the large literature on upper bounds on complexity of convex optimization lesser attention has been paid to the fundamental hardn4516420ess of these problems given the extensive use of convex optimization in machine learning and statistics gaining an understanding of these complexity theoretic issues is important in this paper we study the complexity of stochastic convex optimization in an oracle model of computation we introduce a new notion of discrepancy between functions and use it to reduce problems of stochastic convex optimization to statistical parameter estimation which can be lower bounded using information theoretic methods using this approach we improve upon known results and obtain tight minimax complexity estimates for various function classes

on estimating a density using hellinger distance and some other strange facts on s interesse ici aux possibles vitesses d estimation d une densite a support compact dans m sous des hypotheses de regularite lorsque la perte est mesuree par le carre de la distance de hellinger on regardera aussi le cas connu des normes mathbb l q pour 1 q 2 et le risque est le risque minimax sur la famille on donne une methode generale permettant de traiter les problemes dans le cadre de la theorie de l approximation sous des conditions concernant l entropie metrique et l e capacite des familles a estimer les rapports entre regularite et entropie metrique etant bien connus nous pourrons aussi traiter les cas classiques et d autres qui le sont moins sous des conditions de bornes inferieures les vitesses sont celles observees pour la norme mathbb l q mais elles different dans le cas general on montre aussi que les restrictions sur la compacite du support ou la regularite sont indispensables et que leur absence mene a l impossibilite d obtenir une estimation raisonnable en ce sens que n importe quelle suite d estimateurs sera arbitrairement mauvaise en un point au moins un resultat analogue est vrai sous des conditions de regularite

energy concentration in band limited extrapolation the authors present a novel bandlimited extrapolation method that incorporates qualitative prior knowledge of the distribution of signal energy in both the time domain spatial domain and the frequency domain the extrapolation is easily computed by simple noniterative algorithm the prior information is incorporated by means of a weighted energy measure and the resulting solution is derived by straightforward hilbert space optimization the algorithm accommodates complex valued sequences nonuniform sampling and arbitrary spectral support and is applicable in one or several dimensions many existing extrapolation algorithms are noted as special cases examples of signal extrapolation and spectrum estimation using the new algorithm are given in one and two dimensions algorithm modification for noisy data is investigated and error bounds are presented

on the dirichlet problem of mixed type for lower hybrid waves in axisymmetric cold plasmas for a class of linear second order partial differential equations of mixed elliptic hyperbolic type which includes a well known model for analyzing possible heating in axisymmetric cold plasmas we give results on the weak well posedness of the dirichlet problem and show that such solutions are characterized by a variational principle the weak solutions are shown to be saddle points of natural functionals suggested by the divergence form of the pdes moreover the natural domains of the functionals are the weighted sobolev spaces to which the solutions belong in addition all critical levels will be characterized in terms of global extrema of the functionals restricted to suitable infinite dimensional linear subspaces these subspaces are defined in terms of a robust spectral theory with weights which is associated to the linear operator and is developed herein similar characterizations for the weighted eigenvalue problem and nonlinear variants will also be given finally topological methods are employed to obtain existence results for nonlinear problems including perturbations in the gradient which are then applied to the well posedness of the linear problem with lower order terms

multivariate smoothing spline functions given data z i g t i varepsilon i 1 leqq i leqq n where g is the unknown function the t i are known d dimensional variables in a domain omega and the varepsilon i are i

a unified framework for spline estimators this dissertation develops a uni ed framework to study the asymptotic properties of all periodic spline based estimators that is of regression penalized and smoothing splines the explicit form of the periodic demmler reinsch basis of general degree in terms of exponential splines allows to derive the exact expression for the equivalent kernel of all spline estimators simultaneously the corresponding bandwidth which drives the asymptotic behavior of spline estimators is shown to be a function of both the number of knots and the smoothing parameter a strategy for the optimal bandwidth selection is discussed

rates of convergence for minimum contrast estimators we shall present here a general study of minimum contrast estimators in a nonparametric setting although our results are also valid in the classical parametric case for independent observations these estimators include many of the most popular estimators in various situations such as maximum likelihood estimators least squares and other estimators of the regression function estimators for mixture models or deconvolution the main theorem relates the rate of convergence of those estimators to the entropy structure of the space of parameters optimal rates depending on entropy conditions are already known at least for some of the models involved and they agree with what we get for minimum contrast estimators as long as the entropy counts are not too large but under some circumstances large entropies or changes in the entropy structure due to local perturbations the resulting the rates are only suboptimal counterexamples are constructed which show that the phenomenon is real for non parametric maximum likelihood or regression this proves that under purely metric assumptions our theorem is optimal and that minimum contrast estimators happen to be suboptimal

variable bandwidth kernel estimators of regression curves on montre qu en termes de l erreur en moyenne quadratique integree asymptotique les estimateurs du noyau avec un certain choix de largeur de bande local sont superieurs aux estimateurs ordinaires du noyau avec un choix de largeur de bande global si on utilise des largeurs de bandes optimales

a class of singular gierer meinhardt systems of elliptic boundary value problems abstract we establish existence results for singular gierer meinhardt elliptic systems with zero dirichlet boundary conditions with the different range of growth rates of the source term we use the schauder fixed point theorem and upper lower solution methods to obtain the existence results we also study boundary estimates for certain singularities

asymptotic equivalence of density estimation and gaussian white noise signal recovery in gaussian white noise with variance tending to zero has served for some time as a representative model for nonparametric curve estimation having all the essential traits in a pure form the equivalence has mostly been stated informally but an approximation in the sense of le cam s deficiency distance would make it precise the models are then asymptotically equivalent for all purposes of statistical decision with bounded loss in nonparametrics a first result of this kind has recently been established for gaussian regression we consider the analogous problem for the experiment given by n i i d observations having density f on the unit interval our basic result concerns the parameter space of densities which are in a holder ball with exponent 1 2 and which are uniformly bounded away from zero we show that an i i d sample of size n with density f is globally asymptotically equivalent to a white noise experiment with drift f l 2 and variance 1 4n l this represents a nonparametric analog of le cam s heteroscedastic gaussian approximation in the finite dimensional case the proof utilizes empirical process techniques related to the hungarian construction white noise models on f and log f are also considered allowing for various automatic asymptotic risk bounds in the i i d model from white noise

a new lower bound for multiple hypothesis testing the purpose of this correspondence is to give a new easily tractable and sharp lower bound for the maximal error in multiple hypothesis testing with an application to nonasymptotic lower bounds for the minimax risk of estimators

some inequalities for information divergence and related measures of discrimination inequalities which connect information divergence with other measures of discrimination or distance between probability distributions are used in information theory and its applications to mathematical statistics ergodic theory and other scientific fields we suggest new inequalities of this type often based on underlying identities as a consequence we obtain certain improvements of the well known pinsker inequality our study depends on two measures of discrimination called capacitory discrimination and triangular discrimination the discussion contains references to related research and comparison with other measures of discrimination e g ali silvey csiszar 1996 1966 divergences and in particular the hellinger distance

potential theory for regular and mach reflection of a shock at a wedge if a plane shock hits a wedge a self similar pattern of reflected shocks travels outward as the shock moves forward in time the nature of the pattern is explored for weak incident shocks strength b and small wedge angles 2 w through potential theory a number of different scalings some study of mixed equations and matching asymptotics for the different scalings the self similar equations are of mixed type a linearization gives a linear mixed flow valid away from a sonic curve near the sonic curve a shock solution is constructed in another scaling except near the zone of interaction between the incident shock and the wall where a special scaling is used the parameter c1 2w 1 b ranges from 0 to here is the polytropic constant and c1 is the sound speed behind the incident shock for 2 regular reflection weak or strong can occur and the whole pattern is reconstructed to lowest order in shock strength for 1 2 mach reflection occurs and the flow behind the reflection is subsonic and can be constructed in principle with an open elliptic problem and matched the case 0 can be solved for 1 2 2 or even larger the flow behind a mach reflection may be transonic and further investigation must be made to determine what happens the basic pattern of reflection is an almost semi circular shock issuing for regular reflection from the reflection point on the wedge and for mach reflection matched with a local interaction flow assuming their nature choosing the least entropy generation the weak regular reflection will occur for sufficiently large von neumann paradox an accumulation point of vorticity occurs on the wedge above the leading point 1994 john wiley sons inc

robust estimation in models for independent non identically distributed data this paper concerns robust estimation of the parameter 0 which indexes a parametric model for independent non identically distributed data for reasonable choices of contamination neighborhood and of what is to be estimated when the parametric model does not hold we characterize asymptotically minimax robust estimates of 0 when applied to the normal regression model the theory yields recipes for the influence curves of optimal robust regression and scale estimates the contamination neighborhood does not assume regression plus error structure the regression and scale parameters are estimated simultaneously and the theory establishes r oles for estimates with redescending influence curves as well as for those with monotone influence curves when applied to the logit and probit models the theory recommends influence curves which differ markedly from those of the maximum likelihood estimates except in the i i d case

data compression and histograms in this paper the relationship between code length and the selection of the number of bins for a histogram density is considered for a sequence of iid observations on 0 1 first we use a shortest code length criterion to select the number of bins for a histogram a uniform almost sure asymptotic expansion for the code length is given and it is used to prove the asymptotic optimality of the selection rule in addition the selection rule is consistent if the true density is uniform 0 1 secondly we deal with the problem what is the best achievable average code length with underlying density functionf minimax lower bounds are derived for the average code length over certain smooth classes of underlying densitiesf for the smooth class with bounded first derivatives the rate in the lower bound is shown to be achieved by a code based on a sequence of histograms whose number of bins is changed predictively moreover this best code can be modified to ensure that the almost sure version of the code length has asymptotically the same behavior as its expected value i e the average code length

adaptive estimation of a quadratic functional by model selection we consider the problem of estimating s 2 when s belongs to some separable hilbert space and one observes the gaussian process y t langles t rangle sigmal t for all t epsilon mathbb h where l is some gaussian isonormal process this framework allows us in particular to consider the classical gaussian sequence model for which mathbb h l 2 mathbb n and l t sum lambda geq1 t lambda varepsilon lambda where varepsilon lambda lambda geq1 is a sequence of i i d standard normal variables our approach consists in considering some at most countable families of finite dimensional linear subspaces of mathbb h the models and then using model selection via some conveniently penalized least squares criterion to build new estimators of s 2 we prove a general nonasymptotic risk bound which allows us to show that such penalized estimators are adaptive on a variety of collections of sets for the parameter s depending on the family of models from which they are built in particular in the context of the gaussian sequence model a convenient choice of the family of models allows defining estimators which are adaptive over collections of hyperrectangles ellipsoids l p bodies or besov bodies we take special care to describe the conditions under which the penalized estimator is efficient when the level of noise sigma tends to zero our construction is an alternative to the one by efroimovich and low for hyperrectangles and provides new results otherwise

a class of quasilinear degenerate elliptic problems abstract we establish the existence of solutions for a class of quasilinear degenerate elliptic equations the equations in this class satisfy a structure condition which provides ellipticity in the interior of the domain and degeneracy only on the boundary equations of transonic gas dynamics for example satisfy this property in the region of subsonic flow and are degenerate across the sonic surface we prove that the solution is smooth in the interior of the domain but may exhibit singular behavior at the degenerate boundary the maximal rate of blow up at the degenerate boundary is bounded by the degree of degeneracy in the principal coefficients of the quasilinear elliptic operator our methods and results apply to the problems recently studied by several authors which include the unsteady transonic small disturbance equation the pressure gradient equations of the compressible euler equations and the singular quasilinear anisotropic elliptic problems and extend to the class of equations which satisfy the structure condition such as the shallow water equation compressible isentropic two dimensional euler equations and general two dimensional nonlinear wave equations our study provides a general framework to analyze degenerate elliptic problems arising in the self similar reduction of a broad class of two dimensional cauchy problems

on the use of singular value decomposition and decimation in discrete time band limited signal extrapolation the problem of extrapolating a band limited signal in discrete time is viewed as one of solving an underdetermined system of linear equations choosing the minimum norm least squares mnls solution is one criterion for singling out an extrapolation from all the possible solutions to the linear system use of the moore penrose inverse yields the mnls solution and singular value decomposition svd provides a means for implementing the moore penrose inverse an expression for the mean square error incurred in solving a linear system via svd is derived this can be used to estimate the number of singular values needed to form the inverse the error expression also indicates that decimation can be applied in the extrapolation problem to reduce the high computational cost of svd without degrading the extrapolation the results developed for the one dimensional case are extended to higher dimensions examples of the svd approach to extrapolation are given along with examples using other extrapolation techniques for comparison the svd approach compares favorably with known mnls extrapolation methods

approximation dans les espaces metriques et theorie de l estimation we investigate the relations between the speed of estimation and the metric structure of the parameter space especially in the case when its metric dimension is infinite given some distance d on generally hellinger distance in the case of n i i d variables we consider the minimax risk for n observations r n q mathop inf sup limits t n theta in theta text ie theta d q theta t n t n being any estimate of we shall look for functions r such that for positive constants c 1 q and c 2 q c 1 r q n r n q c2 r q n r n is the speed of estimation and we shall show under fairly general conditions including i i d variables and regular cases of markov chains and stationnary gaussian processes that r n is determined up to multiplicative constants by the metric structure of we shall also give a construction for some sort of universal estimates the risk of which is bounded by c 2 r q n in all cases where the preceding theory applies

some results in linear interpolation theory using a well known form for the inverse of a symmetric toeplitz matrix some results in linear interpolation theory are derived for an autoregressive process it is shown that interpolation at the mid point of a data record yields the minimum interpolation error also some results for infinite length interpolators are simply derived

field estimation from randomly located binary noisy sensors the estimation of bounded multivariate fields from 1 bit quantized and dithered noisy observations is considered we consider two models for random sensor deployment based on regular monte carlo simple random sampling and stratified sampling we propose linear estimators and for both sensors deployment methods we establish exact expressions for the bias and variance of the estimates including integrated mean square errors we show in particular that estimates of the field on the basis of stratified sensor locations always outperform estimates based on regular monte carlo sensor locations for both estimation schemes we also establish central limit theorems which can be used to compute the probability of events involving the estimates including confidence intervals

sparse pca optimal rates and adaptive estimation principal component analysis pca is one of the most commonly used statistical procedures with a wide range of applications this paper considers both minimax and adaptive estimation of the principal subspace in the high dimensional setting under mild technical conditions we first establish the optimal rates of convergence for estimating the principal subspace which are sharp with respect to all the parameters thus providing a complete characterization of the difficulty of the estimation problem in term of the convergence rate the lower bound is obtained by calculating the local metric entropy and an application of fano s lemma the rate optimal estimator is constructed using aggregation which however might not be computationally feasible we then introduce an adaptive procedure for estimating the principal subspace which is fully data driven and can be computed efficiently it is shown that the estimator attains the optimal rates of convergence simultaneously over a large collection of the parameter spaces a key idea in our construction is a reduction scheme which reduces the sparse pca problem to a high dimensional multivariate regression problem this method is potentially also useful for other related problems

rates of contraction for posterior distributions in lr metrics 1 r the frequentist behavior of nonparametric bayes estimates more specifically rates of contraction of the posterior distributions to shrinking lr norm neighborhoods 1 r of the unknown parameter are studied a theorem for nonparametric density estimation is proved under general approximation theoretic assumptions on the prior the result is applied to a variety of common examples including gaussian process wavelet series normal mixture and histogram priors the rates of contraction are minimax optimal for 1 r 2 but deteriorate as r increases beyond 2 in the case of gaussian nonparametric regression a gaussian prior is devised for which the posterior contracts at the optimal rate in all lr norms 1 r

free boundary and mixed boundary value problems for quasilinear elliptic equations tangential oblique derivative and degenerate dirichlet boundary problems abstract we present holder estimates and holder gradient estimates for a class of free boundary problems with tangential oblique derivative boundary conditions provided the oblique vector does not vanish at any point on the boundary we also establish the existence result for a general class of quasilinear degenerate problems of this type including nonlinear wave systems and the unsteady transonic small disturbance equation

minimax nonparametric classification i rates of convergence this paper studies minimax aspects of nonparametric classification we first study minimax estimation of the conditional probability of a class label given the feature variable this function say f is assumed to be in a general nonparametric class we show the minimax rate of convergence under square l sub 2 loss is determined by the massiveness of the class as measured by metric entropy the second part of the paper studies minimax classification the loss of interest is the difference between the probability of misclassification of a classifier and that of the bayes decision as is well known an upper bound on risk for estimating f gives an upper bound on the risk for classification but the rate is known to be suboptimal for the class of monotone functions this suggests that one does not have to estimate f well in order to classify well however we show that the two problems are in fact of the same difficulty in terms of rates of convergence under a sufficient condition which is satisfied by many function classes including besov sobolev lipschitz and bounded variation this is somewhat surprising in view of a result of devroye gorfi and lugosi see a probabilistic theory of pattern recognition new york springer verlag 1996

a singular quasilinear anisotropic elliptic boundary value problem ii we consider the problem uauxx ubuyy p x 0 with a 0 b 0 on a smooth convex bounded region in r2 with dirichlet boundary conditions we show that if the positive function p is uniformly bounded away from zero then the problem has a classical solution

lower bounds for the minimax risk using f divergences and applications lower bounds involving f divergences between the underlying probability measures are proved for the minimax risk in estimation problems our proofs just use simple convexity facts special cases and straightforward corollaries of our bounds include well known inequalities for establishing minimax lower bounds such as fano s inequality pinsker s inequality and inequalities based on global entropy conditions two applications are provided a new minimax lower bound for the reconstruction of convex bodies from noisy support function measurements and a different proof of a recent minimax lower bound for the estimation of a covariance matrix

riemann problems for the two dimensional unsteady transonic small disturbance equation we study a two parameter family of riemann problems for the unsteady transonic small disturbance utsd equation also called the two dimensional burgers equation which is used to model the transition from regular to mach reflection for weak shock waves the related initial value problem consists of oblique shock data in the upper half plane with two parameters a and b corresponding to the slopes of the initial shock waves the study of quasi steady solutions leads to a problem that changes type when written in self similar coordinates the problem is hyperbolic in the region where the flow is supersonic and elliptic where the flow is subsonic in this paper we give a complete description of the flow in the hyperbolic region by resolving the hyperbolic wave interactions in the form of quasi one dimensional riemann problems in the region of physical space where the flow is subsonic we pose the related free boundary problems and discuss the behavior of the subsonic solution using results from our previo

a linear time varying system framework for noniterative discrete time band limited signal extrapolation a technique for the analysis and design of noniterative algorithms for discrete time band limited signal extrapolation is described the approach involves modeling the extrapolation process as a linear time varying ltv system or filter together with a previously developed fourier theory for ltv systems this model provides a frequency domain transfer function representation for the extrapolation system this representation serves as a powerful tool for characterizing and comparing the reconstruction properties of several well known least squares optimal algorithms for band limited extrapolation moreover the frequency domain setting provides a conceptually attractive means for understanding the process of extrapolation itself additionally a least squares approximation methodology for designing ltv filters for band limited extrapolation is developed the design technique is shown to unify a broad class of algorithms for extrapolating discrete time data and further to provide a means for designing new and improved extrapolation algorithms

spline smoothing and optimal rates of convergence in nonparametric regression models linear estimation is considered in nonparametric regression models of the form yi f xi ei xi e a b where the zero mean errors are uncorrelated with common variance a2 and the response function f is assumed only to have a bounded square integrable qth derivative the linear estimator which minimizes the maximum mean squared error summed over the observation points is derived and the exact minimax rate of convergence is obtained for practical problems where bounds on 11 f2q 11 and a2 may be unknown generalized cross validation is shown to give an adaptive estimator which achieves the minimax optimal rate under the additional assumption of normality 1 the model consider the nonparametric regression model

optimal bandwidth selection in nonparametric regression function estimation kernel estimators of an unknown multivariate regression function are investigated a bandwidth selection rule is considered which can be formulated in terms of cross validation under mild assumptions on the kernel and the unknown regression function it is seen that this rule is asymptotically optimal

rates of convergence of minimum distance estimators and kolmogorov s entropy on construit des estimateurs robustes et on montre que le taux de convergence resultant depend naturellement d une fonction entropie

a free boundary problem for a quasi linear degenerate elliptic equation regular reflection of weak shocks we prove the existence of a solution to the weak regular reflection problem for the unsteady transonic small disturbance utsd model for shock reflection by a wedge in weak regular reflection the state immediately behind the reflected shock is supersonic and constant the flow becomes subsonic further downstream the equation in self similar coordinates is degenerate at the sonic line the reflected shock becomes transonic and begins to curve there its position is the solution to a free boundary problem for the degenerate equation using the rankine hugoniot conditions along the reflected shock we derive an evolution equation for the transonic shock and an oblique derivative boundary condition at the unknown shock position by regularizing the degenerate problem we construct uniform bounds we apply local compactness arguments to extract a limit that solves the problem the solution is smooth in the interior and continuous up to the degenerate boundary this work completes a stage in our program to construct self similar solutions of two dimensional riemann problems in a series of papers we developed techniques for solving the degenerate elliptic equations that arise in self similar reductions of hyperbolic conservation laws in other papers especially in joint work with gary lieberman we developed techniques for solving free boundary problems of the type that arise from rankine hugoniot relations for the first time in this paper we combine these approaches and show that they are compatible although our construction is limited to a finite part of the unbounded subsonic region it suggests that this approach has the potential to solve a variety of problems in weak shock reflection including mach and von neumann reflection in the utsd equation and the analogous problems for the unsteady full potential equation 2002 john wiley sons inc

on the existence of positive solutions of quasilinear elliptic boundary value problems abstract we establish the existence of positive solutions to a class of quasilinear anisotropic problems which have either sublinear or superlinear nonlinearity with a b nonnegative constants and positive constants one example is if b a 0 1 has a solution on the other hand if b a 1 superlinear case then there exists a 0 such that for 0 1 has at least one solution and for no solution exists

bandwidth choice for nonparametric regression this paper is concerned with the problem of choosing a bandwidth parameter for nonparametric regression we analyze a tapered fourier series estimate and discuss the relationship of this estimate to a kernel estimate we first consider a method based on an unbiased estimate of mean square error and show that the bandwidth thus chosen is asymptotically optimal other methods are examined as well and are shown to be asymptotically equivalent a small simulation shows however that for small or moderate sample size the methods perform quite differently

singular gierer meinhardt systems of elliptic boundary value problems abstract we establish existence results for singular gierer meinhardt elliptic systems with zero dirichlet boundary conditions gierer meinhardt systems are model problems for pattern formations of spatial tissue structures of morphogenesis the mathematical difficulties are that the system becomes singular near the boundary and it is non quasimonotone we show the existence of positive solutions for the activator inhibitor model with common sources

regularized policy iteration with nonparametric function spaces we study two regularization based approximate policy iteration algorithms namely reg lspi and reg brm to solve reinforcement learning and planning problems in discounted markov decision processes with large state and finite action spaces the core of these algorithms are the regularized extensions of the least squares temporal difference lstd learning and bellman residual minimization brm which are used in the algorithms policy evaluation steps regularization provides a convenient way to control the complexity of the function space to which the estimated value function belongs and as a result enables us to work with rich nonparametric function spaces we derive efficient implementations of our methods when the function space is a reproducing kernel hilbert space we analyze the statistical properties of reg lspi and provide an upper bound on the policy evaluation error and the performance loss of the policy returned by this method our bound shows the dependence of the loss on the number of samples the capacity of the function space and some intrinsic properties of the underlying markov decision process the dependence of the policy evaluation bound on the number of samples is minimax optimal this is the first work that provides such a strong guarantee for a nonparametric approximate policy iteration algorithm

estimation of a regression function by maxima of minima of linear functions in this paper estimation of a regression function from independent and identically distributed random variables is considered estimates are defined by minimization of the empirical l2 risk over a class of functions which are defined as maxima of minima of linear functions results concerning the rate of convergence of the estimates are derived in particular it is shown that for smooth regression functions satisfying the assumption of single index models the estimate is able to achieve up to some logarithmic factor the corresponding optimal one dimensional rate of convergence hence under these assumptions the estimate is able to circumvent the so called curse of dimensionality the small sample behavior of the estimates is illustrated by applying them to simulated data

probability inequalities for likelihood ratios and convergence rates of sieve mles let y 1 y n be independent identically distributed with density p 0 and let f be a space of densities we show that the supremum of the likelihood ratios i 1 n p y i p 0 y i where the supremum is over p f with p l 2 p 0 1 2 2 e is exponentially small with probability exponentially close to 1 the exponent is proportional to ne 2 the only condition required for this to hold is that e exceeds a value determined by the bracketing hellinger entropy of f a similar inequality also holds if we replace f by f n and p 0 by q n where q n is an approximation to p 0 in a suitable sense these results are applied to establish rates of convergence of sieve mles furthermore weak conditions are given under which the optimal rate e n defined by h e n f ne n 2 where h f is the hellinger entropy of f is nearly achievable by sieve estimators

existence and stability of perturbed transonic shocks for compressible steady potential flows abstract we establish an existence theorem for perturbed transonic steady potential flows where the boundary of the subsonic region consists of a transonic shock and thus the governing equation becomes a free boundary problem we establish gradient estimates and use the fixed point theorem of caristi s which does not require a compactness of the map to establish the existence result we provide new techniques to handle the gradients and show that the flow is strictly elliptic inside of the subsonic region

on maximum likelihood estimation in infinite dimensional parameter spaces an approximate maximum likelihood estimate is known to be consistent under some compactness and integrability conditions in this paper we study its convergence rate and its asymptotic efficiency in estimating smooth functionals of the parameter we provide conditions under which the rate of convergence can be established this rate is essentially governed by the size of the space of score functions as measured by an entropy index we also show that for a large class of smooth functionals the plug in maximum likelihood estimate is asymptotically efficient that is it achieves the minimal fisher information bound the theory is illustrated by several nonparametric or semiparametric examples

from stein s unbiased risk estimates to the method of generalized cross validation this paper concerns the method of generalized cross validation gcv a promising way of choosing between linear estimates based on stein estimates and the associated unbiased risk estimates stein 1981 a new approach to gcv is developed many consistency results are obtained for the cross validated steinized estimates in the contexts of nearest neighbor nonparametric regression model selection ridge regression and smoothing splines moreover the associated stein s unbiased risk estimate is shown to be uniformly consistent in assessing the true loss not the risk consistency properties are examined as well when the sampling error is unknown finally we propose a variant of gcv to handle the case that the dimension of the raw data is known to be greater than that of their expected values

constraints on possible singularities for the unsteady transonic small disturbance utsd equations we discuss the singular behavior of solutions to two dimensional general secondorder uniformly elliptic equations in divergence form with bounded measurable coefficients and discontinuous dirichlet data along a portion of a lipschitz boundary we show that the conjugate to the solution develops a singularity that is at least logarithmic along the boundary at the points of discontinuity in the boundary data a problem like this arises in the study of self similar solutions to the hyperbolic conservation laws in two space dimensions given by the unsteady transonic small disturbance utsd flow equations these solutions model the reflection of a weak shock wave upon a thin wedge in the regime where the von neumann paradox applies the present result provides a step in the direction of understanding the nature of the solutions to the utsd equations near a triple point it shows that the flow behind the triple point cannot be strictly subsonic under some mild assumptions on the solutions c 1999 john wiley sons inc

subsonic solutions to compressible transonic potential problems for isothermal self similar flows and steady flows abstract we establish boundary and interior gradient estimates and show that no supersonic bubble appears inside of a subsonic region for transonic potential flows for both self similar isothermal and steady problems we establish an existence result for the self similar isothermal problem and improve the hopf maximum principle to show that the flow is strictly elliptic inside of the subsonic region for the steady problem

adapting to unknown smoothness via wavelet shrinkage abstract we attempt to recover a function of unknown smoothness from noisy sampled data we introduce a procedure sureshrink that suppresses noise by thresholding the empirical wavelet coefficients the thresholding is adaptive a threshold level is assigned to each dyadic resolution level by the principle of minimizing the stein unbiased estimate of risk sure for threshold estimates the computational effort of the overall procedure is order n log n as a function of the sample size n sureshrink is smoothness adaptive if the unknown function contains jumps then the reconstruction essentially does also if the unknown function has a smooth piece then the reconstruction is essentially as smooth as the mother wavelet will allow the procedure is in a sense optimally smoothness adaptive it is near minimax simultaneously over a whole interval of the besov scale the size of this interval depends on the choice of mother wavelet we know from a previous paper by the authors that traditional smoot

subsonic solutions for compressible transonic potential flows abstract we establish an existence theorem for transonic isentropic potential flows where the subsonic region is bounded by the sonic line and thus the governing equation may become degenerate on the boundary partly or entirely it has been conjectured by experiments and numerical studies that the self similar multidimensional flow changes its type namely hyperbolic far from the origin supersonic region and elliptic near the origin subsonic region furthermore the potential equation has a different nonlinearity compared to other transonic problems such as the unsteady transonic small disturbance equation the nonlinear wave equation and the pressure gradient equation namely the coefficients of the potential equation depend on the gradients while others are independent of the gradients we provide techniques to handle the gradients establish interior and boundary gradient estimates for the potential flow in a convex region and answer the conjecture that is the flow is strictly elliptic and the region is subsonic

covering numbers for bounded variation functions abstract in this paper we provide upper and lower estimates for the minimal number of functions needed to represent a bounded variation function with an accuracy of epsilon with respect to l 1 distance

asymptotic equivalence of nonparametric autoregression and nonparametric regression it is proved that nonparametric autoregression is asymptotically equivalent in the sense of le cam s deficiency distance to nonparametric regression with random design as well as with regular nonrandom design

a comparison of cross validation techniques in density estimation in the setting of nonparametric multivariate density estimation theorems are established which allow a comparison of the kullback leibler and the least squares cross validation methods of smoothing parameter selection the family of delta sequence estimators including kemel orthogonal series histogram and histospline estimators is considered these theorems also show that either type of cross validation can be used to compare different estimators e g kernel versus orthogonal series 1 introduction consider the problem of trying to estimate a d dimensional probability density function f x using a random sample xl xn from f most proposed estimators of f depend on a smoothing parameter say x e dr whose selection is crucial to the performance of the estimator in this paper for the large class of delta sequence estimators theorems are obtained which allow comparison of two smoothing parameter selectors which are known to be asymptotically optimal an important consequence of these results is that either smoothing parameter selector may be used for a data based comparison of two density estimators for example kernel versus orthogonal series another attractive feature of these results is that they are set in a quite general framework special cases of which provide simpler proofs of several recent asymptotic optimality results in sections 2 and 3 the family of delta sequence estimators and the smoothing parameter selectors are given the theorems are stated in section 4 with some remarks in section 5 the rest of the paper consists of proofs

wavelet shrinkage asymptopia much recent effort has sought asymptotically minimax methods for recovering infinite dimensional objects curves densities spectral densities images from noisy data a now rich and complex body of work develops nearly or exactly minimax estimators for an array of interesting problems unfortunately the results have rarely moved into practice for a variety of reasons among them being similarity to known methods computational intractability and lack of spatial adaptivity we discuss a method for curve estimation based on n noisy data translate the empirical wavelet coefficients towards the origin by an amount 2 log n n the proposal differs from those in current use is computationally practical and is spatially adaptive it thus avoids several of the previous objections further the method is nearly minimax both for a wide variety of loss functions pointwise error global error measured in l p norms pointwise and global error in estimation of derivatives and for a wide range of smoothness classes including standard holder and sobolev classes and bounded variation this is a much broader near optimality than anything previously proposed we draw loose parallels with near optimality in robustness and also with the broad near eigenfunction properties of wavelets themselves finally the theory underlying the method is interesting as it exploits a correspondence between statistical questions and questions of optimal recovery and information based complexity

existence and regularity of nonnegative solution of a singular quasi linear anisotropic elliptic boundary value problem with gradient terms abstract in this paper we consider the singular quasi linear anisotropic elliptic boundary value problem p f 1 u u x x u y y g u u q f u 0 x y u 0 where is a smooth bounded domain in r 2 0 q 2 f 1 0 0 f 1 t 0 t 0 f 1 is a smooth function in and is a non decreasing function in 0 g t 0 g is a smooth function in 0 0 and is a non increasing function in 0 f t 0 f is a smooth function in 0 0 and is a strictly decreasing function in 0 clearly this is a boundary degenerate elliptic problem if f 1 0 0 we show that the solution of the dirichlet boundary value problem p is smooth in the interior and continuous or lipschitz continuous up to the degenerate boundary and give the conditions for which gradients of solutions are bounded or unbounded we believe that these results on regularity of the solution should be very useful

distributed field estimation with randomly deployed noisy binary sensors the reconstruction of a bounded deterministic field from binary quantized observations of sensors which are randomly deployed over the field domain is studied the sensor observations are corrupted by bounded additive noise the study focuses on the extremes of lack of deterministic control in the sensor deployment lack of knowledge of the noise distribution and lack of sensing precision and reliability such adverse conditions are motivated by possible real world scenarios where a large collection of low cost crudely manufactured sensors are mass deployed in an environment where little can be assumed about the ambient noise a simple estimator that reconstructs the entire field from these unreliable binary quantized noisy observations is proposed technical conditions for the almost sure and mean squared error mse convergence of the estimate to the field as the number of sensors tends to infinity are derived and their implications are discussed for finite dimensional bounded variation and sobolev differentiable function classes specific mse decay rates are derived

adaptive interpolation of discrete time signals that can be modeled as autoregressive processes this paper presents an adaptive algorithm for the restoration of lost sample values in discrete time signals that can locally be described by means of autoregressive processes the only restrictions are that the positions of the unknown samples should be known and that they should be embedded in a sufficiently large neighborhood of known samples the estimates of the unknown samples are obtained by minimizing the sum of squares of the residual errors that involve estimates of the autoregressive parameters a statistical analysis shows that for a burst of lost samples the expected quadratic interpolation error per sample converges to the signal variance when the burst length tends to infinity the method is in fact the first step of an iterative algorithm in which in each iteration step the current estimates of the missing samples are used to compute the new estimates furthermore the feasibility of implementation in hardware for real time use is established the method has been tested on artificially generated auto regressive processes as well as on digitized music and speech signals

metalearners for estimating heterogeneous treatment effects using machine learning there is growing interest in estimating and analyzing heterogeneous treatment effects in experimental and observational studies we describe a number of metaalgorithms that can take advantage of any supervised learning or regression method in machine learning and statistics to estimate the conditional average treatment effect cate function metaalgorithms build on base algorithms such as random forests rfs bayesian additive regression trees barts or neural networks to estimate the cate a function that the base algorithms are not designed to estimate directly we introduce a metaalgorithm the x learner that is provably efficient when the number of units in one treatment group is much larger than in the other and can exploit structural properties of the cate function for example if the cate function is linear and the response functions in treatment and control are lipschitz continuous the x learner can still achieve the parametric rate under regularity conditions we then introduce versions of the x learner that use rf and bart as base learners in extensive simulation studies the x learner performs favorably although none of the metalearners is uniformly the best in two persuasion field experiments from political science we demonstrate how our x learner can be used to target treatment regimes and to shed light on underlying mechanisms a software package is provided that implements our methods

the minimum description length principle in coding and modeling we review the principles of minimum description length and stochastic complexity as used in data compression and statistical modeling stochastic complexity is formulated as the solution to optimum universal coding problems extending shannon s basic source coding theorem the normalized maximized likelihood mixture and predictive codings are each shown to achieve the stochastic complexity to within asymptotically vanishing terms we assess the performance of the minimum description length criterion both from the vantage point of quality of data compression and accuracy of statistical inference context tree modeling density estimation and model selection in gaussian linear regression serve as examples

asymptotic optimality of regular sequence designs we study linear estimators for the weighted integral of a stochastic process the process may only be observed on a finite sampling design the error is defined in a mean square sense and the process is assumed to satisfy sacks ylvisaker regularity conditions of order r n o we show that sampling at the quantiles of a particular density already yields asymptotically optimal estimators hereby we extend the results of sacks and ylvisaker for regularity r o or 1 and we confirm a conjecture by eubank smith and smith

efficient shrinkage in parametric models this paper introduces shrinkage for general parametric models we show how to shrink maximum likelihood estimators towards parameter subspaces de ned by general nonlinear restrictions we derive the asymptotic distribution and risk of the generalized shrinkage estimator using a local asymptotic framework we show that if the shrinkage dimension exceeds two the asymptotic risk of the shrinkage estimator is strictly less than that of the mle this reduction holds globally in the parameter space we show that the reduction in asymptotic risk is substantial even for moderately large values of the parameters the formula simplify in a very convenient way in the context of high dimensional models we derive a simple bound for the asymptotic risk we also provide a new large sample minimax e ciency bound we use the concept of local asymptotic minimax bounds a generalization of the conventional asymptotic minimax bounds the

higher regularity for free boundary and tangential derivative problems and global solutions to regular reflections for the unsteady transonic small disturbance utsd equations abstract we establish c 2 estimates for solutions of a class of quasilinear elliptic equations with free boundary and tangential derivative boundary problems using this regularity result we show the existence of global solutions to regular shock reflections for the unsteady transonic small disturbance utsd equation we also present lipschitz estimates near the degenerate dirichlet boundary the sonic boundary for the utsd equation

a numerical study of riemann problems for the two dimensional unsteady transonic small disturbance equation we study a two parameter family of riemann problems for the unsteady transonic small disturbance utsd equation also called the two dimensional burgers equation the two parameters a and b which define oblique shock initial data correspond to the slopes of the initial shock waves in the upper half plane for each a and b the three constant states in the upper half plane satisfy the rankine hugoniot conditions across the shocks this leads to a two parameter family of oblique shock interaction problems in this paper we present a numerical study of global solution behavior for the values of a and b in a previously obtained bifurcation diagram our study supplements the related theoretical results and conjectures recently obtained by s canic and b l keyfitz we employ a high resolution numerical method which reveals fine solution structures our findings confirm theoretical results and conjectures about the solution patterns and deepen the understanding of the structure of several intricate wave in

multivariate two sample tests based on nearest neighbors abstract a new class of simple tests is proposed for the general multivariate two sample problem based on the possibly weighted proportion of all k nearest neighbor comparisons in which observations and their neighbors belong to the same sample large values of the test statistics give evidence against the hypothesis h of equality of the two underlying distributions asymptotic null distributions are explicitly determined and shown to involve certain nearest neighbor interaction probabilities simple infinite dimensional approximations are supplied the unweighted version yields a distribution free test that is consistent against all alternatives optimally weighted statistics are also obtained and asymptotic efficiencies are calculated each of the tests considered is easily adapted to a permutation procedure that conditions on the pooled sample power performance for finite sample sizes is assessed in simulations

a global subsonic solution to an interacting transonic shock for the self similar nonlinear wave equation abstract we present the existence of the subsonic solution to a two dimensional riemann problem governed by a self similar nonlinear wave equation where the boundary of the subsonic region consists of a transonic shock and the sonic circle thus the governing equation becomes a free boundary problem on the transonic shock and degenerates on the sonic circle by utilizing the barrier methods and iterative methods we show the well posedness of the transonic shock in the entire subsonic region and thus establish the global solution this result does not rely on any smallness of riemann data

an asymptotic property of model selection criteria probability models are estimated by use of penalized log likelihood criteria related to akaike 1973 information criterion aic and minimum description length mdl the accuracies of the density estimators are shown to be related to the tradeoff between three terms the accuracy of approximation the model dimension and the descriptive complexity of the model classes the asymptotic risk is determined under conditions on the penalty term and is shown to be minimax optimal for some cases as an application we show that the optimal rate of convergence is simultaneously achieved for log densities in sobolev spaces w sub 2 sup s u without knowing the smoothness parameter s and norm parameter u in advance applications to neural network models and sparse density function estimation are also provided

convex and nonconvex formulations for mixed regression with two components minimax optimal rates we consider the mixed regression problem with two components under adversarial and stochastic noise we give a convex optimization formulation that provably recovers the true solution as well as a nonconvex formulation that works under more general settings and remains tractable upper bounds are provided on the recovery errors for both arbitrary noise and stochastic noise models we also give matching minimax lower bounds up to log factors showing that our algorithm is information theoretical optimal in a precise sense our results represent the first tractable algorithm guaranteeing successful recovery with tight bounds on recovery errors and sample complexity moreover we pinpoint the statistical cost of mixtures our minimax optimal results indicate that the mixture poses a fundamentally more difficult problem in the low snr regime where the learning rate changes

covering numbers for convex functions in this paper we study the covering numbers of the space of convex and uniformly bounded functions in multidimension we find optimal upper and lower bounds for the epsilon covering number of cal c a b d b in the l p metric 1 leq p in terms of the relevant constants where d geq 1 a b 0 and cal c a b d b denotes the set of all convex functions on a b d that are uniformly bounded by b we summarize previously known results on covering numbers for convex functions and also provide alternate proofs of some known results our results have direct implications in the study of rates of convergence of empirical minimization procedures as well as optimal convergence rates in the numerous convexity constrained function estimation problems

lower bounds in pattern recognition and learning lower bounds are derived for the performance of any pattern recognition algorithm which using training data selects a discrimination rule from a certain class of rules the bounds involve the vapnik chervonenkis dimension of the class and l the minimal error probability within the class we provide lower bounds when l 0 the usual assumption in valiant s theory of learning and l 0

resampling consistency of substitution estimators on the basis of n i i d random variables with a common unknown distribution p we wish to estimate a functional n p an obvious and very general approach to this problem is to find an estimator p n of p first and then construct a so called substitution estimator n p n of n p in this paper we investigate how to choose the estimator p n so that the substitution estimator n p n will be consistent

efficient robust estimates in parametric models let p n an open subset ofr k be a regular parametric model for a sample ofn independent identically distributed observations this paper describes estimates t n n 1 of which are asymptotically efficient under the parametric model and are robust under small deviations from that model in essence the estimates are adaptively modified one step maximum likelihood estimates which adjust themselves according to how well the parametric model appears to fit the data when the fit seems poor t n discounts observations that would have large influence on the value of the usual one step mle the estimates t n are shown to be asymptotically minimax in the hajek lecam sense for a hellinger ball contamination model an alternative construction of robust asymptotically minimax estimates as modified mle s is described for canonical exponential families

minimax estimation in sparse canonical correlation analysis canonical correlation analysis is a widely used multivariate statistical technique for exploring the relation between two sets of variables this paper considers the problem of estimating the leading canonical correlation directions in high dimensional settings recently under the assumption that the leading canonical correlation directions are sparse various procedures have been proposed for many high dimensional applications involving massive data sets however there has been few theoretical justification available in the literature in this paper we establish rate optimal nonasymptotic minimax estimation with respect to an appropriate loss function for a wide range of model spaces two interesting phenomena are observed first the minimax rates are not affected by the presence of nuisance parameters namely the covariance matrices of the two sets of random variables though they need to be estimated in the canonical correlation analysis problem second we allow the presence of the residual canonical correlation directions however they do not influence the minimax rates under a mild condition on eigengap a generalized sin theta theorem and an empirical process bound for gaussian quadratic forms under rank constraint are used to establish the minimax upper bounds which may be of independent interest

geometrizing rates of convergence ii consider estimating a functional t f of an unknown distribution f in mathbf f from data x 1 cdots x n i i d f let omega varepsilon denote the modulus of continuity of the functional t over mathbf f computed with respect to hellinger distance for well behaved loss functions l t we show that inf t n sup mathbf f e fl t n t f is equivalent to l omega n 1 2 to within constants whenever t is linear and mathbf f is convex the same conclusion holds in three nonlinear cases estimating the rate of decay of a density estimating the mode and robust nonparametric regression we study the difficulty of testing between the composite infinite dimensional hypotheses h 0 t f leq t and h 1 t f geq t delta our results hold in the cases studied because the difficulty of the full infinite dimensional composite testing problem is comparable to the difficulty of the hardest simple two point testing subproblem

convergence rates of posterior distributions for non i i d observations we consider the asymptotic behavior of posterior distributions and bayes estimators based on observations which are required to be neither independent nor identically distributed we give general results on the rate of convergence of the posterior measure relative to distances derived from a testing criterion we then specialize our results to independent nonidentically distributed observations markov processes stationary gaussian time series and the white noise model we apply our general results to several examples of infinite dimensional statistical models including nonparametric regression with normal errors binary regression poisson regression an interval censoring model whittle estimation of the spectral density of a time series and a nonlinear autoregressive model

an elliptic problem arising from the unsteady transonic small disturbance equation abstract we prove a theorem on existence of a weak solution of the dirichlet problem for a quasilinear elliptic equation with a degeneracy on one part of the boundary the degeneracy is of a type keldysh type associated with singular behavior blow up of a derivative at the boundary we define an associated operator which is continuous pseudo monotone and coercive and show that a weak solution displaying singular behavior at the boundary exists

estimating the intensity of a random measure by histogram type estimators the purpose of this paper is to estimate the intensity of some random measure n on a set mathcal x by a piecewise constant function on a finite partition of mathcal x given a possibly large family mathcal m of candidate partitions we build a piecewise constant estimator histogram on each of them and then use the data to select one estimator in the family choosing the square of a hellinger type distance as our loss function we show that each estimator built on a given partition satisfies an analogue of the classical squared bias plus variance risk bound moreover the selection procedure leads to a final estimator satisfying some oracle type inequality with as usual a possible loss corresponding to the complexity of the family mathcal m when this complexity is not too high the selected estimator has a risk bounded up to a universal constant by the smallest risk bound obtained for the estimators in the family for suitable choices of the family of partitions we deduce uniform risk bounds over various classes of intensities our approach applies to the estimation of the intensity of an inhomogenous poisson process among other counting processes or the estimation of the mean of a random vector with nonnegative components