information theoretic determination of minimax rates of convergence we present some general results determining minimax bounds on statistical risk for density estimation based on certain information theoretic considerations these bounds depend only on metric entropy conditions and are used to identify the minimax rates of convergence

on inverse problems with unknown operators consider a problem of recovery of a smooth function signal image f spl isin spl fscr spl isin l sub 2 0 1 sup d passed through an unknown filter and then contaminated by a noise a typical model discussed in the paper is described by a stochastic differential equation dy sub f sup spl epsi t hf t dt spl epsi dw t t spl isin 0 1 sup d spl epsi 0 where h is a linear operator modeling the filter and w is a brownian motion sheet modeling a noise the aim is to recover f with asymptotically as spl epsi spl rarr 0 minimax mean integrated squared error traditionally the problem is studied under the assumption that the operator h is known then the ill posedness of the problem is the main concern in this paper a more complicated and more realistic case is considered where the operator is unknown instead a training set of n pairs e sub l y e sub l sup spl sigma l 1 2 n where e sub l is an orthonormal system in l sub 2 and y e sub l sup spl sigma denote the solutions of stochastic differential equations of the above type with f e sub l and spl epsi spl sigma is available an optimal in a minimax sense over considered operators and signals data driven recovery of the signal is suggested the influence of spl epsi spl sigma and n on the recovery is thoroughly studied in particular we discuss an interesting case of a larger noise during the training and present formulas for threshold levels for n beyond which no improvement in recovery of input signals occurs we also discuss the case where h is an unknown perturbation of a known operator we describe a class of perturbations for which the accuracy of recovery of the signal is asymptotically the same up to a constant as in the case of precisely known operator

a new boundary element method for bending of plates on elastic foundations abstract the bending of plates on a winkler foundation according to kirchhoff s theory is solved by using an original boundary integral equation method involving the fundamental solution for plate flexure problems an integral representation for the second member pressure of the foundation of the equation is given by discretizing the integral equation it is possible to eliminate the boundary unknowns so that one is reduced to solving a linear system the solutions of which are deflections inside the domain to illustrate the potentialities of this method several problems with various boundary conditions loads and values of the modulus of the foundation are successfully solved

local asymptotics for polynomial spline regression in this paper we develop a general theory of local asymptotics for least squares estimates over polynomial spline spaces in a regression problem the polynomial spline spaces we consider include univariate splines tensor product splines and bivariate or multivariate splines on triangulations we establish asymptotic normality of the estimate and study the magnitude of the bias due to spline approximation the asymptotic normality holds uniformly over the points where the regression function is to be estimated and uniformly over a broad class of design densities error distributions and regression functions the bias is controlled by the minimum l norm of the error when the target regression function is approximated by a function in the polynomial spline space that is used to define the estimate the control of bias relies on the stability in l norm of l 2 projections onto polynomial spline spaces asymptotic normality of least squares estimates over polynomial or trigonometric polynomial spaces is also treated by the general theory in addition a preliminary analysis of additive models is provided

on completeness and consistency in nonparametric instrumental variable models this paper provides positive testability results for the identification condition in a nonparametric instrumental variable model known as completeness and it links the outcome of the test to properties of an estimator of the structural function in particular i show that the data can provide empirical evidence in favor of both an arbitrarily small identified set as well as an arbitrarily small asymptotic bias of the estimator this is the case for a large class of complete distributions as well as certain incomplete distributions as a byproduct the results can be used to estimate an upper bound of the diameter of the identified set and to obtain an easy to report estimator of the identified set itself

nonparametric estimation in case of endogenous selection this paper addresses the problem of estimation of a nonparametric regression function from selectively observed data when selection is endogenous our approach relies on independence between covariates and selection conditionally on potential outcomes endogeneity of regressors is also allowed for in the exogenous and endogenous case consistent two step estimation procedures are proposed and their rates of convergence are derived pointwise asymptotic distribution of the estimators is established in addition bootstrap uniform confidence bands are obtained finite sample properties are illustrated in a monte carlo simulation study and an empirical illustration

spectral precoding for rectangularly pulsed ofdm spectrally precoded orthogonal frequency division multiplexing ofdm is a promising rectangularly pulsed ofdm signaling format which can provide very small power spectral sidelobes while allowing for efficient implementation by fast fourier transform and guard interval insertion in this paper general constraints on spectral precoding are developed for ofdm signals with zero padding zp ofdm or cyclic prefix cp ofdm to warrant the desirable spectral property that the power spectral sidelobes decay asymptotically as f 2k 2 where k is a preassigned positive integer in accordance with the constraints block partitioning is adopted to construct a general signaling format to facilitate the precoder design of the spectrally precoded cp ofdm that can provide fast decaying sidelobes new correlative and orthogonal precoders are also devised so that the desirable spectral property of fast sidelobe decaying is achieved with spectrally precoded zp ofdm and cp ofdm signals

on estimating a density using hellinger distance and some other strange facts on s interesse ici aux possibles vitesses d estimation d une densite a support compact dans m sous des hypotheses de regularite lorsque la perte est mesuree par le carre de la distance de hellinger on regardera aussi le cas connu des normes mathbb l q pour 1 q 2 et le risque est le risque minimax sur la famille on donne une methode generale permettant de traiter les problemes dans le cadre de la theorie de l approximation sous des conditions concernant l entropie metrique et l e capacite des familles a estimer les rapports entre regularite et entropie metrique etant bien connus nous pourrons aussi traiter les cas classiques et d autres qui le sont moins sous des conditions de bornes inferieures les vitesses sont celles observees pour la norme mathbb l q mais elles different dans le cas general on montre aussi que les restrictions sur la compacite du support ou la regularite sont indispensables et que leur absence mene a l impossibilite d obtenir une estimation raisonnable en ce sens que n importe quelle suite d estimateurs sera arbitrairement mauvaise en un point au moins un resultat analogue est vrai sous des conditions de regularite

smoothed gmm for quantile models abstract we consider estimation of finite dimensional parameters identified by general conditional quantile restrictions including instrumental variables quantile regression within a generalized method of moments framework moment functions are smoothed to aid both computation and precision consistency and asymptotic normality are established under weaker assumptions than previously seen in the literature allowing dependent data and nonlinear structural models simulations illustrate the finite sample properties an in depth empirical application estimates the consumption euler equation derived from quantile utility maximization advantages of quantile euler equations include robustness to fat tails decoupling risk attitude from the elasticity of intertemporal substitution and error free log linearization

projection estimation in multiple regression with application to functional anova models a general theory on rates of convergence of the least squares projection estimate in multiple regression is developed the theory is applied to the functional anova model where the multivariate regression function is modeled as a specified sum of a constant term main effects functions of one variable and selected interaction terms functions of two or more variables the least squares projection is onto an approximating space constructed from arbitrary linear spaces of functions and their tensor products respecting the assumed anova structure of the regression function the linear spaces that serve as building blocks can be any of the ones commonly used in practice polynomials trigonometric polynomials splines wavelets and finite elements the rate of convergence result that is obtained reinforces the intuition that low order anova modeling can achieve dimension reduction and thus overcome the curse of dimensionality moreover the components of the projection estimate in an appropriately defined anova decomposition provide consistent estimates of the corresponding components of the regression function when the regression function does not satisfy the assumed anova form the projection estimate converges to its best approximation of that form

boundary element analysis of thick reissner plates on two parameter foundation abstract this paper introduces a consistent theory for boundary element analysis of thick reissner plates resting on a one or two parameter elastic foundation fundamental solutions and kernel functions have been derived as combinations of a group for plates on a one parameter foundation and another group representing the effect of a second foundation parameter domain integral loading terms have been reduced for cases with uniformly and linearly distributed loadings and concentrated shear forces and bending moments case studies with different loading and boundary conditions have been analysed and the boundary element results have confirmed the soundness and accuracy of the developed theory

rates of convergence for minimum contrast estimators we shall present here a general study of minimum contrast estimators in a nonparametric setting although our results are also valid in the classical parametric case for independent observations these estimators include many of the most popular estimators in various situations such as maximum likelihood estimators least squares and other estimators of the regression function estimators for mixture models or deconvolution the main theorem relates the rate of convergence of those estimators to the entropy structure of the space of parameters optimal rates depending on entropy conditions are already known at least for some of the models involved and they agree with what we get for minimum contrast estimators as long as the entropy counts are not too large but under some circumstances large entropies or changes in the entropy structure due to local perturbations the resulting the rates are only suboptimal counterexamples are constructed which show that the phenomenon is real for non parametric maximum likelihood or regression this proves that under purely metric assumptions our theorem is optimal and that minimum contrast estimators happen to be suboptimal

instrumental variable estimation of nonseparable models there are many environments where knowledge of a structural relationship is required to answer questions of interest also nonseparability of a structural disturbance is a key feature of many models here we consider nonparametric identification and estimation of a model that is monotonic in a nonseparable scalar disturbance which disturbance is independent of instruments this model leads to conditional quantile restrictions we give local identification conditions for the structural equations from those quantile restrictions we find that a modified completeness condition is sufficient for local identification we also consider estimation via a nonparametric minimum distance estimator the estimator minimizes the sum of squares of predicted values from a nonparametric regression of the quantile residual on the instruments we show consistency of this estimator

density estimation by wavelet thresholding density estimation is a commonly used test case for nonparametric estimation methods we explore the asymptotic properties of estimators based on thresholding of empirical wavelet coefficients minimax rates of convergence are studied over a large range of besov function classes b pq and for a range of global l p error measures 1 p p some form of nonlinearity is essential since the minimax linear estimators are suboptimal by polynomial powers of n a second approach using an approximation of a gaussian white noise model in a mallows metric is used to attain exactly optimal rates of convergence for quadratic error p 2

bounding quantile demand functions using revealed preference inequalities this paper develops a new technique for the estimation of consumer demand models with unobserved heterogeneity subject to revealed preference inequality restrictions particular attention is given to nonseparable heterogeneity the inequality restrictions are used to identify bounds on quantile demand functions a nonparametric estimator for these bounds is developed and asymptotic properties are derived an empirical application using data from the u k family expenditure survey illustrates the usefulness of the methods by deriving bounds and con dence sets for estimated quantile demand functions

a new fundamental solution for boundary element analysis of thin plates on winkler foundation this paper introduces an efficient boundary element approach for the analysis of thin plates with arbitrary shapes and boundary conditions resting on an elastic winkler foundation boundary integral equations with three degrees of freedom per node are derived without unknown corner terms a fundamental solution based upon newly defined modified kelvin functions is formulated and it leads to a simple solution to the problem of divergent integrals reduction of domain loading terms for cases of distributed and concentrated loading is also provided case studies including plates with free edge conditions are demonstrated and the boundary element results are compared with corresponding analytical solutions the presented formulations provide a very accurate boundary element solution for plates with different shapes and boundary conditions

simultaneous optimization of shape and flow parameters for combined free and forced convection in vertical channels simultaneous optimization of shape and flow parameters is performed for a combined free and forced convection flow through vertical rectangular channels with moving walls the laminar flow is assumed to be fully developed in the axial direction the wall velocity the axial pressure gradient and the channel height in the transverse plane are taken as the optimization parameters the sensitivity expressions of both the objective function and the flow rate constraint of optimization are obtained in terms of the relevant physical variables as well as adjoint variables which satisfy additional p d e s all equations are discretized using the finite element method numerical results are provided for the present constrained optimization problem for various values of the problem parameters which include the moving wall segment size and the rayleigh number the results indicate that with increased rayleigh number the optimal values of the wall velocity and the axial pressure gradient are increased while the optimal value of the channel height is decreased general sensitivity expressions are also presented in the appendix which might be utilized for arbitrary boundary variations along with arbitrary optimization objectives in other investigations 1997 by john wiley sons ltd

on density estimation in the view of kolmogorov s ideas in approximation theory the paper deals with upper and lower bounds for the quality of probability density estimation connections are established between these problems and the theory of approximation of functions particularly it is demonstrated how some of kolmogorov s concepts work

hellinger consistency of certain nonparametric maximum likelihood estimators consider a class mathscr p p theta theta in theta of probability measures on a measurable space mathscr x mathscr a dominated by a sigma finite measure mu let f theta dp theta d mu theta in theta and let theta n be a maximum likelihood estimator based on n independent observations from p theta 0 theta 0 in theta we use results from empirical process theory to obtain convergence for the hellinger distance h f hat theta n f theta 0 under certain entropy conditions on the class of densities f theta theta in theta the examples we present are a model with interval censored observations smooth densities monotone densities and convolution models in most examples the convexity of the class of densities is of special importance

numerical methods for analysis of plates on tensionless elastic foundations abstract a numerical methodology for analysis of plates resting on tensionless elastic foundations described either by the winkler model or as an elastic half space is presented in this paper the contact surface is assumed unbonded and frictionless the finite element method is used to discretize the plate and foundation to overcome the difficulties in solving the plate foundation equilibrium equations together with the inequality constraints due to the frictionless unilateral contact condition a variational formulation equivalent to these equations is presented from which three alternative linear complementary problems lcp are derived and solved by lemke s complementary pivoting algorithm in the first formulation the lcp variables are the plate displacements and the elastic foundation reaction in the second the lcp is derived in terms of the elastic foundation reaction and in the third formulation the variables are the elastic foundation displacements and the gap between the bodies once the lcp is solved the no contact regions where the plate lifts up away from the foundation and the sub grade reaction as well as the plate displacements and stresses can be easily obtained the methodology is illustrated by three examples and the results are compared with existing analytical and numerical results found in the literature

estimation of nonparametric conditional moment models with possibly nonsmooth moments this paper studies nonparametric estimation of conditional moment models in which the residual functions could be nonsmooth with respect to the unknown functions of endogenous variables it is a problem of nonparametric nonlinear instrumental variables iv estimation and a difficult nonlinear ill posed inverse problem with an unknown operator we first propose a penalized sieve minimum distance smd estimator of the unknown functions that are identified via the conditional moment models we then establish its consistency and convergence rate in strong metric allowing for possibly non compact function parameter spaces possibly non compact finite or infinite dimensional sieves with flexible lower semicompact or convex penalty or finite dimensional linear sieves without penalty under relatively low level sufficient conditions and for both mildly and severely ill posed problems we show that the convergence rates for the nonlinear ill posed inverse problems coincide with the known minimax optimal rates for the nonparametric mean iv regression we illustrate the theory by two important applications root n asymptotic normality of the plug in penalized smd estimator of a weighted average derivative of a nonparametric nonlinear iv regression and the convergence rate of a nonparametric additive quantile iv regression we also present a simulation study and an empirical estimation of a system of nonparametric quantile iv engel curves

nonparametric instrumental variables estimation of a quantile regression model we consider nonparametric estimation of a regression function that is identified by requiring a specified quantile of the regression error conditional on an instrumental variable to be zero the resulting estimating equation is a nonlinear integral equation of the first kind which generates an ill posed inverse problem the integral operator and distribution of the instrumental variable are unknown and must be estimated nonparametrically we show that the estimator is mean square consistent derive its rate of convergence in probability and give conditions under which this rate is optimal in a minimax sense the results of monte carlo experiments show that the estimator behaves well in finite samples

a simple lemma on greedy approximation in hilbert space and convergence rates for projection pursuit regression and neural network training a general convergence criterion for certain iterative sequences in hilbert space is presented for an important subclass of these sequences estimates of the rate of convergence are given under very mild assumptions these results establish an 0 1 f4n nonsampling convergence rate for projection pursuit regression and neural network training where n represents the number of ridge functions neurons or coefficients in a greedy basis expansion

high dimensional analysis of semidefinite relaxations for sparse principal components principal component analysis pca is a classical method for dimensionality reduction based on extracting the dominant eigenvectors of the sample covariance matrix however pca is well known to behave poorly in the large p small n setting in which the problem dimension p is comparable to or larger than the sample size n this paper studies pca in this high dimensional regime but under the additional assumption that the maximal eigenvector is sparse say with at most k nonzero components we consider a spiked covariance model in which a base matrix is perturbed by adding a k sparse maximal eigenvector and we analyze two computationally tractable methods for recovering the support set of this maximal eigenvector as follows a a simple diagonal thresholding method which transitions from success to failure as a function of the rescaled sample size dia n p k n k 2 log p k and b a more sophisticated semidefinite programming sdp relaxation which succeeds once the rescaled sample size sdp n p k n klog p k is larger than a critical threshold in addition we prove that no method including the best method which has exponential time complexity can succeed in recovering the support if the order parameter sdp n p k is below a threshold our results thus highlight an interesting trade off between computational and statistical efficiency in high dimensional inference

density estimation by stochastic complexity the results by p hall and e j hannan 1988 on optimization of histogram density estimators with equal bin widths by minimization of the stochastic complexity are extended and sharpened in two separate ways as the first contribution two generalized histogram estimators are constructed the first has unequal bin widths which together with the number of the bins are determined by minimization of the stochastic complexity using dynamic programming the other estimator consists of a mixture of equal bin width estimators each of which is defined by the associated stochastic complexity as the main contribution in the present work two theorems are proved which together extend the universal coding theorems to a large class of data generating densities the first gives an asymptotic upper bound for the code redundancy in the order of magnitude achieved with a special predictive type of histogram estimator which sharpens a related bound the second theorem states that this bound cannot be improved upon by any code whatsoever

approximation dans les espaces metriques et theorie de l estimation we investigate the relations between the speed of estimation and the metric structure of the parameter space especially in the case when its metric dimension is infinite given some distance d on generally hellinger distance in the case of n i i d variables we consider the minimax risk for n observations r n q mathop inf sup limits t n theta in theta text ie theta d q theta t n t n being any estimate of we shall look for functions r such that for positive constants c 1 q and c 2 q c 1 r q n r n q c2 r q n r n is the speed of estimation and we shall show under fairly general conditions including i i d variables and regular cases of markov chains and stationnary gaussian processes that r n is determined up to multiplicative constants by the metric structure of we shall also give a construction for some sort of universal estimates the risk of which is bounded by c 2 r q n in all cases where the preceding theory applies

efficient estimation of semiparametric conditional moment models with possibly nonsmooth residuals this paper considers semiparametric efficient estimation ofconditional moment models with possibly nonsmooth residuals in unknown parametric components and unknown functions h of endogenous variables we show that 1 the penalized sieve minimum distance psmd estima tor can simultaneously achieve root n asymptotic normality ofand nonparametric optimal convergence rate of allowing for noncompact function parameter spaces 2 a simple weighted bootstrap procedure consistently estimates the limiting distribution of the psmd 3 the semi parametric efficiency bound formula of ai and chen 2003 remains valid for conditional models with nonsmooth residuals and the optimally weighted psmd estimator achieves the bound 4 the centered profiled optimally weighted psmd criterion is asymptotically chi square distributed we illustrate our theories using a partially linear quantile instrumental variables iv regression a monte carlo study and an empirical estimation of the shape invariant quantile iv engel curves

convergence rates of general regularization methods for statistical inverse problems and applications previously the convergence analysis for linear statistical inverse problems has mainly focused on spectral cut off and tikhonov type estimators spectral cut off estimators achieve minimax rates for a broad range of smoothness classes and operators but their practical usefulness is limited by the fact that they require a complete spectral decomposition of the operator tikhonov estimators are simpler to compute but still involve the inversion of an operator and achieve minimax rates only in restricted smoothness classes in this paper we introduce a unifying technique to study the mean square error of a large class of regularization methods spectral methods including the aforementioned estimators as well as many iterative methods such as nu methods and the landweber iteration the latter estimators converge at the same rate as spectral cut off but require only matrix vector products our results are applied to various problems in particular we obtain precise convergence rates for satellite gradiometry l 2 boosting and errors in variable problems

estimation of integral functionals of a density let be a smooth function of k 2 variables we shall investigate in this paper the rates of convergence of estimators of t f f f x f x f k x x dx when f belongs to some class of densities of smoothness s we prove that when 2k 1 4 one can define an estimator t n of t f based on n i i d observations of density f on the real line which converges at the semiparametric rate 1 n on the other hand when s 2k 1 4 t f cannot be estimated at a rate faster than n with 4 s k 4s 1 we shall also provide some extensions to the multidimensional case those results extend previous works of levit of bickel and ritov and of donoho and nussbaum on estimation of quadratic functionals

universal coding information prediction and estimation a connection between universal codes and the problems of prediction and statistical estimation is established a known lower bound for the mean length of universal codes is sharpened and generalized and optimum universal codes constructed the bound is defined to give the information in strings relative to the considered class of processes the earlier derived minimum description length criterion for estimation of parameters including their number is given a fundamental information theoretic justification by showing that its estimators achieve the information in the strings it is also shown that one cannot do prediction in gaussian autoregressive moving average arma processes below a bound which is determined by the information in the data

convergence rate of sieve estimates in this paper we develop a general theory for the convergence rate of sieve estimates maximum likelihood estimates mle s and related estimates obtained by optimizing certain empirical criteria in general parameter spaces in many cases especially when the parameter space is infinite dimensional maximization over the whole parameter space is undesirable in such cases one has to perform maximization over an approximating space sieve of the original parameter space and allow the size of the approximating space to grow as the sample size increases this method is called the method of sieves in the case of the maximum likelihood estimation an mle based on a sieve is called a sieve mle we found that the convergence rate of a sieve estimate is governed by a the local expected values variances and l 2 entropy of the criterion differences and b the approximation error of the sieve a robust nonparametric regression problem a mixture problem and a nonparametric regression problem are discussed as illustrations of the theory we also found that when the underlying space is too large the estimate based on optimizing over the whole parameter space may not achieve the best possible rates of convergence whereas the sieve estimate typically does not suffer from this difficulty

minimax nonparametric classification i rates of convergence this paper studies minimax aspects of nonparametric classification we first study minimax estimation of the conditional probability of a class label given the feature variable this function say f is assumed to be in a general nonparametric class we show the minimax rate of convergence under square l sub 2 loss is determined by the massiveness of the class as measured by metric entropy the second part of the paper studies minimax classification the loss of interest is the difference between the probability of misclassification of a classifier and that of the bayes decision as is well known an upper bound on risk for estimating f gives an upper bound on the risk for classification but the rate is known to be suboptimal for the class of monotone functions this suggests that one does not have to estimate f well in order to classify well however we show that the two problems are in fact of the same difficulty in terms of rates of convergence under a sufficient condition which is satisfied by many function classes including besov sobolev lipschitz and bounded variation this is somewhat surprising in view of a result of devroye gorfi and lugosi see a probabilistic theory of pattern recognition new york springer verlag 1996

lower bounds for the minimax risk using f divergences and applications lower bounds involving f divergences between the underlying probability measures are proved for the minimax risk in estimation problems our proofs just use simple convexity facts special cases and straightforward corollaries of our bounds include well known inequalities for establishing minimax lower bounds such as fano s inequality pinsker s inequality and inequalities based on global entropy conditions two applications are provided a new minimax lower bound for the reconstruction of convex bodies from noisy support function measurements and a different proof of a recent minimax lower bound for the estimation of a covariance matrix

spectrally precoded ofdm by bilinearly transforming the rectangularly pulsed multicarrier basis signals new basis sets containing full response continuous basis signals are developed for constructing spectrally efficient orthogonal frequency division multiplexing ofdm signals with or without cyclic prefix cp or zero padding zp both multiple product and linear combination formats are presented for these basis sets in the multiple product format the developed full response basis signals are shown to be zero valued at interval edges and thus can be used to construct continuous phase ofdm signals it is analytically shown that the constructed continuous phase ofdm signals exhibit relatively small power spectral sidelobes which fall off as f 4 and can thus provide much higher spectral efficiency than the traditional rectangularly pulsed ofdm signal in the linear combination format the proposed basis signals are shown to be the linear combinations of rectangularly pulsed multicarrier basis signals and thereby allow for the efficient realization of fast fourier transform fft specifically a spectrally precoded signaling structure is developed in conjunction with inverse fft and guard insertion to construct continuous phase ofdm signals with or without cp or zp the corresponding spectral decoders are also developed to enable zero forcing zf demodulation for spectrally precoded ofdm with the zero forcing demodulation the spectrally precoded ofdm schemes are shown to outperform in average error rate characteristics the conventional rectangularly pulsed and correlatively coded ofdm schemes on various delay spread channels

tikhonov regularization for nonparametric instrumental variable estimators abstract we study a tikhonov regularized tir estimator of a functional parameter identified by conditional moment restrictions in a linear model with both exogenous and endogenous regressors the nonparametric instrumental variable estimator is based on a minimum distance principle with penalization by the norms of the parameter and its derivatives after showing its consistency in the sobolev norm and uniform consistency under an embedding condition we derive the expression of the asymptotic mean integrated square error and the rate of convergence the optimal value of the regularization parameter is characterized in two examples we illustrate our theoretical findings and the small sample properties with simulation results finally we provide an empirical application to estimation of an engel curve and discuss a data driven selection procedure for the regularization parameter

rates of convergence of minimum distance estimators and kolmogorov s entropy on construit des estimateurs robustes et on montre que le taux de convergence resultant depend naturellement d une fonction entropie

risk bounds for model selection via penalization performance bounds for criteria for model selection are devel oped using recent theory for sieves the model selection criteria are based on an empirical loss or contrast function with an added penalty term moti vated by empirical process theory and roughly proportional to the number of parameters needed to describe the model divided by the number of ob servations most of our examples involve density or regression estimation settings and we focus on the problem of estimating the unknown density or regression function we show that the quadratic risk of the minimum penal ized empirical contrast estimatoris bounded by an index of the accuracy of the sieve this accuracy index quanties the trade off among the candidate models between the approximation error and parameter dimension relative to sample size if we choose a list of models which exhibit good approximation prop erties with respect to different classes of smoothness the estimator can be simultaneously minimax rate optimal in each of those classes this is what is usually called adaptation the type of classes of smoothness in which one gets adaptation depends heavily on the list of models if too many models are involved in order to get accurate approximation of many wide classes of functions simultaneously it may happen that the estimator is only approx

estimation of possibly misspecified semiparametric conditional moment restriction models with different conditioning variables abstract newey and powell 2003 instrumental variable estimation of nonparametric models econometrica 71 1565 1578 and ai and chen 2003 efficient estimation of conditional moment restrictions models containing unknown functions econometrica 71 1795 1843 propose sieve minimum distance smd estimation of both finite dimensional parameter and infinite dimensional parameter h that are identified through a conditional moment restriction model in which h could depend on endogenous variables this paper modifies their smd procedure to allow for different conditioning variables to be used in different equations and derives the asymptotic properties when the model may be misspecified under low level sufficient conditions we show that i the modified smd estimators of both and h converge to some pseudo true values in probability ii the smd estimators of smooth functionals including the estimator and the average derivative estimator are asymptotically normally distributed and iii the estimators for the asymptotic covariances of the smd estimators of smooth functionals are consistent and easy to compute these results allow for asymptotically valid tests of various hypotheses on the smooth functionals regardless of whether the semiparametric model is correctly specified or not

a galerkin symmetric and direct bie method for kirchhoff elastic plates formulation and implementation a variational boundary element formulation is proposed for the solution of the elastic kirchhoff plate bending problem the stationarity conditions of an augmented potential energy functional are first discussed after addressing the topic of the choice of the test functions a regularization process based on integrations by parts is developed which allows to express the formulation in terms of double integrals the inner being at most weakly singular and the outer regular standard integration procedures may then be applied for their numerical evaluation in the presence of both straight and curved boundaries the normal slope and the vertical displacement must be c0 and c1 continuous respectively numerical examples show through comparisons with analytical solutions that a high accuracy is achieved

probability inequalities for likelihood ratios and convergence rates of sieve mles let y 1 y n be independent identically distributed with density p 0 and let f be a space of densities we show that the supremum of the likelihood ratios i 1 n p y i p 0 y i where the supremum is over p f with p l 2 p 0 1 2 2 e is exponentially small with probability exponentially close to 1 the exponent is proportional to ne 2 the only condition required for this to hold is that e exceeds a value determined by the bracketing hellinger entropy of f a similar inequality also holds if we replace f by f n and p 0 by q n where q n is an approximation to p 0 in a suitable sense these results are applied to establish rates of convergence of sieve mles furthermore weak conditions are given under which the optimal rate e n defined by h e n f ne n 2 where h f is the hellinger entropy of f is nearly achievable by sieve estimators

universal donsker classes and metric entropy when mathfrak f is a universal donsker class then for independent indetically distributed i i d observation mathbf x 1 ldots mathbf x n with an unknown law p for any mathfrak f i in mathfrak f i 1 ldots m quad n 1 2 left mathfrak f 1 left mathbf x 1 right ldots mathfrak f i left mathbf x n right right 1 leq i leq m is asymptotically normal with mean vector n 1 2 left int mathfrak f i left mathbf x n right d mathbf p left x right right 1 leq i leq m and covariance matrix int mathfrak f i mathfrak f j d mathbf p int mathfrak f id mathbf p int mathfrak f jd mathbf p uniformly for mathfrak f i in mathfrak f then for certain statistics formed frome the mathfrak f i left mathbf x k right even where mathfrak f i may be chosen depending on the mathbf x k there will be asymptotic distribution as n rightarrow infty for example for mathbf x 2 statistics where f i are indicators of disjoint intervals depending suitably on mathbf x 1 ldots mathbf x n whose union is the real line mathbf x 2 quadratic forms have limiting distributions roy 1956 and watson 1958 which may however not be mathbf x 2 distributions and may depend on p chernoff and lehmann 1954 universal donsker classes of sets are up to mild measurability conditions just classes satisfying the vapnik cervonenkis comdinatorial conditions defined later in this section donsker the vapnik cervonenkis combinatorial conditions defined later in this section durst and dudley 1981 and dudley 1984 chapter 11 the use of such classes allows a variety of extensions of the roy watson results to general multidimensional sample spaces pollard 1979 and moore and subblebine 1981 vapnik and cervonenkis 1974 indicated application of their families of sets to classification pattern recognition problems more recently the classes have been applied to tree structured classifiacation breiman friedman olshen and stone 1984 chapter 12

ill posed estimation in high dimensional models with instrumental variables abstract this paper is concerned with inference about low dimensional components of a high dimensional parameter vector 0 which is identified through instrumental variables we allow for eigenvalues of the expected outer product of included and excluded covariates denoted by m to shrink to zero as the sample size increases we propose a novel estimator based on desparsification of an instrumental variable lasso estimator which is a regularized version of 2sls with an additional correction term this estimator converges to 0 at a rate depending on the mapping properties of m linear combinations of our estimator of 0 are shown to be asymptotically normally distributed based on consistent covariance estimation our method allows for constructing confidence intervals and statistical tests for single or low dimensional components of 0 in monte carlo simulations we analyze the finite sample behavior of our estimator we apply our method to estimate a logit model of demand for automobiles using real market share data

some incomplete but boundedly complete location families a general result concerning noncompleteness of location families of probability measures on euclidean space is pointed out examples include boundedly complete families such as those generated by certain scale mixtures of the standard gaussian distribution these examples illuminate completeness criteria for location families and compare favourably in simplicity with previously known examples of incomplete boundedly complete nonlocation families

oracle inequalities for inverse problems we consider a sequence space model of statistical linear inverse problems where we need to estimate a function f from indirect noisy observations let a finite set a of linear estimators be given our aim is to mimic the estimator in a that has the smallest risk on the true f under general conditions we show that this can be achieved by simple minimization of an unbiased risk estimator provided the singular values of the operator of the inverse problem decrease as a power law the main result is a nonasymptotic oracle inequality that is shown to be asymptotically exact this inequality can also be used to obtain sharp minimax adaptive results in particular we apply it to show that minimax adaptation on ellipsoids in the multivariate anisotropic case is realized by minimization of unbiased risk estimator without any loss of efficiency with respect to optimal nonadaptive procedures

approximation of density functions by sequences of exponential families 1 introduction consider the estimation of a probability density func tion p x defined on a bounded interval we approximate the logarithm of the density by a basis function expansion consisting of polynomials splines or trigonometric series the expansion yields a regular exponential family within which we estimate the density by the method of maximum likelihood this method of density estimation arises by application of the principle of maxi mum entropy or minimum relative entropy subject to empirical constraints we show that if the logarithm of the density has r square integrable deriva tives f idr log p12 ox then the sequence of density estimators p converges to p in the sense of relative entropy kullback leibler fp log p pn at rate opr 1 m2r m n as m x c and m2 n 0 in the spline and trigonometric cases and m3 n o 0 in the polynomial case where m is the dimension of the family and n is the sample size boundary conditions are assumed for the density in the trigonometric case this convergence rate specializes to op r n 2r 2r 1 by setting m nl 2r 1 when the log density is known to have degree of smoothness at least r analogous convergence results for the relative entropy are shown to hold in general for any class of log density functions and sequence of finite dimensional linear spaces having l2 and l approximation properties the approximation of log densities using polynomials has previously been considered by neyman 1937 to define alternatives for goodness of fit tests by good 1963 as an application of the method of maximum entropy or minimum relative entropy by crain 1974 1976a b 1977 who demonstrates existence and consistency of the maximum likelihood estimator and by mead and

approximation of least squares regression on nested subspaces on presente une methode generale pour approximer le biais et la variance dans une echelle de normes hilbertiennes

information theory and superefficiency the asymptotic risk of efficient estimators with kullback leibler loss in smoothly parametrized statistical models is k 2n where k is the parameter dimension and n is the sample size under fairly general conditions we given a simple information theoretic proof that the set of parameter values where any arbitrary estimator is superefficient is negligible the proof is based on a result of rissanen that codes have asymptotic redundancy not smaller than k 2 log n except in a set of measure 0

information theoretic upper and lower bounds for statistical estimation in this paper we establish upper and lower bounds for some statistical estimation problems through concise information theoretic arguments our upper bound analysis is based on a simple yet general inequality which we call the information exponential inequality we show that this inequality naturally leads to a general randomized estimation method for which performance upper bounds can be obtained the lower bounds applicable for all statistical estimators are obtained by original applications of some well known information theoretic inequalities and approximately match the obtained upper bounds for various important problems moreover our framework can be regarded as a natural generalization of the standard minimax framework in that we allow the performance of the estimator to vary for different possible underlying distributions according to a predefined prior

optimal rates for zero order convex optimization the power of two function evaluations we consider derivative free algorithms for stochastic and nonstochastic convex optimization problems that use only function values rather than gradients focusing on nonasymptotic bounds on convergence rates we show that if pairs of function values are available algorithms for d dimensional optimization that use gradient estimates based on random perturbations suffer a factor of at most sqrt d in convergence rate over traditional stochastic gradient methods we establish such results for both smooth and nonsmooth cases sharpening previous analyses that suggested a worse dimension dependence and extend our results to the case of multiple m ge 2 evaluations we complement our algorithmic development with information theoretic lower bounds on the minimax convergence rate of such problems establishing the sharpness of our achievable results up to constant sometimes logarithmic factors

semiparametric estimation of models with conditional moment restrictions in the presence of nonclassical measurement errors abstract this paper develops a framework for the analysis of semiparametric conditional moment models with endogenous and mismeasured causes which is of empirical importance we show that one set of valid instruments is sufficient to control for both endogeneity and measurement errors of the causes of interest which has been observed in linear parametric models two step consistent estimators of the parameters of interest are proposed we also show that the proposed estimators are consistent with a rate faster than n 1 4 under a certain metric and the proposed estimators of the finite dimensional unknown parameters obtain root n asymptotic normality monte carlo evidences show that the proposed estimators perform well under a variety of identification conditions an application to instrumental variables estimation of engel curves illustrates the usefulness of our method it supports that correcting for both endogeneity and measurement errors on total expenditure is substantial in estimating economically meaningful engel curves

minimum complexity density estimation the authors introduce an index of resolvability that is proved to bound the rate of convergence of minimum complexity density estimators as well as the information theoretic redundancy of the corresponding total description length the results on the index of resolvability demonstrate the statistical effectiveness of the minimum description length principle as a method of inference the minimum complexity estimator converges to true density nearly as fast as an estimator based on prior knowledge of the true subclass of densities interpretations and basic properties of minimum complexity estimators are discussed some regression and classification problems that can be examined from the minimum description length framework are considered

mutual information metric entropy and cumulative relative entropy risk assume p theta theta epsilon theta is a set of probability distributions with a common dominating measure on a complete separable metric space y a state theta epsilon theta is chosen by nature a statistician obtains n independent observations y 1 dots y n from y distributed according to p theta for each time t between 1 and n based on the observations y 1 dots y t 1 the statistician produces an estimated distribution hat p t for p theta and suffers a loss l p theta hat p t the cumulative risk for the statistician is the average total loss up to time n of special interest in information theory data compression mathematical finance computational learning theory and statistical mechanics is the special case when the loss l p theta hat p t is the relative entropy between the true distribution p theta and the estimated distribution hat p t here the cumulative bayes risk from time 1 to n is the mutual information between the random parameter theta and the observations y 1 dots y n new bounds on this mutual information are given in terms of the laplace transform of the hellinger distance between pairs of distributions indexed by parameters in theta from these bounds on the cumulative minimax risk are given in terms of the metric entropy of theta with respect to the hellinger distance the assumptions required for these bounds are very general and do not depend on the choice of the dominating measure they apply to both finite and infinite dimensional theta they apply in some cases where y is infinite dimensional in some cases where y is not compact in some cases where the distributions are not smooth and in some parametric cases where asymptotic normality of the posterior distribution fails

time limited orthogonal multicarrier modulation schemes time limited orthogonal tlo multicarrier modulation mcm schemes with waveform shaping are investigated the waveform shaped scheme with no frame overlap is spectrally inefficient an improved scheme is proposed and it is shown that this scheme allows a certain amount of overlap between adjacent frames without destroying the orthogonality the scheme is therefore spectrally efficient the improved tlo mcm scheme can be regarded as a time domain dual form of chang s band limited orthogonal mcm 1966

nonparametric instrumental regression the focus of this paper is the nonparametric estimation of an instrumental regression function f defined by conditional moment restrictions that stem from a structural econometric model e y f z w 0 and involve endogenous variables y and z and instruments w the function f is the solution of an ill posed inverse problem and we propose an estimation procedure based on tikhonov regularization the paper analyzes identification and overidentification of this model and presents asymptotic properties of the estimated nonparametric instrumental regression function

identification in nonseparable models weak nonparametric restrictions are developed sufficient to identify the values of derivatives of structural functions in which latent random variables are nonseparable these derivatives can exhibit stochastic variation in a microeconometric context this allows the impact of a policy intervention as measured by the value of a structural derivative to vary across people who are identical as measured by covariates when the restrictions are satisfied quantiles of the distribution of a policy impact across people can be identified the identification restrictions are local in the sense that they are specific to the values of the covariates and the specific quantiles of latent variables at which identification is sought the conditions do not include the commonly required independence of latent variables and covariates they include local versions of the classical rank and order conditions and local quantile insensitivity conditions values of structural derivatives are identified by functionals of quantile regression functions and can be estimated using the same functionals applied to estimated quantile regression functions

inference in nonparametric instrumental variables with partial identification this paper develops methods for hypothesis testing in a nonparametric instrumental variables setting within a partial identification framework we construct and derive the asymptotic distribution of a test statistic for the hypothesis that at least one element of the identified set satisfies a conjectured restriction the same test statistic can be employed under identification in which case the hypothesis is whether the true model satisfies the posited property an almost sure consistent bootstrap procedure is provided for obtaining critical values possible applications include testing for semiparametric specifications as well as building confidence regions for certain functionals on the identified set as an illustration we obtain confidence intervals for the level and slope of brazilian fuel engel curves a monte carlo study examines finite sample performance

sparsity and error analysis of empirical feature based regularization schemes we consider a learning algorithm generated by a regularization scheme with a concave regularizer for the purpose of achieving sparsity and good learning rates in a least squares regression setting the regularization is induced for linear combinations of empirical features constructed in the literatures of kernel principal component analysis and kernel projection machines based on kernels and samples in addition to the separability of the involved optimization problem caused by the empirical features we carry out sparsity and error analysis giving bounds in the norm of the reproducing kernel hilbert space based on a priori conditions which do not require assumptions on sparsity in terms of any basis or system in particular we show that as the concave exponent q of the concave regularizer increases to 1 the learning ability of the algorithm improves some numerical simulations for both artificial and real mhc peptide binding data involving the lq regularizer and the scad penalty are presented to demonstrate the sparsity and error analysis

rate optimal estimation for a general class of nonparametric regression models with unknown link functions this paper discusses a nonparametric regression model that naturally generalizes neural network models the model is based on a finite number of one dimensional transformations and can be estimated with a one dimensional rate of convergence the model contains the generalized additive model with unknown link function as a special case for this case it is shown that the additive components and link function can be estimated with the optimal rate by a smoothing spline that is the solution of a penalized least squares criterion

boundary element analysis of a plate on elastic foundations the objective of this study is to examine the applicability of the boundary element method to analysing a plate on elastic foundation the fundamental solution of the problem is presented as a fourier bessel integral for the computation of the values of the fundamental solution an algorithm was developed in which the fourier bessel integral was decomposed into an alternative convergent sequence equations based on the direct and indirect boundary element method were derived for a plate situated on a one or two parametric elastic foundation according to the theory presented computer programs based on the direct and indirect boundary element method were developed these programs can be used for examining the behaviour of a smooth boundary plate on a one or two parametric elastic foundation the computer programs were tested by several examples the results obtained by using a small number of boundary elements compared favourably to the results obtained by a fine finite element mesh the study shows that the boundary element method is applicable to the analysis of a plate on elastic foundation

an iv model of quantile treatment effects the ability of quantile regression models to characterize the heterogeneous impact of variables on different points of an outcome distribution makes them appealing in many economic applications however in observational studies the variables of interest e g education prices are often endogenous making conventional quantile regression inconsistent and hence inappropriate for recovering the causal effects of these variables on the quantiles of economic outcomes in order to address this problem we develop a model of quantile treatment effects qte in the presence of endogeneity and obtain conditions for identification of the qte without functional form assumptions the principal feature of the model is the imposition of conditions that restrict the evolution of ranks across treatment states this feature allows us to overcome the endogeneity problem and recover the true qte through the use of instrumental variables the proposed model can also be equivalently viewed as a structural simultaneous equation model with nonadditive errors where qte can be interpreted as the structural quantile effects sqe

semi nonparametric iv estimation of shape invariant engel curves this paper studies a shape invariant engel curve system with endogenous total expenditure in which the shape invariant specification involves a common shift parameter for each demographic group in a pooled system of nonparametric engel curves we focus on the identification and estimation of both the nonparametric shapes of the engel curves and the parametric specification of the demographic scaling parameters the identification condition relates to the bounded completeness and the estimation procedure applies the sieve minimum distance estimation of conditional moment restrictions allowing for endogeneity we establish a new root mean squared convergence rate for the nonparametric instrumental variable regression when the endogenous regressor could have unbounded support root n asymptotic normality and semiparametric efficiency of the parametric components are also given under a set of low level sufficient conditions our empirical application using the u k family expenditure survey shows the importance of adjusting for endogeneity in terms of both the nonparametric curvatures and the demographic parameters of systems of engel curves

a general minimax result for relative entropy suppose nature picks a probability measure p sub spl theta on a complete separable metric space x at random from a measurable set p sub spl theta p spl theta spl theta spl isin spl theta then without knowing spl theta a statistician picks a measure q on s finally the statistician suffers a loss d p sub 0 spl par q the relative entropy between p sub spl theta and q we show that the minimax and maximin values of this game are always equal and there is always a minimax strategy in the closure of the set of all bayes strategies this generalizes previous results of gallager 1979 and davisson and leon garcia 1980

specification testing in nonparametric instrumental variable estimation abstract in nonparametric instrumental variable estimation the function being estimated is the solution to an integral equation a solution may not exist if for example the instrument is not valid this paper discusses the problem of testing the null hypothesis that a solution exists against the alternative that there is no solution we give necessary and sufficient conditions for existence of a solution and show that uniformly consistent testing of an unrestricted null hypothesis is not possible uniformly consistent testing is possible however if the null hypothesis is restricted by assuming that any solution to the integral equation is smooth many functions of interest in applied econometrics including demand functions and engel curves are expected to be smooth the paper presents a statistic for testing the null hypothesis that a smooth solution exists the test is consistent uniformly over a large class of probability distributions of the observable random variables for which the integral equation has no smooth solution the finite sample performance of the test is illustrated through monte carlo experiments

an asymptotic property of model selection criteria probability models are estimated by use of penalized log likelihood criteria related to akaike 1973 information criterion aic and minimum description length mdl the accuracies of the density estimators are shown to be related to the tradeoff between three terms the accuracy of approximation the model dimension and the descriptive complexity of the model classes the asymptotic risk is determined under conditions on the penalty term and is shown to be minimax optimal for some cases as an application we show that the optimal rate of convergence is simultaneously achieved for log densities in sobolev spaces w sub 2 sup s u without knowing the smoothness parameter s and norm parameter u in advance applications to neural network models and sparse density function estimation are also provided

nonlinear estimation for linear inverse problems with error in the operator we study two nonlinear methods for statistical linear inverse problems when the operator is not known the two constructions combine galerkin regularization and wavelet thresholding their performances depend on the underlying structure of the operator quantified by an index of sparsity we prove their rate optimality and adaptivity properties over besov classes

geometrizing rates of convergence ii consider estimating a functional t f of an unknown distribution f in mathbf f from data x 1 cdots x n i i d f let omega varepsilon denote the modulus of continuity of the functional t over mathbf f computed with respect to hellinger distance for well behaved loss functions l t we show that inf t n sup mathbf f e fl t n t f is equivalent to l omega n 1 2 to within constants whenever t is linear and mathbf f is convex the same conclusion holds in three nonlinear cases estimating the rate of decay of a density estimating the mode and robust nonparametric regression we study the difficulty of testing between the composite infinite dimensional hypotheses h 0 t f leq t and h 1 t f geq t delta our results hold in the cases studied because the difficulty of the full infinite dimensional composite testing problem is comparable to the difficulty of the hardest simple two point testing subproblem

a lower bound on the error in nonparametric regression type problems let x1 y1 xn yn be a sample denote the conditional density of ylxi xi as f ylxi 0 xi and 0 an element of a metric space 9 d a lower bound is provided for the d error in estimating 0 the order of the bound depends on the local behavior of the kullback information of the conditional density as an application we consider the case where 1 is the space of q smooth functions on 0 i d metrized with the lr distance 1 r oo

adaptive wavelet galerkin methods for linear inverse problems we introduce and analyze numerical methods for the treatment of inverse problems based on an adaptive wavelet galerkin discretization these methods combine the theoretical advantages of the wavelet vaguelette decomposition wvd in terms of optimally adapting to the unknown smoothness of the solution together with the numerical simplicity of galerkin methods in a first step we simply combine a thresholding algorithm on the data with a galerkin inversion on a fixed linear space in a second step a more elaborate method performs the inversion by an adaptive procedure in which a smaller space adapted to the solution is iteratively constructed this leads to a significant reduction of the computational cost

a bem based meshless method for plates on biparametric elastic foundation with internal supports abstract in this paper a bem based meshless method is developed for the analysis of plates on a biparametric elastic foundation which in addition to the boundary supports are also supported inside the domain on isolated points a group of plies and or line supports continuous plates the presented method is achieved using the concept of the analog equation method aem of katsikadelis according to this method the original governing differential equation is replaced by an equivalent problem for plates without internal supports not resting on an elastic foundation subjected to an appropriate fictitious load in addition to the transverse external loads under the same boundary conditions the fictitious load is established using a technique based on bem and approximated by radial basis functions series the solution of the actual problem is obtained from the known integral representation of the solution for the classical plate bending problem which is derived using the fundamental solution of the biharmonic equation thus the kernels of the boundary integral equations are conveniently established and evaluated the presented method has all the advantages of the pure bem to validate the effectiveness accuracy as well as the applicability of the proposed method numerical results of various example problems are presented

information theoretic asymptotics of bayes methods in the absence of knowledge of the true density function bayesian models take the joint density function for a sequence of n random variables to be an average of densities with respect to a prior the authors examine the relative entropy distance d sub n between the true density and the bayesian density and show that the asymptotic distance is d 2 log n c where d is the dimension of the parameter vector therefore the relative entropy rate d sub n n converges to zero at rate log n n the constant c which the authors explicitly identify depends only on the prior density function and the fisher information matrix evaluated at the true parameter value consequences are given for density estimation universal data compression composite hypothesis testing and stock market portfolio selection