maximum likelihood estimation of functionals of discrete distributions we consider the problem of estimating functionals of discrete distributions and focus on a tight up to universal multiplicative constants for each specific functional nonasymptotic analysis of the worst case squared error risk of widely used estimators we apply concentration inequalities to analyze the random fluctuation of these estimators around their expectations and the theory of approximation using positive linear operators to analyze the deviation of their expectations from the true functional namely their bias we explicitly characterize the worst case squared error risk incurred by the maximum likelihood estimator mle in estimating the shannon entropy h p sum i 1 s p i ln p i and the power sum f alpha p sum i 1 s p i alpha alpha 0 up to universal multiplicative constants for each fixed functional for any alphabet size s leq infty and sample size n for which the risk may vanish as a corollary for shannon entropy estimation we show that it is necessary and sufficient to have n gg s observations for the mle to be consistent in addition we establish that it is necessary and sufficient to consider n gg s 1 alpha samples for the mle to consistently estimate f alpha p 0 the minimax rate optimal estimators for both problems require s ln s and s 1 alpha ln s samples which implies that the mle has a strictly sub optimal sample complexity when 1 we show that the worst case squared error rate of convergence for the mle is n 2 alpha 1 for infinite alphabet size while the minimax squared error rate is n ln n 2 alpha 1 when alpha geq 3 2 the mle achieves the minimax optimal rate n 1 regardless of the alphabet size as an application of the general theory we analyze the dirichlet prior smoothing techniques for shannon entropy estimation in this context one approach is to plug in the dirichlet prior smoothed distribution into the entropy functional while the other one is to calculate the bayes estimator for entropy under the dirichlet prior for squared error which is the conditional expectation we show that in general such estimators do not improve over the maximum likelihood estimator no matter how we tune the parameters in the dirichlet prior this approach cannot achieve the minimax rates in entropy estimation the performance of the minimax rate optimal estimator with n samples is essentially at least as good as that of dirichlet smoothed entropy estimators with n ln n samples

minimax estimation of linear functionals over nonconvex parameter spaces the minimax theory for estimating linear functionals is extended to the case of a finite union of convex parameter spaces upper and lower bounds for the minimax risk can still be described in terms of a modulus of continuity however in contrast to the theory for convex parameter spaces rate optimal procedures are often required to be nonlinear a construction of such nonlinear procedures is given the results developed in this paper have important applications to the theory of adaptation

on efficient implicit obdd based algorithms for maximal matchings the maximal matching problem i e the computation of a matching that is not a proper subset of another matching is a fundamental optimization problem and maximal matching algorithms have been used as submodules for problems like maximal node disjoint paths or maximum flow since in some applications very large graphs have to be processed a research branch has emerged which is concerned with the design and analysis of implicit algorithms for classical graph problems input graphs are given as characteristic boolean functions of their edge sets and problems have to be solved by functional operations as obdds which are closely related to deterministic finite automata are a well known data structure for boolean functions obdd based algorithms are used as a heuristic approach to handle very large graphs here an implicit obdd based maximal matching algorithm is presented that uses only a polylogarithmic number of functional operations with respect to the number of vertices of the input graph in order to investigate the algorithm s behavior on large and structured networks it is analyzed on grid graphs it is shown that the overall running time and the space requirement is also polylogarithmic furthermore we present another algorithm similar to the well known karp sipser approach and we investigate the representation size of maximal matchings

on the size of binary decision diagrams representing boolean functions we consider the size of the representation of boolean functions by several classes of binary decision diagrams bdds also called branching programs namely the classes of arbitrary bdds of real time bdd rbdd i e bdds where each computation path is limited to the number of variables of free bdds fbdds also called read once only branching programs of ordered bdds obdds i e fbdds where variables are tested in the same order along all paths and binary decision trees bdts using well known techniques we first establish asymptotically sharp bounds as a function of n on the minimum size of arbitrary bdds representing almost all boolean functions of n variables and provide asymptotic lower and upper bounds differing only by a factor of two on the minimum size obdds representing almost all boolean functions of n variables we then using a method to obtain exponential lower bounds on complexity of computation of boolean functions by rbdd fbdd and obdd that originated in breitbart 1968 present the highest such bounds to date and also present improved bounds on the relative economy of description of particular boolean functions by the above classes of bdds for each nontrivial pair of bdd classes considered we exhibit infinite families of boolean functions representable much more concisely by bdds in one class than by bdds in the other

efficient estimation of integral functionals of a density bickel and ritov s estimator is a quite intricate expression based on kernel estimators of the density in this paper we propose an alternative and somehow simpler method of estimation based on orthogonal projections this

a note on nonparametric estimation of linear functionals precise asymptotic descriptions of the minimax affine risks and bias variance tradeoffs for estimating linear functionals are given for a broad class of moduli the results are complemented by illustrative examples including one where it is possible to construct an estimator which is fully adaptive over a range of parameter spaces

estimation of integral functionals of a density let be a smooth function of k 2 variables we shall investigate in this paper the rates of convergence of estimators of t f f f x f x f k x x dx when f belongs to some class of densities of smoothness s we prove that when 2k 1 4 one can define an estimator t n of t f based on n i i d observations of density f on the real line which converges at the semiparametric rate 1 n on the other hand when s 2k 1 4 t f cannot be estimated at a rate faster than n with 4 s k 4s 1 we shall also provide some extensions to the multidimensional case those results extend previous works of levit of bickel and ritov and of donoho and nussbaum on estimation of quadratic functionals

a characterization of positive quadrature formulae a positive quadrature formula with n nodes which is exact for polynomials of degree in r 1 0 r is based on the zeros of certain quasi orthogonal polynomials of degree n we show that the quasi orthogonal polynomials that lead to the positive quadrature formulae can all be expressed as characteristic polynomials of a symmetric tridiagonal matrix with positive subdiagonal entries as a consequence for a fixed n every positive quadrature formula is a gaussian quadrature formula for some nonnegative measure

minimax rates of entropy estimation on large alphabets via best polynomial approximation consider the problem of estimating the shannon entropy of a distribution over k elements from n independent samples we show that the minimax mean square error is within the universal multiplicative constant factors of big k n log k big 2 log 2 k n if n exceeds a constant factor of k log k otherwise there exists no consistent estimator this refines the recent result of valiant and valiant that the minimal sample size for consistent entropy estimation scales according to theta k log k the apparatus of the best polynomial approximation plays a key role in both the construction of optimal estimators and by a duality argument the minimax lower bound

quasi orthogonal polynomials quadrature and interpolation abstract we investigate the quasi orthogonal polynomials by expressing them as characteristic polynomials of symmetric matrices many properties of zeros of quasi orthogonal polynomials are derived from this formulation we derive new formulae for quadrature and interpolation based on the zeros of quasi orthogonal polynomials and prove the mean convergence of lagrange interpolation

nonquadratic estimators of a quadratic functional estimation of a quadratic functional over parameter spaces that are not quadratically convex is considered it is shown in contrast to the theory for quadratically convex parameter spaces that optimal quadratic rules are often rate suboptimal in such cases minimax rate optimal procedures are constructed based on local thresholding these nonquadratic procedures are sometimes fully efficient even when optimal quadratic rules have slow rates of convergence moreover it is shown that when estimating a quadratic functional nonquadratic procedures may exhibit different elbow phenomena than quadratic procedures

testing composite hypotheses hermite polynomials and optimal estimation of a nonsmooth functional a general lower bound is developed for the minimax risk when estimating an arbitrary functional the bound is based on testing two composite hypotheses and is shown to be effective in estimating the nonsmooth functional 1 n i from an observation y n i n this problem exhibits some features that are significantly different from those that occur in estimating conventional smooth functionals this is a setting where standard techniques fail to yield sharp results a sharp minimax lower bound is established by applying the general lower bound technique based on testing two composite hypotheses a key step is the construction of two special priors and bounding the chi square distance between two normal mixtures an estimator is constructed using approximation theory and hermite polynomials and is shown to be asymptotically sharp minimax when the means are bounded by a given value m it is shown that the minimax risk equals 2 m 2 log log n log n 2 asymptotically where is the bernstein constant the general techniques and results developed in the present paper can also be used to solve other related problems

jacobi and laguerre quasi orthogonal approximations and related interpolations in this paper we investigate jacobi quasi orthogonal approximation and generalized jacobi gauss lobatto interpolation we also propose laguerre quasi orthogonal approximation and generalized laguerre gauss radau interpolation a series of sharp results on these approximations are established which are applicable to spectral and pseudospectral methods for mixed nonhomogeneous boundary value problems of high order

minimax risk over hyperrectangles and implications consider estimating the mean of a standard gaussian shift when that mean is known to lie in an orthosymmetric quadratically convex set in l 2 such sets include ellipsoids hyperrectangles and l p bodies with p 2 the minimax risk among linear estimates is within 25 of the minimax risk among all estimates the minimax risk among truncated series estimates is within a factor 4 44 of the minimax risk this implies that the difficulty of estimation a statistical quantity is measured fairly precisely by the n widths a geometric quantity if the set is not quadratically convex as in the case of l p bodies with p 2 things change appreciably minimax linear estimators may be out performed arbitrarily by nonlinear estimates the ordinary kolmogorov n widths still determine the difficulty of linear estimation but the difficulty of nonlinear estimation is tied to the inner bernstein n widths which can be far smaller essential use is made of a new heuristic that the difficulty of the hardest rectangular subproblem is equal to the difficulty of the full problem

quasi orthogonality and differential equations abstract the following theorem is proved quasi orthogonal polynomials corresponding to the semi classical or laguerre hahn families and the orthogonal polynomials themselves verify differential equations of same order reducible to two or four by the hahn theorem

on estimation of the lr norm of a regression function be observed with noise in the present paper we study the problem of nonparametric estimation of certain nonsmooth functionals of f specifically l r norms f r of f known from the literature results on functional estimation deal mostly with two extreme cases estimating a smooth differentiable in l 2 functional or estimating a singular functional like the value of f at certain point or the maximum of f in the first case the convergence rate typically is n 1 2 n being the number of observations in the second case the rate of convergence coincides with the one of estimating the function f itself in the corresponding norm we show that the case of estimating f r is in some sense intermediate between the above extremes the optimal rate of convergence is worse than n 1 2 but is better than the rate of convergence of nonparametric estimates of f the results depend on the value of r for r even integer the rate occurs to be n 2 1 1 r where is the degree of smoothness if r is not an even integer then the nonparametric rate n 2 1 can be improved but only by a logarithmic in n factor

minimax estimation of a functional on a structured high dimensional model we introduce a new method of estimation of parameters in semi parametric and nonparametric models the method is based on estimating equations that are u statistics in the observations the u statistics are based on higher order influence functions that extend ordinary linear influence functions of the parameter of interest and represent higher derivatives of this parameter for parameters for which the representation cannot be perfect the method leads to a bias variance trade off and results in estimators that converge at a slower than n rate in a number of examples the resulting rate can be shown to be optimal we are particularly interested in estimating parameters in models with a nuisance parameter of high dimension or low regularity where the parameter of interest cannot be estimated at n rate but we also consider efficient n estimation using novel nonlinear estimators the general approach is applied in detail to the example of estimating a mean response when the response is not always observed

estimating nonquadratic functionals of a density using haar wavelets consider the problem of estimating f where is a smooth function and f is a density with given order of regularity s special attention is paid to the case t t 3 it has been shown that for low values of s the n 1 2 rate of convergence is not achievable uniformly over the class of objects of regularity s in fact a lower bound for this rate is n 4s 1 4s for 0 s 1 4 as for the upper bound using a taylor expansion it can be seen that it is enough to provide an estimate for the case x x 3 that is the aim of this paper our method makes intensive use of special algebraic and wavelet properties of the haar basis

on the complexity of vlsi implementations and graph representations of boolean functions with application to integer multiplication lower bound results on boolean function complexity under two different models are discussed the first is an abstraction of tradeoffs between chip area and speed in very large scale integrated vlsi circuits the second is the ordered binary decision diagram obdd representation used as a data structure for symbolically representing and manipulating boolean functions the lower bounds demonstrate the fundamental limitations of vlsi as an implementation medium and that of the obdd as a data structure it is shown that the same technique used to prove that any vlsi implementation of a single output boolean function has area time complexity at sup 2 omega n sup 2 also proves that any obdd representation of the function has omega c sup n vertices for some c 1 but that the converse is not true an integer multiplier for word size n with outputs numbered 0 least significant through 2n 1 most significant is described for the boolean function representing either output i 1 or output 2n i 1 where 1

on estimation of l_ r lr norms in gaussian white noise models we provide a complete picture of asymptotically minimax estimation of l r norms for any r ge 1 of the mean in gaussian white noise model over nikolskii besov spaces in this regard we complement the work of lepski et al probab theory relat fields 113 2 221 253 1999 who considered the cases of r 1 with poly logarithmic gap between upper and lower bounds and r even with asymptotically sharp upper and lower bounds over holder spaces we additionally consider the case of asymptotically adaptive minimax estimation and demonstrate a difference between even and non even r in terms of an investigator s ability to produce asymptotically adaptive minimax estimators without paying a penalty

quasi orthogonality and zeros of some 2ϕ2 and 3ϕ2 polynomials abstract we state and prove the q extension of a result due to johnston and jordaan cf johnston and jordaan 2015 and make use of this result the orthogonality of q laguerre little q jacobi q meixner and al salam carlitz i polynomials as well as contiguous relations satisfied by the polynomials to establish the quasi orthogonality of certain 2 2 and 3 2 polynomials the location and interlacing properties of the real zeros of these quasi orthogonal polynomials are studied interlacing properties of the zeros of q laguerre quasi orthogonal polynomials l n z q when 2 1 with those of l n 1 1 z q and l n 1 z q are also considered

algebraic and spectral properties of some quasiorthogonal polynomials encountered in quantum radiation the nodal structure of the wavefunctions of a large class of quantum mechanical potentials is often governed by the distribution of zeros of real quasiorthogonal polynomials it is known that these polynomials i may be described by an arbitrary linear combination of two orthogonal polynomials pn x and ii have real and simple zeros here the three term recurrence relation the second order differential equation and the distribution of zeros of quasiorthogonal polynomials of the classical class i e when pn x is a jacobi laguerre or hermite polynomial are derived and analyzed specifically the exact values of the newton sum rules and the wkb density of zeros of these polynomials are found

randomized obdd based graph algorithms abstract implicit graph algorithms deal with the characteristic function e of the edge set e of a graph g v e encoding the nodes by binary vectors e can be represented by an ordered binary decision diagram obdd which is a well known data structure for boolean functions obdd based graph algorithms solve graph optimization problems by mainly using functional operations and are a heuristic approach to cope with massive graphs these algorithms heavily rely on a compact representation of the underlying boolean functions which is why all previously known obdd based algorithms are deterministic since random functions are not compressible in general here the first randomized obdd based algorithms are presented where random functions with limited independence are used to overcome the large representation size on the theoretical part we investigate the size of obdds representing k wise independent random functions in addition we give a construction of almost k wise independent random functions by means of a random obdd generation and show an almost matching lower bound of the obdd size representing such random functions on the algorithmic part randomization often facilitates the design of simple algorithms which in the context of obdd based algorithms means a small number of functional operations and as few input variables of the used boolean functions as possible we present two randomized implicit algorithms a randomized minimum spanning tree algorithm and a maximal matching algorithm the matching algorithm needs o log 3 v functional operations in expectation and uses functions with at most 3 log v variables which is both better than the best known deterministic algorithms w r t functional operations and variables the algorithm may be of independent interest the experimental evaluation of the algorithms shows that they outperform known obdd based algorithms for the maximal matching problem and the minimum spanning tree problem

on quasi orthogonal polynomials their differential equations discriminants and electrostatics abstract in this paper we develop a general theory of quasi orthogonal polynomials we first derive three term recurrence relation and second order differential equations for quasi orthogonal polynomials we also give an expression for their discriminants in terms of the recursion coefficients of the corresponding orthogonal polynomials in addition we investigate an electrostatic equilibrium problem where the equilibrium position of movable charges is attained at the zeros of the quasi orthogonal polynomials the examples of the freud weight w x e x 4 2 t x 2 and the jacobi weight w x 1 x 1 x are discussed in some detail finally we consider the nonlinear orthogonality preserving transformation and related matrix problem

on the estimation of quadratic functionals we discuss the difficulties of estimating quadratic functionals based on observations y t from the white noise model y t int t 0 f u du sigma w t quad t in lbrack 0 1 rbrack where w t is a standard wiener process on lbrack 0 1 rbrack the optimal rates of convergence as sigma rightarrow 0 for estimating quadratic functionals under certain geometric constraints are found specifically the optimal rates of estimating int 1 0 lbrack f k x rbrack 2 dx under hyperrectangular constraints sum f x j f leq cj alpha and weighted l p body constraints sum p f sum infty 1 j r x j f p leq c are computed explicitly where x j f is the j th fourier bessel coefficient of the unknown function f we develop lower bounds based on testing two highly composite hypercubes and address their advantages the attainable lower bounds are found by applying the hardest one dimensional approach as well as the hypercube method we demonstrate that for estimating regular quadratic functionals i e the functionals which can be estimated at rate o sigma 2 the difficulties of the estimation are captured by the hardest one dimensional subproblems and for estimating nonregular quadratic functionals i e no o sigma 2 consistent estimator exists the difficulties are captured at certain finite dimensional the dimension goes to infinity as sigma rightarrow 0 hypercube subproblems

minimax estimation of functionals of discrete distributions we propose a general methodology for the construction and analysis of essentially minimax estimators for a wide class of functionals of finite dimensional parameters and elaborate on the case of discrete distributions where the support size s is unknown and may be comparable with or even much larger than the number of observations n we treat the respective regions where the functional is nonsmooth and smooth separately in the nonsmooth regime we apply an unbiased estimator for the best polynomial approximation of the functional whereas in the smooth regime we apply a bias corrected version of the maximum likelihood estimator mle we illustrate the merit of this approach by thoroughly analyzing the performance of the resulting schemes for estimating two important information measures 1 the entropy h p sum i 1 s p i ln p i and 2 f alpha p sum i 1 s p i alpha alpha 0 we obtain the minimax l 2 rates for estimating these functionals in particular we demonstrate that our estimator achieves the optimal sample complexity n asymp s ln s for entropy estimation we also demonstrate that the sample complexity for estimating f alpha p 0 is n asymp s 1 alpha ln s which can be achieved by our estimator but not the mle for 1 we show the minimax l 2 rate for estimating f alpha p is n ln n 2 alpha 1 for infinite support size while the maximum l 2 rate for the mle is n 2 alpha 1 for all the above cases the behavior of the minimax rate optimal estimators with n samples is essentially that of the mle plug in rule with n ln n samples which we term effective sample size enlargement we highlight the practical advantages of our schemes for the estimation of entropy and mutual information we compare our performance with various existing approaches and demonstrate that our approach reduces running time and boosts the accuracy moreover we show that the minimax rate optimal mutual information estimator yielded by our framework leads to significant performance boosts over the chow liu algorithm in learning graphical models the wide use of information measure estimation suggests that the insights and estimators obtained in this paper could be broadly applicable