distribution estimation consistent in total variation and in two types of information divergence the problem of the nonparametric estimation of a probability distribution is considered from three viewpoints the consistency in total variation the consistency in information divergence and consistency in reversed order information divergence these types of consistencies are relatively strong criteria of convergence and a probability distribution cannot be consistently estimated in either type of convergence without any restrictions on the class of probability distributions allowed histogram based estimators of distribution are presented which under certain conditions converge in total variation in information divergence and in reversed order information divergence to the unknown probability distribution some a priori information about the true probability distribution is assumed in each case as the concept of consistency in information divergence is stronger than that of convergence in total variation additional assumptions are imposed in the cases of informational divergences

on convergence of information contained in quantized observations by the well known information processing theorem quantizations of observations reduce values of convex information functionals such as the information divergence fisher information or shannon information this paper deals with the convergence of the reduced values of these functionals to their original unreduced values for various sequences p sub n of partitions of the observation space there is extensive literature dealing with this convergence when the partitions are nested in the sense p sub n spl sub p sub n 1 for all n a systematic study for nonnested partitions p sub n often considered in the literature seems to be missing this paper tries to partially fill this gap it proves the convergence for the most common types of partitions the results are formulated for generalized information divergences csiszar 1963 1967 1973 divergences generalized fisher information and generalized shannon information their applicability is illustrated on the barron 1992 estimators of probability distributions

asymptotic normality of the posterior in relative entropy we show that the relative entropy between a posterior density formed from a smooth likelihood and prior and a limiting normal form tends to zero in the independent and identically distributed case the mode of convergence is in probability and in mean applications to code lengths in stochastic complexity and to sample size selection are discussed

on pairs of f divergences and their joint range we compare two f divergences and prove that their joint range is the convex hull of the joint range for distributions supported on only two points the proofs use various methods from topology and convex analysis

optimization of barron density estimates we investigate a nonparametric estimator of the probability density introduced by barron 1988 1989 earlier papers established its consistency in a strong sense e g in the expected information divergence or expected chi square divergence this paper pays main attention to the expected chi square divergence criterion we give a new motivation of the barron estimator by showing that a maximum likelihood estimator mle of a density from a family important in practice is consistent in expected information divergence but not in expected chi square divergence we also present new and practically applicable conditions of consistency in the expected chi square divergence main attention is paid to optimization in the sense of the mentioned criterion of the two objects specifying the barron estimator the dominating probability density and the decomposition of the observation space into finitely many bins both problems are explicitly solved under certain regularity assumptions about the estimated density a simulation study illustrates the results in exponential rayleigh and weibull families

adaptive coding and prediction of sources with large and infinite alphabets the problem of predicting a sequence x1 x2 generated by a discrete source with unknown statistics is considered each letter xt 1 is predicted using the information on the word x1x2 hellip xt only this problem is of great importance for data compression because of its use to estimate probability distributions for ppm algorithms and other adaptive codes on the other hand such prediction is a classical problem which has received much attention its history can be traced back to laplace we address the problem where the sequence is generated by an independent and identically distributed i i d source with some large or even infinite alphabet and suggest a class of new methods of prediction

density estimation and random variate generation using multilayer networks in this paper we consider two important topics density estimation and random variate generation we present a framework that is easily implemented using the familiar multilayer neural network first we develop two new methods for density estimation a stochastic method and a related deterministic method both methods are based on approximating the distribution function the density being obtained by differentiation in the second part of the paper we develop new random number generation methods our methods do not suffer from some of the restrictions of existing methods in that they can be used to generate numbers from any density provided that certain smoothness conditions are satisfied one of our methods is based on an observed inverse relationship between the density estimation process and random number generation we present two variants of this method a stochastic and a deterministic version we propose a second method that is based on a novel control formulation of the problem where a controller network is trained to shape a given density into the desired density we justify the use of all the methods that we propose by providing theoretical convergence results in particular we prove that the l sub spl infin convergence to the true density for both the density estimation and random variate generation techniques occurs at a rate o log log n n sup 1 spl epsi 2 where n is the number of data points and spl epsi can be made arbitrarily small for sufficiently smooth target densities this bound is very close to the optimally achievable convergence rate under similar smoothness conditions also for comparison the l sub 2 root mean square rms convergence rate of a positive kernel density estimator is o n sup 2 5 when the optimal kernel width is used we present numerical simulations to illustrate the performance of the proposed density estimation and random variate generation methods in addition we present an extended introduction and bibliography that serves as an overview and reference for the practitioner

an accelerated fft algorithm for thermoelastic and non linear composites a fast numerical algorithm to compute the local and overall responses of non linear composite materials is developed this alternative formulation allows us to improve the convergence of the existing method of moulinec and suquet e g comput meth appl mech eng 1998 157 1 2 69 94 in the present method a non linear elastic or conducting material is replaced by infinitely many locally linear thermoelastic materials with moduli that depend on the values of the local fields this makes it possible to use the advantages of an algorithm developed by eyre and milton eur phys j appl phys 1999 6 1 41 47 which has faster convergence the method is applied to compute the local fields as well as the effective response of non linear conducting and elastic periodic composites copyright 2008 john wiley sons ltd

no empirical probability measure can converge in the total variation sense for all distributions for any sequence of empirical probability measures n on the borel sets of the real line and any 0 there exists a singular continuous probability measure such that inf sup n 9m n a a a 1 2 almost surely

combining galerkin approximation techniques with the principle of hashin and shtrikman to derive a new fft based numerical method for the homogenization of composites we report on the mathematical analysis of two different fft based numerical schemes for the homogenization of composite media within the framework of linear elasticity the basic scheme of moulinec and suquet 1994 1998 9 and 10 and the energy based scheme of brisard and dormieux 2010 13 casting these two schemes as galerkin approximations of the same variational problem allows us to assert their well posedness and convergence more importantly we extend in this work their domains of application by relieving some stringent conditions on the reference material which were previously thought necessary the origins of the flaws of each scheme are identified and a third scheme is proposed which seems to combine the strengths of the basic and energy based schemes while leaving out their weaknesses finally a rule is proposed for handling heterogeneous pixels voxels a situation frequently met when images of real materials are used as input to these schemes

the performance of universal encoding universal coding theory is surveyed from the viewpoint of the interplay between delay and redundancy the price for universality turns out to be acceptably small

short note accelerating a fft based solver for numerical homogenization of periodic media by conjugate gradients in this short note we present a new technique to accelerate the convergence of a fft based solver for numerical homogenization of complex periodic media proposed by moulinec and suquet 1 the approach proceeds from discretization of the governing integral equation by the trigonometric collocation method due to vainikko 2 to give a linear system which can be efficiently solved by conjugate gradient methods computational experiments confirm robustness of the algorithm with respect to its internal parameters and demonstrate significant increase of the convergence rate for problems with high contrast coefficients at a low overhead per iteration

estimation of the information by an adaptive partitioning of the observation space we demonstrate that it is possible to approximate the mutual information arbitrarily closely in probability by calculating the relative frequencies on appropriate partitions and achieving conditional independence on the rectangles of which the partitions are made empirical results including a comparison with maximum likelihood estimators are presented

on divergences and informations in statistics and information theory the paper deals with the f divergences of csiszar generalizing the discrimination information of kullback the total variation distance the hellinger divergence and the pearson divergence all basic properties of f divergences including relations to the decision errors are proved in a new manner replacing the classical jensen inequality by a new generalized taylor expansion of convex functions some new properties are proved too e g relations to the statistical sufficiency and deficiency the generalized taylor expansion also shows very easily that all f divergences are average statistical informations differences between prior and posterior bayes errors mutually differing only in the weights imposed on various prior distributions the statistical information introduced by de groot and the classical information of shannon are shown to be extremal cases corresponding to alpha 0 and alpha 1 in the class of the so called arimoto alpha informations introduced in this paper for 0 alpha 1 by means of the arimoto alpha entropies some new examples of f divergences are introduced as well namely the shannon divergences and the arimoto alpha divergences leading for alphauarr1 to the shannon divergences square roots of all these divergences are shown to be metrics satisfying the triangle inequality the last section introduces statistical tests and estimators based on the minimal f divergence with the empirical distribution achieved in the families of hypothetic distributions for the kullback divergence this leads to the classical likelihood ratio test and estimator

guaranteed upper lower bounds on homogenized properties by fft based galerkin method abstract guaranteed upper lower bounds on homogenized coefficients arising from the periodic cell problem are calculated in a scalar elliptic setting our approach builds on the recent variational reformulation of the moulinec suquet 1994 fast fourier transform fft homogenization scheme by vond ejc et al 2014 which is based on the conforming galerkin approximation with trigonometric polynomials upper lower bounds are obtained by adjusting the primal dual finite element framework developed independently by dvo ak 1993 and wi ckowski 1995 to the fft based galerkin setting we show that the discretization procedure differs for odd and non odd number of grid points thanks to the helmholtz decomposition inherited from the continuous formulation the duality structure is fully preserved for the odd discretizations in the latter case a more complex primal dual structure is observed due to presence of the trigonometric polynomials associated with the nyquist frequencies these theoretical findings are confirmed with numerical examples to conclude the main advantage of the fft based approach over conventional finite element schemes is that the primal and the dual problems are treated on the same basis and this property can be extended beyond the scalar elliptic setting

density estimation by stochastic complexity the results by p hall and e j hannan 1988 on optimization of histogram density estimators with equal bin widths by minimization of the stochastic complexity are extended and sharpened in two separate ways as the first contribution two generalized histogram estimators are constructed the first has unequal bin widths which together with the number of the bins are determined by minimization of the stochastic complexity using dynamic programming the other estimator consists of a mixture of equal bin width estimators each of which is defined by the associated stochastic complexity as the main contribution in the present work two theorems are proved which together extend the universal coding theorems to a large class of data generating densities the first gives an asymptotic upper bound for the code redundancy in the order of magnitude achieved with a special predictive type of histogram estimator which sharpens a related bound the second theorem states that this bound cannot be improved upon by any code whatsoever

compression based methods for nonparametric prediction and estimation of some characteristics of time series we address the problem of online prediction for time series we show that any universal code or a universal data compressor can be used as a basis for constructing asymptotically optimal methods for this problem for a certain class of stationary and ergodic processes

combining fft methods and standard variational principles to compute bounds and estimates for the properties of elastic composites abstract in this paper we develop a computational approach based on variational principles combined with fft technique to determine the effective elastic properties of composite materials the unit cell problem of the composite is recast in a weak and integral form by making use of classic variational approaches based on the strain and the stress potential and the hashin shtrikman variational principles the problem is discretized with fourier series and the stationary point is computed numerically by means of an iterative scheme these algorithms use a representation of the local elastic tensor on the double grid twice the size of the discretization which is introduced by the integral formulation it advantageously accounts for the exact geometry of the microstructure and allows the derivation of rigorous bounds of the effective elasticity coefficients in the second part of the paper we establish a hierarchy between the different fft solutions the strain the stress based solutions coming from the classic variational formulations and the polarization based solution derived from the hashin shtrikman principle which depends on the choice of the elastic moduli of the reference material it is proved that the strain and the stress based solutions deliver optimal bounds since they are always better than those obtained from the hashin shtrikman variational principle when the same discretization is used alternatively when the elastic moduli of the reference material is negative the polarization based solution provides an estimate of the effective properties which is comprised between the strain based upper bound and the stress based lower bound

weakly convergent nonparametric forecasting of stationary time series the conditional distribution of the next outcome given the infinite past of a stationary process can be inferred from finite but growing segments of the past several schemes are known for constructing pointwise consistent estimates but they all demand prohibitive amounts of input data we consider real valued time series and construct conditional distribution estimates that make much more efficient use of the input data the estimates are consistent in a weak sense and the question whether they are pointwise consistent is still open for finite alphabet processes one may rely on a universal data compression scheme like the lempel ziv 1978 algorithm to construct conditional probability mass function estimates that are consistent in expected information divergence consistency in this strong sense cannot be attained in a universal sense for all stationary processes with values in an infinite alphabet but weak consistency can some applications of the estimates to on line forecasting regression and classification are discussed

best asymptotic normality of the kernel density entropy estimator for smooth densities in the random sampling setting we estimate the entropy of a probability density distribution by the entropy of a kernel density estimator using the double exponential kernel under mild smoothness and moment conditions we show that the entropy of the kernel density estimator equals a sum of independent and identically distributed i i d random variables plus a perturbation which is asymptotically negligible compared to the parametric rate n sup 1 2 an essential part in the proof is obtained by exhibiting almost sure bounds for the kullback leibler divergence between the kernel density estimator and its expected value the basic technical tools are doob s submartingale inequality and convexity jensen s inequality

fourier based schemes with modified green operator for computing the electrical response of heterogeneous media with accurate local fields a modified green operator is proposed as an improvement of fou rier based numerical schemes commonly used for computing the electrical or thermal response of heterogeneous media contrary to other methods the number of iterations necessary to achieve convergence tends to a finite value when the contrast of properties between the phases becomes infinite furtherm ore it is shown that the method produces much more accurate local field s inside highly conducting and quasi insulating phases as well as in the vicinity of the phases interfaces these good propert ies stem from the discretization of green s function which is consistent with the pixel grid while retaining the local nature of the op erator that acts on the polarization field finally a fast im plementation of the direct scheme of moulinec et al 1994 that allows for parcimonious memory use is proposed

about the asymptotic accuracy of barron density estimates by extending the information theoretic arguments of previous papers dealing with the barron type density estimates and their consistency in information divergence and chi square divergence the problem of consistency in csiszar s spl phi divergence is motivated for general convex functions spl phi the problem of consistency in spl phi divergence is solved for all spl phi with spl phi 0 spl infin and spl phi t o t ln t when t spl rarr spl infin the problem of consistency in the expected spl phi divergence is solved for all spl phi with t spl phi 1 t spl phi t o t sup 2 when t spl rarr spl infin various stronger versions of these asymptotic restrictions are considered too assumptions about the model needed for the consistency are shown to depend on how strong these restrictions are

on the asymptotic properties of a nonparametric l sub 1 test statistic of homogeneity we present two simple and explicit procedures for testing homogeneity of two independent multivariate samples of size n the nonparametric tests are based on the statistic t sub n which is the l sub 1 distance between the two empirical distributions restricted to a finite partition both tests reject the null hypothesis of homogeneity if t sub n becomes large i e if t sub n exceeds a threshold we first discuss chernoff type large deviation properties of t sub n this results in a distribution free strong consistent test of homogeneity then the asymptotic null distribution of the test statistic is obtained leading to an asymptotically spl alpha level test procedure

a steiner formula in the lp brunn minkowski theory abstract we prove an analogue of the classical steiner formula for the l p affine surface area of a minkowski outer parallel body for any real parameter p we show that the classical steiner formula and the steiner formula of lutwak s dual brunn minkowski theory are special cases of this new steiner formula this new steiner formula and its localized versions lead to new curvature measures that have not appeared before in the literature they have the intrinsic volumes of the classical brunn minkowski theory as well as the dual quermassintegrals of the dual brunn minkowski theory as special cases properties of these new quantities are investigated a connection to information theory among them a steiner formula for the s th mixed l p affine surface area of a minkowski outer parallel body for any real parameters p and s is also given

complexity regularized tree structured partition for mutual information estimation a new histogram based mutual information estimator using data driven tree structured partitions tsp is presented in this paper the derived tsp is a solution to a complexity regularized empirical information maximization with the objective of finding a good tradeoff between the known estimation and approximation errors a distribution free concentration in equality for this tree structured learning problem as well as finite sample performance bounds for the proposed histogram based solution is derived it is shown that this solution is density free strongly consistent and that it provides with an arbitrary high probability an optimal balance between the mentioned estimation and approximation errors finally for the emblematic scenario of independence i x y 0 it is shown that the tsp estimate converges to zero with o e n1 3 log log n

universal weak variable length source coding on countably infinite alphabets motivated from the fact that universal source coding on countably infinite alphabets infty alphabets is not feasible this work introduces the notion of almost lossless source coding analog to the weak variable length source coding problem studied by han ieee trans inf theory vol 46 no 4 pp 1217 1226 jul 2000 almost lossless source coding aims at relaxing the lossless block wise assumption to allow an average per letter distortion that vanishes asymptotically as the block length tends to infinity in this setup we show on one hand that shannon entropy characterizes the minimum achievable rate similarly to the case of finite alphabet sources while on the other that almost lossless universal source coding becomes feasible for the family of finite entropy stationary memoryless sources with infty alphabets furthermore we study a stronger notion of almost lossless universality that demands uniform convergence of the average per letter distortion to zero where we establish a necessary and sufficient condition for the so called family of envelope distributions to achieve it remarkably this condition is the same necessary and sufficient condition needed for the existence of a strongly minimax lossless universal source code for the family of envelope distributions finally we show that an almost lossless coding scheme offers faster rate of convergence for the minimax redundancy compared to the well known information radius developed for the lossless case at the expense of tolerating a non zero distortion that vanishes to zero as the block length grows this shows that even when lossless universality is feasible an almost lossless scheme can offer different regimes on the rates of convergence of the worst case redundancy versus the worst case distortion

nonparametric estimation of conditional distributions estimation of conditional distributions is considered it is assumed that the conditional distribution is either discrete or that it has a density with respect to the lebesgue measure partitioning estimates of the conditional distribution are constructed and results concerning consistency and rate of convergence of the integrated total variation error of the estimates are presented

approximation of density functions by sequences of exponential families 1 introduction consider the estimation of a probability density func tion p x defined on a bounded interval we approximate the logarithm of the density by a basis function expansion consisting of polynomials splines or trigonometric series the expansion yields a regular exponential family within which we estimate the density by the method of maximum likelihood this method of density estimation arises by application of the principle of maxi mum entropy or minimum relative entropy subject to empirical constraints we show that if the logarithm of the density has r square integrable deriva tives f idr log p12 ox then the sequence of density estimators p converges to p in the sense of relative entropy kullback leibler fp log p pn at rate opr 1 m2r m n as m x c and m2 n 0 in the spline and trigonometric cases and m3 n o 0 in the polynomial case where m is the dimension of the family and n is the sample size boundary conditions are assumed for the density in the trigonometric case this convergence rate specializes to op r n 2r 2r 1 by setting m nl 2r 1 when the log density is known to have degree of smoothness at least r analogous convergence results for the relative entropy are shown to hold in general for any class of log density functions and sequence of finite dimensional linear spaces having l2 and l approximation properties the approximation of log densities using polynomials has previously been considered by neyman 1937 to define alternatives for goodness of fit tests by good 1963 as an application of the method of maximum entropy or minimum relative entropy by crain 1974 1976a b 1977 who demonstrates existence and consistency of the maximum likelihood estimator and by mead and

an fft based galerkin method for homogenization of periodic media in 1994 moulinec and suquet introduced an efficient technique for the numerical resolution of the cell problem arising in homogenization of periodic media the scheme is based on a fixed point iterative solution to an integral equation of the lippmann schwinger type with action of its kernel efficiently evaluated by the fast fourier transform techniques the aim of this work is to demonstrate that the moulinec suquet setting is actually equivalent to a galerkin discretization of the cell problem based on approximation spaces spanned by trigonometric polynomials and a suitable numerical integration scheme for the latter framework and scalar elliptic problems we prove convergence of the approximate solution to the weak solution including a priori estimates for the rate of convergence for sufficiently regular data and the effects of numerical integration moreover we also show that the variational structure implies that the resulting non symmetric system of linear equations can be solved by the conjugate gradient method apart from providing a theoretical support to fast fourier transform based methods for numerical homogenization these findings significantly improve on the performance of the original solver and pave the way to similar developments for its many generalizations proposed in the literature

divergence for s concave and log concave functions abstract we prove new entropy inequalities for log concave and s concave functions that strengthen and generalize recently established reverse log sobolev and poincare inequalities for such functions this leads naturally to the concept of f divergence and in particular relative entropy for s concave and log concave functions we establish their basic properties among them the affine invariant valuation property applications are given in the theory of convex bodies

uniformly powerful goodness of fit tests the simple hypothesis is tested that the distribution of independent random variables x1 x2 xn is a given probability measure po let 7r be any sequence of partitions the alternative hypothesis is the set of probability measures p with ea ip a po a i 8 where 8 0 note the dependence of this set of alternatives on the sample size it is shown that if the effective cardinality of the partitions is of the same order as the sample size then sequences of tests exist with uniformly exponentially small probabilities of error conversely if the effective cardinality is of larger order than the sample size then no such sequence of tests exists the effective cardinality is the number of sets in the partition which exhaust all but a negligible portion of the probability under the null hypothesis

multivariate density estimation by bayesian sequential partitioning consider a class of densities that are piecewise constant functions over partitions of the sample space defined by sequential coordinate partitioning we introduce a prior distribution for a density in this function class and derive in closed form the marginal posterior distribution of the corresponding partition a computationally efficient method based on sequential importance sampling is presented for the inference of the partition from this posterior distribution compared to traditional approaches such as the kernel method or the histogram the bayesian sequential partitioning bsp method proposed here is capable of providing much more accurate estimates when the sample space is of moderate to high dimension we illustrate this by simulated as well as real data examples the examples also demonstrate how bsp can be used to design new classification methods competitive with the state of the art

mutual information metric entropy and cumulative relative entropy risk assume p theta theta epsilon theta is a set of probability distributions with a common dominating measure on a complete separable metric space y a state theta epsilon theta is chosen by nature a statistician obtains n independent observations y 1 dots y n from y distributed according to p theta for each time t between 1 and n based on the observations y 1 dots y t 1 the statistician produces an estimated distribution hat p t for p theta and suffers a loss l p theta hat p t the cumulative risk for the statistician is the average total loss up to time n of special interest in information theory data compression mathematical finance computational learning theory and statistical mechanics is the special case when the loss l p theta hat p t is the relative entropy between the true distribution p theta and the estimated distribution hat p t here the cumulative bayes risk from time 1 to n is the mutual information between the random parameter theta and the observations y 1 dots y n new bounds on this mutual information are given in terms of the laplace transform of the hellinger distance between pairs of distributions indexed by parameters in theta from these bounds on the cumulative minimax risk are given in terms of the metric entropy of theta with respect to the hellinger distance the assumptions required for these bounds are very general and do not depend on the choice of the dominating measure they apply to both finite and infinite dimensional theta they apply in some cases where y is infinite dimensional in some cases where y is not compact in some cases where the distributions are not smooth and in some parametric cases where asymptotic normality of the posterior distribution fails

nonproduct data dependent partitions for mutual information estimation strong consistency and applications a new framework for histogram based mutual information estimation of probability distributions equipped with density functions in rd b rd is presented in this work a general histogram based estimate is proposed considering nonproduct data dependent partitions and sufficient conditions are stipulated to guarantee a strongly consistent estimate for mutual information two emblematic families of density free strongly consistent estimates are derived from this result one based on statistically equivalent blocks the gessaman s partition and the other on a tree structured vector quantization scheme

a bound on the financial value of information it is shown that each bit of information at most doubles the resulting wealth in the general stock market setup this information bound on the growth of wealth is actually attained for certain probability distributions on the market investigated by j kelly 1956 the bound is shown to be a special case of the result that the increase in exponential growth of wealth achieved with true knowledge of the stock market distribution f over that achieved with incorrect knowledge g is bounded above by the entropy of f relative to g

the size of quadratic p adic linearization disks we find the exact radius of linearization disks at indifferent fixed points of quadratic maps in cp we also show that the radius is invariant under power series perturbations localizing all periodic orbits of these quadratic like maps we then show that periodic points are not the only obstruction for linearization in so doing we provide the first known examples in the dynamics of polynomials over cp where the boundary of the linearization disk does not contain any periodic point

minimax redundancy for the class of memoryless sources let x sup n x sub 1 x sub n be a memoryless source with unknown distribution on a finite alphabet of size k we identify the asymptotic minimax coding redundancy for this class of sources and provide a sequence of asymptotically minimax codes equivalently we determine the limiting behavior of the minimax relative entropy min sub qxn max sub pxn d p sub xn spl par q sub xn where the maximum is over all independent and identically distributed i i d source distributions and the minimum is over all joint distributions we show in this paper that the minimax redundancy minus k 1 2 log n 2 spl pi e converges to log spl int spl radic det i spl theta d spl theta log spl gamma 1 2 sup k spl gamma k 2 where i spl theta is the fisher information and the integral is over the whole probability simplex the bayes strategy using jeffreys prior is shown to be asymptotically maximin but not asymptotically minimax in our setting the boundary risk using jeffreys prior is higher than that of interior points we provide a sequence of modifications of jeffreys prior that put some prior mass near the boundaries of the probability simplex to pull down that risk to the asymptotic minimax level in the limit

mixing strategies for density estimation general results on adaptive density estimation are obtained with respect to any countable collection of estimation strategies under kullback leibler and squared l 2 losses it is shown that without knowing which strategy works best for the underlying density a single strategy can be constructed by mixing the proposed ones to be adaptive in terms of statistical risks a consequence is that under some mild conditions an asymptotically minimax rate adaptive estimator exists for a given countable collection of density classes that is a single estimator can be constructed to be simultaneously minimax rate optimal for all the function classes being considered a demonstration is given for high dimensional density estimation on 0 1 d where the constructed estimator adapts to smoothness and interaction order over some piecewise besov classes and is consistent for all the densities with finite entropy

consistent nonparametric tests of independence three simple and explicit procedures for testing the independence of two multi dimensional random variables are described two of the associated test statistics l1 log likelihood are defined when the empirical distribution of the variables is restricted to finite partitions a third test statistic is defined as a kernel based independence measure two kinds of tests are provided distribution free strong consistent tests are derived on the basis of large deviation bounds on the test statistics these tests make almost surely no type i or type ii error after a random sample size asymptotically level tests are obtained from the limiting distribution of the test statistics for the latter tests the type i error converges to a fixed non zero value and the type ii error drops to zero for increasing sample size all tests reject the null hypothesis of independence if the test statistics become large the performance of the tests is evaluated experimentally on benchmark data

information theoretic asymptotics of bayes methods in the absence of knowledge of the true density function bayesian models take the joint density function for a sequence of n random variables to be an average of densities with respect to a prior the authors examine the relative entropy distance d sub n between the true density and the bayesian density and show that the asymptotic distance is d 2 log n c where d is the dimension of the parameter vector therefore the relative entropy rate d sub n n converges to zero at rate log n n the constant c which the authors explicitly identify depends only on the prior density function and the fisher information matrix evaluated at the true parameter value consequences are given for density estimation universal data compression composite hypothesis testing and stock market portfolio selection