asymptotic equivalence of nonparametric regression and white noise the principal result is that under conditions to any nonparametric regression problem there corresponds an asymptotically equivalent sequence of white noise with drift problems and conversely this asymptotic equivalence is in a global and uniform sense any normalized risk function attainable in one problem is asymptotically attainable in the other with the difference in normalized risks converging to zero uniformly over the entire parameter space the results are constructive a recipe is provided for producing these asymptotically equivalent procedures some implications and generalizations of the principal result are also discussed

rates of convergence for bayes and maximum likelihood estimation for mixture of normal densities we study the rates of convergence of the maximum likelihood estimator mle and posterior distribution in density estimation problems where the densities are location or location scale mixtures of normal distributions with the scale parameter lying between two positive numbers the true density is also assumed to lie in this class with the true mixing distribution either compactly supported or having sub gaussian tails we obtain bounds for hellinger bracketing entropies for this class and from these bounds we deduce the convergence rates of sieve mles in hellinger distance the rate turns out to be log n n where k 1 is a constant that depends on the type of mixtures and the choice of the sieve next we consider a dirichlet mixture of normals as a prior on the unknown density we estimate the prior probability of a certain kullback leibler type neighborhood and then invoke a general theorem that computes the posterior convergence rate in terms the growth rate of the hellinger entropy and the concentration rate of the prior the posterior distribution is also seen to converge at the rate log n n in where k now depends on the tail behavior of the base measure of the dirichlet process

bayesian aspects of some nonparametric problems we study the bayesian approach to nonparametric function estimation problems such as nonparametric regression and signal estimation we consider the asymptotic properties of bayes procedures for conjugate gaussian priors we show that so long as the prior puts nonzero measure on the very large parameter set of interest then the bayes estimators are not satisfactory more specifically we show that these estimators do not achieve the correct minimax rate over norm bounded sets in the parameter space thus all bayes estimators for proper gaussian priors have zero asymptotic efficiency in this minimax sense we then present a class of priors whose bayes procedures attain the optimal minimax rate of convergence these priors may be viewed as compound or hierarchical mixtures of suitable gaussian distributions

bayes procedures for adaptive inference in inverse problems for the white noise model we study empirical and hierarchical bayes approaches to the problem of estimating an infinite dimensional parameter in mildly ill posed inverse problems we consider a class of prior distributions indexed by a hyperparameter that quantifies regularity we prove that both methods we consider succeed in automatically selecting this parameter optimally resulting in optimal convergence rates for truths with sobolev or analytic smoothness without using knowledge about this regularity both methods are illustrated by simulation examples

a bernstein von mises theorem in the nonparametric right censoring model in the recent bayesian nonparametric literature many examples have been reported in which bayesian estimators and posterior distributions do not achieve the optimal convergence rate indicating that the bernstein von mises theorem does not hold in this article we give a positive result in this direction by showing that the bernstein von mises theorem holds in survival models for a large class of prior processes neutral to the right we also show that for an arbitrarily given convergence rate n with 0 1 2 a prior process neutral to the right can be chosen so that its posterior distribution achieves the convergence rate n

on bayesian consistency we consider a sequence of posterior distributions based on a data dependent prior which we shall refer to as a pseudoposterior distribution and establish simple conditions under which the sequence is hellinger consistent it is shown how investigations into these pseudo posteriors assist with the understanding of some true posterior distributions including polya trees the infinite dimensional exponential family and mixture models

convergence of latent mixing measures in finite and infinite mixture models this paper studies convergence behavior of latent mixing measures that arise in finite and infinite mixture models using transportation distances i e wasserstein metrics the relationship between wasserstein distances on the space of mixing measures and f divergence functionals such as hellinger and kullback leibler distances on the space of mixture distributions is investigated in detail using various identifiability conditions convergence in wasserstein metrics for discrete measures implies convergence of individual atoms that provide support for the measures thereby providing a natural interpretation of convergence of clusters in clustering applications where mixture models are typically employed convergence rates of posterior distributions for latent mixing measures are established for both finite mixtures of multivariate distributions and infinite mixtures based on the dirichlet process

information consistency of nonparametric gaussian process methods bayesian nonparametric models are widely and successfully used for statistical prediction while posterior consistency properties are well studied in quite general settings results have been proved using abstract concepts such as metric entropy and they come with subtle conditions which are hard to validate and not intuitive when applied to concrete models furthermore convergence rates are difficult to obtain by focussing on the concept of information consistency for bayesian gaussian process gp models consistency results and convergence rates are obtained via a regret bound on cumulative log loss these results depend strongly on the covariance function of the prior process thereby giving a novel interpretation to penalization with reproducing kernel hilbert space norms and to commonly used covariance function classes and their parameters the proof of the main result employs elementary convexity arguments only a theorem of widom is used in order to obtain precise convergence rates for several covariance functions widely used in practice

learning causal bayesian network structures from experimental data we propose a method for the computational inference of directed acyclic graphical structures given data from experimental interventions order space markov chain monte carlo equi energy sampling importance weighting and stream based computation are combined to create a fast algorithm for learning causal bayesian network structures

on bayesian supremum norm contraction rates building on ideas from castillo and nickl 6 a method is pro vided to study nonparametric bayesian posterior convergence rates when strong measures of distances such as the sup norm are considered in particular we show that likelihood methods can achieve optimal minimax sup norm rates in density estimation on the unit interval the introduced methodology is used to prove that commonly used families of prior distri butions on densities namely log density priors and dyadic random density histograms can indeed achieve optimal sup norm rates of convergence new results are also derived in the gaussian white noise model as a further il lustration of the presented techniques ams 2000 subject classications primary 62g20 secondary 62g05 62g07

on the convergence of spline collocation methods for solving fractional differential equations in the first part of this paper we study the regularity properties of solutions of initial value problems of linear multi term fractional differential equations we then use these results in the convergence analysis of a polynomial spline collocation method for solving such problems numerically using an integral equation reformulation and special non uniform grids global convergence estimates are derived from these estimates it follows that the method has a rapid convergence if we use suitable nonuniform grids and the nodes of the composite gaussian quadrature formulas as collocation points theoretical results are verified by some numerical examples

posterior consistency of dirichlet mixtures in density estimation a dirichlet mixture of normal densities is a useful choice for a prior distribution on densities in the problem of bayesian density estimation in the recent years efficient markov chain monte carlo method for the computation of the posterior distribution has been developed the method has been applied to data arising from different fields of interest the important issue of consistency was however left open in this paper we settle this issue in affirmative

on the bernstein von mises theorem with infinite dimensional parameters if there are many independent identically distributed observations governed by a smooth finite dimensional statistical model the bayes estimate and the maximum likelihood estimate will be close furthermore the posterior distribution of the parameter vector around the posterior mean will be close to the distribution of the maximum likelihood estimate around truth thus bayesian confidence sets have good frequentist coverage properties and conversely however even for the simplest infinite dimensional models such results do not hold the object here is to give some examples

from ɛ entropy to kl entropy analysis of minimum information complexity density estimation we consider an extension of e entropy to a kl divergence based complexity measure for randomized density estimation methods based on this extension we develop a general information theoretical inequality that measures the statistical complexity of some deterministic and randomized density estimators consequences of the new inequality will be presented in particular we show that this technique can lead to improvements of some classical results concerning the convergence of minimum description length and bayesian posterior distributions moreover we are able to derive clean finite sample convergence bounds that are not obtainable using previous approaches

on the consistency of bayesian function approximation using step functions we consider the problem of estimating a step function with an unknown number of jumps under noisy observations on a grid under mild assumptions the bayesian approach is shown to produce a consistent estimate even when the underlying true function is not piecewise constant a simple prior is constructed to illustrate our assumptions

numerical solution of semidifferential equations by collocation method we study the numerical solution of semidifferential equation of order 4 by collocation spline method we derive a system of equations that characterizing the numerical solution some numerical examples are also presented to illustrate our results

convergence rate of sieve estimates in this paper we develop a general theory for the convergence rate of sieve estimates maximum likelihood estimates mle s and related estimates obtained by optimizing certain empirical criteria in general parameter spaces in many cases especially when the parameter space is infinite dimensional maximization over the whole parameter space is undesirable in such cases one has to perform maximization over an approximating space sieve of the original parameter space and allow the size of the approximating space to grow as the sample size increases this method is called the method of sieves in the case of the maximum likelihood estimation an mle based on a sieve is called a sieve mle we found that the convergence rate of a sieve estimate is governed by a the local expected values variances and l 2 entropy of the criterion differences and b the approximation error of the sieve a robust nonparametric regression problem a mixture problem and a nonparametric regression problem are discussed as illustrations of the theory we also found that when the underlying space is too large the estimate based on optimizing over the whole parameter space may not achieve the best possible rates of convergence whereas the sieve estimate typically does not suffer from this difficulty

on rates of convergence for posterior distributions in infinite dimensional models this paper introduces a new approach to the study of rates of convergence for posterior distributions it is a natural extension of a recent approach to the study of bayesian consistency in particular we improve on current rates of convergence for models including the mixture of dirichlet process model and the random bernstein polynomial model

rates of contraction for posterior distributions in lr metrics 1 r the frequentist behavior of nonparametric bayes estimates more specifically rates of contraction of the posterior distributions to shrinking lr norm neighborhoods 1 r of the unknown parameter are studied a theorem for nonparametric density estimation is proved under general approximation theoretic assumptions on the prior the result is applied to a variety of common examples including gaussian process wavelet series normal mixture and histogram priors the rates of contraction are minimax optimal for 1 r 2 but deteriorate as r increases beyond 2 in the case of gaussian nonparametric regression a gaussian prior is devised for which the posterior contracts at the optimal rate in all lr norms 1 r

a semiparametric bernstein von mises theorem for gaussian process priors this paper is a contribution to the bayesian theory of semiparametric estimation we are interested in the so called bernstein von mises theorem in a semiparametric framework where the unknown quantity is f with the parameter of interest and f an infinite dimensional nuisance parameter two theorems are established one in the case with no loss of information and one in the information loss case with gaussian process priors the general theory is applied to three specific models the estimation of the center of symmetry of a symmetric function in gaussian white noise a time discrete functional data analysis model and cox s proportional hazards model in all cases the range of application of the theorems is investigated by using a family of gaussian priors parametrized by a continuous parameter

bernstein von mises theorems for gaussian regression with increasing number of regressors this paper brings a contribution to the bayesian theory of nonparametric and semiparametric estimation we are interested in the asymptotic normality of the posterior distribution in gaussian linear regression models when the number of regressors increases with the sample size two kinds of bernstein von mises theorems are obtained in this framework nonparametric theorems for the parameter itself and semiparametric theorems for functionals of the parameter we apply them to the gaussian sequence model and to the regression of functions in sobolev and c classes in which we get the minimax convergence rates adaptivity is reached for the bayesian estimators of functionals in our applications

risk bounds for model selection via penalization performance bounds for criteria for model selection are devel oped using recent theory for sieves the model selection criteria are based on an empirical loss or contrast function with an added penalty term moti vated by empirical process theory and roughly proportional to the number of parameters needed to describe the model divided by the number of ob servations most of our examples involve density or regression estimation settings and we focus on the problem of estimating the unknown density or regression function we show that the quadratic risk of the minimum penal ized empirical contrast estimatoris bounded by an index of the accuracy of the sieve this accuracy index quanties the trade off among the candidate models between the approximation error and parameter dimension relative to sample size if we choose a list of models which exhibit good approximation prop erties with respect to different classes of smoothness the estimator can be simultaneously minimax rate optimal in each of those classes this is what is usually called adaptation the type of classes of smoothness in which one gets adaptation depends heavily on the list of models if too many models are involved in order to get accurate approximation of many wide classes of functions simultaneously it may happen that the estimator is only approx

thomas bayes walk on manifolds convergence of the bayes posterior measure is considered in canonical statistical settings where observations sit on a geometrical object such as a compact manifold or more generally on a compact metric space verifying some conditions a natural geometric prior based on randomly rescaled solutions of the heat equation is considered upper and lower bound posterior contraction rates are derived

posterior consistency of nonparametric conditional moment restricted models this paper addresses the estimation of the nonparametric conditional moment restricted model that involves an infinite dimensional parameter g0 we estimate it in a quasi bayesian way based on the limited information likelihood and investigate the impact of three types of priors on the posterior consistency i truncated prior priors supported on a bounded set ii thin tail prior a prior that has very thin tail outside a growing bounded set and iii normal prior with nonshrinking variance in addition g0 is allowed to be only partially identified in the frequentist sense and the parameter space does not need to be compact the posterior is regularized using a slowly growing sieve dimension and it is shown that the posterior converges to any small neighborhood of the identified region we then apply our results to the nonparametric instrumental regression model finally the posterior consistency using a random sieve dimension parameter is studied

adaptive bayesian multivariate density estimation with dirichlet mixtures summary we show that rate adaptive multivariate density estimation can be performed using bayesian methods based on dirichlet mixtures of normal kernels with a prior distribution on the kernel s covariance matrix parameter we derive sufficient conditions on the prior specification that guarantee convergence to a true density at a rate that is minimax optimal for the smoothness class to which the true density belongs no prior knowledge of smoothness is assumed the sufficient conditions are shown to hold for the dirichlet location mixture of normals prior with a gaussian base measure and an inverse wishart prior on the covariance matrix parameter locally h older

a theorem on the asymptotic eigenvalue distribution of toeplitz block toeplitz matrices we extend szego s eigenvalue distribution theorem to block toeplitz matrices a sequence of such matrices arises from the autocorrelation of a 2 d discrete random process we show that the eigenvalues of the matrix sequence are asymptotically distributed like the samples of the random process 2 d power spectrum

the consistency of posterior distributions in nonparametric problems we give conditions that guarantee that the posterior probability of every hellinger neighborhood of the true distribution tends to 1 almost surely the conditions are 1 a requirement that the prior not put high mass near distributions with very rough densities and 2 a requirement that the prior put positive mass in kullback leibler neighborhoods of the true distribution the results are based on the idea of approximating the set of distributions with a finite dimensional set of distributions with sufficiently small hellinger bracketing metric entropy we apply the results to some examples

new approaches to bayesian consistency we use martingales to study bayesian consistency we derive sufficient conditions for both hellinger and kullback leibler consistency which do not rely on the use of a sieve alternative sufficient conditions for hellinger consistency are also found and demonstrated on examples

spline collocation methods for linear multi term fractional differential equations abstract some regularity properties of the solution of linear multi term fractional differential equations are derived based on these properties the numerical solution of such equations by piecewise polynomial collocation methods is discussed the results obtained in this paper extend the results of pedas and tamme 2011 15 where we have assumed that in the fractional differential equation the order of the highest derivative of the unknown function is an integer in the present paper we study the attainable order of convergence of spline collocation methods for solving general linear fractional differential equations using caputo form of the fractional derivatives and show how the convergence rate depends on the choice of the grid and collocation points theoretical results are verified by some numerical examples

an analysis of bayesian inference for nonparametric regression abstract the observation model y sub i beta i n epsilon sub i 1 or n is considered where the epsilon s are i i d mean zero and variance sigma sq and beta is an unknown smooth function a gaussian prior distribution is specified by assuming beta is the solution of a high order stochastic differential equation the estimation error delta beta beta average is analyzed where beta average is the posterior expectation of beta asymptotic posterior and sampling distributional approximations are given for abs val del square when abs val square is one of a family of norms natural to the problem it is shown that the frequentist coverage probability of a variety of 1 alpha posterior probability regions tends to be larger than 1 alpha but will be infinitely often less than any epsilon 0 as n approaches infinity with prior probability 1 a related continuous time signal estimation problem is also studied keywords bayesian inference nonparametric regression confidence regions signal extraction smoothing splices

bayesian regression with nonparametric heteroskedasticity abstract this paper studies large sample properties of a semiparametric bayesian approach to inference in a linear regression model the approach is to model the distribution of the regression error term by a normal distribution with the variance that is a flexible function of covariates the main result of the paper is a semiparametric bernstein von mises theorem under misspecification even when the distribution of the regression error term is not normal the posterior distribution of the properly recentered and rescaled regression coefficients converges to a normal distribution with the zero mean and the variance equal to the semiparametric efficiency bound

numerical solution of fractional integro differential equations by collocation method abstract this paper deals with the numerical solution of fractional integro differential equations of the type d q y t p t y t f t 0 t k t s y s d s t i 0 1 by polynomial spline functions we derive a system of equations that characterizing the numerical solution some numerical examples are also provided to illustrated our results

an asymptotic property of model selection criteria probability models are estimated by use of penalized log likelihood criteria related to akaike 1973 information criterion aic and minimum description length mdl the accuracies of the density estimators are shown to be related to the tradeoff between three terms the accuracy of approximation the model dimension and the descriptive complexity of the model classes the asymptotic risk is determined under conditions on the penalty term and is shown to be minimax optimal for some cases as an application we show that the optimal rate of convergence is simultaneously achieved for log densities in sobolev spaces w sub 2 sup s u without knowing the smoothness parameter s and norm parameter u in advance applications to neural network models and sparse density function estimation are also provided

convergence rates of posterior distributions we consider the asymptotic behavior of posterior distributions and bayes estimators for infinite dimensional statistical models we give general results on the rate of convergence of the posterior measure these are applied to several examples including priors on finite sieves log spline models dirichlet processes and interval censoring

convergence rates of posterior distributions for non i i d observations we consider the asymptotic behavior of posterior distributions and bayes estimators based on observations which are required to be neither independent nor identically distributed we give general results on the rate of convergence of the posterior measure relative to distances derived from a testing criterion we then specialize our results to independent nonidentically distributed observations markov processes stationary gaussian time series and the white noise model we apply our general results to several examples of infinite dimensional statistical models including nonparametric regression with normal errors binary regression poisson regression an interval censoring model whittle estimation of the spectral density of a time series and a nonlinear autoregressive model

rates of convergence of posterior distributions we compute the rate at which the posterior distribution concentrates around the true parameter value the spaces we work in are quite general and include infinite dimensional cases the rates are driven by two quantities the size of the space as measured by bracketing entropy and the degree to which the prior concentrates in a small ball around the true parameter we consider two examples

consistency of bernstein polynomial posteriors a bernstein prior is a probability measure on the space of all the distribution functions on 0 1 under very general assumptions it selects absolutely continuous distribution functions whose densities are mixtures of known beta densities the bemstein prior is of interest in bayesian nonparametric inference with continuous data we study the consistency of the posterior from a bernstein prior we first show that under mild assumptions the posterior is weakly consistent for any distribution function p o on 0 1 with continuous and bounded lebesgue density with slightly stronger assumptions on the prior the posterior is also hellinger consistent this implies that the predictive density from a bernstein prior which is a bayesian density estimate converges in the hellinger sense to the true density assuming that it is continuous and bounded we also study a sieve maximum likelihood version of the density estimator and show that it is also hellinger consistent under weak assumptions when the order of the bernstein polynomial i e the number of components in the beta distribution mixture is truncated we show that under mild restrictions the posterior concentrates on the set of pseudotrue densities finally we study the behaviour of the predictive density numerically and we also study a hybrid bayes maximum likelihood density estimator