volterra method for the radial schrodinger equation a new volterra type integral equation method for the numerical solution of the radial schrodinger equation is investigated the method carried out in configuration space is based on the conversion of differential equations into a system of volterra type integral equations together with the application of a spectral type clenshaw curtis quadrature through numerical examples the volterra type integral equation method is shown to be more efficient and more accurate than the integral equation method based on the fredholm formulation j comput phys 134 1997 134 and better than finite difference methods accompanying c code for the volterra type method is available upon request

high dimensional classification via nonparametric empirical bayes and maximum likelihood inference we propose new nonparametric empirical bayes methods for high dimensional classification our classifiers are designed to approximate the bayes classifier in a hypothesized hierarchical model where the prior distributions for the model parameters are estimated nonparametrically from the training data as is common with nonparametric empirical bayes the proposed classifiers are effective in high dimensional settings even when the underlying model parameters are in fact nonrandom we use nonparametric maximum likelihood estimates of the prior distributions following the elegant approach studied by kiefer wolfowitz in the 1950s however our implementation is based on a recent convex optimization framework for approximating these estimates that is well suited for large scale problems we derive new theoretical results on the accuracy of the approximate estimator which help control the misclassification rate of one of our classifiers we show that our methods outperform several existing methods in simulations and perform well when gene expression microarray data is used to classify cancer patients

ideal spatial adaptation by wavelet shrinkage summary with ideal spatial adaptation an oracle furnishes information about how best to adapt a spatially variable estimator whether piecewise constant piecewise polynomial variable knot spline or variable bandwidth kernel to the unknown function estimation with the aid of an oracle offers dramatic advantages over traditional linear estimation by nonadaptive kernels however it is a priori unclear whether such performance can be obtained by a procedure relying on the data alone we describe a new principle for spatially adaptive estimation selective wavelet reconstruction we show that variable knot spline fits and piecewise polynomial fits when equipped with an oracle to select the knots are not dramatically more powerful than selective wavelet reconstruction with an oracle we develop a practical spatially adaptive method riskshrink which works by shrinkage of empirical wavelet coefficients riskshrink mimics the performance of an oracle for selective wavelet reconstruction as well as it is possible to do so a new inequality in multivariate normal decision theory which we call the oracle inequality shows that attained performance differs from ideal performance by at most a factor of approximately 2 log n where n is the sample size moreover no estimator can give a better guarantee than this within the class of spatially adaptive procedures riskshrink is essentially optimal relying only on the data it comes within a factor log 2 n of the performance of piecewise polynomial and variableknot spline methods equipped with an oracle in contrast it is unknown how or if piecewise polynomial methods could be made to function this well when denied access to an oracle and forced to rely on data alone

rates of convergence for bayes and maximum likelihood estimation for mixture of normal densities we study the rates of convergence of the maximum likelihood estimator mle and posterior distribution in density estimation problems where the densities are location or location scale mixtures of normal distributions with the scale parameter lying between two positive numbers the true density is also assumed to lie in this class with the true mixing distribution either compactly supported or having sub gaussian tails we obtain bounds for hellinger bracketing entropies for this class and from these bounds we deduce the convergence rates of sieve mles in hellinger distance the rate turns out to be log n n where k 1 is a constant that depends on the type of mixtures and the choice of the sieve next we consider a dirichlet mixture of normals as a prior on the unknown density we estimate the prior probability of a certain kullback leibler type neighborhood and then invoke a general theorem that computes the posterior convergence rate in terms the growth rate of the hellinger entropy and the concentration rate of the prior the posterior distribution is also seen to converge at the rate log n n in where k now depends on the tail behavior of the base measure of the dirichlet process

on the numerical solution of two point boundary value problems abstract this paper presents a new numerical method for the solution of linear two point boundary value problems of ordinary differential equations after reducing the differential equation to a second kind integral equation the latter are discretized via a high order nystrom scheme a somewhat involved analytical apparatus is then constructed which allows for the solution of the discrete system using o n dot p2 operations where n is the number of nodes on the interval and p is the desired order of convergence thus the advantages of the integral equation formulation small condition number insensitivity to boundary layers insensitivity to end point singularities etc are retained while achieving a computational efficiency previously available only to finite difference or finite element methods keywords two point boundary value problems integral equations chebyshev polynomials approximation theory

adapting to unknown sparsity by controlling the false discovery rate we attempt to recover an n dimensional vector observed in white noise where n is large and the vector is known to be sparse but the degree of sparsity is unknown we consider three different ways of defining sparsity of a vector using the fraction of nonzero terms imposing power law decay bounds on the ordered entries and controlling the lp norm for p small we obtain a procedure which is asymptotically minimax for l r loss simultaneously throughout a range of such sparsity classes the optimal procedure is a data adaptive thresholding scheme driven by control of the false discovery rate fdr fdr control is a relatively recent innovation in simultaneous testing ensuring that at most a certain fraction of the rejected null hypotheses will correspond to false rejections in our treatment the fdr control parameter qn also plays a determining role in asymptotic minimaxity if q lim qn 0 1 2 and also qn log n we get sharp asymptotic minimaxity simultaneously over a wide range of sparse parameter spaces and loss functions on the other hand q lim qn 1 2 1 forces the risk to exceed the minimax risk by a factor growing with q to our knowledge this relation between ideas in simultaneous inference and asymptotic decision theory is new our work provides a new perspective on a class of model selection rules which has been introduced recently by several authors these new rules impose complexity penalization of the form 2 log potential model size actual model size we exhibit a close connection with fdr controlling procedures under stringent control of the false discovery rate

adaptive nonparametric bayesian inference using location scale mixture priors we study location scale mixture priors for nonparametric statistical problems including multivariate regression density estimation and classification we show that a rate adaptive procedure can be obtained if the prior is properly constructed in particular we show that adaptation is achieved if a kernel mixture prior on a regression function is constructed using a gaussian kernel an inverse gamma bandwidth and gaussian mixing weights

asymptotic behaviour of the posterior distribution in overfitted mixture models abstract summary we study the asymptotic behaviour of the posterior distribution in a mixture model when the number of components in the mixture is larger than the true number of components a situation which is commonly referred to as an overfitted mixture we prove in particular that quite generally the posterior distribution has a stable and interesting behaviour since it tends to empty the extra components this stability is achieved under some restriction on the prior which can be used as a guideline for choosing the prior some simulations are presented to illustrate this behaviour

aggregation of predictors for non stationary sub linear processes and online adaptive forecasting of time varying autoregressive processes in this work we study the problem of aggregating a finite number of predictors for non stationary sub linear processes we provide oracle inequalities relying essentially on three ingredients 1 a uniform bound of the ell 1 norm of the time varying sub linear coefficients 2 a lipschitz assumption on the predictors and 3 moment conditions on the noise appearing in the linear representation two kinds of aggregations are considered giving rise to different moment conditions on the noise and more or less sharp oracle inequalities we apply this approach for deriving an adaptive predictor for locally stationary time varying autoregressive tvar processes it is obtained by aggregating a finite number of well chosen predictors each of them enjoying an optimal minimax convergence rate under specific smoothness conditions on the tvar coefficients we show that the obtained aggregated predictor achieves a minimax rate while adapting to the unknown smoothness to prove this result a lower bound is established for the minimax rate of the prediction risk for the tvar process numerical experiments complete this study an important feature of this approach is that the aggregated predictor can be computed recursively and is thus applicable in an online prediction context

group linear empirical bayes estimates for a heteroscedastic normal mean abstractthe problem of estimating the mean of a normal vector with known but unequal variances introduces substantial difficulties that impair the adequacy of traditional empirical bayes estimators by taking a different approach that treats the known variances as part of the random observations we restore symmetry and thus the effectiveness of such methods we suggest a group linear empirical bayes estimator which collects observations with similar variances and applies a spherically symmetric estimator to each group separately the proposed estimator is motivated by a new oracle rule which is stronger than the best linear rule and thus provides a more ambitious benchmark than that considered in the previous literature our estimator asymptotically achieves the new oracle risk under appropriate conditions and at the same time is minimax the group linear estimator is particularly advantageous in situations where the true means and observed variances are empirically dependent to demonstrate the meri

bayes and empirical bayes do they merge bayesian inference is attractive for its coherence and good frequentist properties however eliciting a honest prior may be difficult and a common practice is to take an empirical bayes approach using some empirical estimate of the prior hyperparameters despite not rigorous the underlying idea is that for sufficiently large sample size empirical bayes leads to similar inferential answers as a proper bayesian inference however precise mathematical results seem missing in this work we give more rigorous results in terms of merging of bayesian and empirical bayesian posterior distributions we study two notions of merging bayesian weak merging and frequentist merging in total variation we also show that under regularity conditions empirical bayes asymptotically gives an oracle selection of the prior hyperparameters examples include empirical bayes density estimation with dirichlet process mixtures

model based gaussian and non gaussian clustering abstract the classification maximum likelihood approach is sufficiently general to encompass many current clustering algorithms including those based on the sum of squares criterion and on the criterion of friedman and rubin 1967 however as currently implemented it does not allow the specification of which features orientation size and shape are to be common to all clusters and which may differ between clusters also it is restricted to gaussian distributions and it does not allow for noise we propose ways of overcoming these limitations a reparameterization of the covariance matrix allows us to specify that some features but not all be the same for all clusters a practical framework for non gaussian clustering is outlined and a means of incorporating noise in the form of a poisson process is described an approximate bayesian method for choosing the number of clusters is given the performance of the proposed methods is studied by simulation with encouraging results the methods are applied to the analysis of a data set arising in the study of diabetes and the results seem better than those of previous analyses rh

quantum time of arrival goursat problem the construction of quantum time of arrival operator conjugate to the system hamiltonian leads to a particular linear homogeneous goursat problem in this work we demonstrate how to approximate the solution of the mentioned differential equation both semi analytically and numerically with the goal of calculating the largest eigenvalue of the associated confined time of arrival operator in the analytical approximation we used the partial sum expansion of the solution and showed that the resulting largest eigenvalue converges as the number of terms increases the result shows that for the parameters considered in this paper the approximation is sufficient up to the fourth order correction term in the numerical approximation we develop a non iterative formula to obtain the numerical solution of the goursat problem the performance of the non iterative method is compared with the known numerical techniques in literature numerical results show that the non iterative algorithm is more accurate and faster

a recursive algorithm for mixture of densities estimation recursive algorithms for the estimation of mixtures of densities have attracted a lot of attention in the last 10 years here an algorithm for recursive estimation is studied it complements existing approaches in the literature as it is based on conditions that are usually very weak for example the parameter space over which the mixture is taken does not need to be necessarily bounded the essence of the procedure is to combine density estimation via empirical characteristic function together with an iterative hilbert space approximation algorithm the conditions for consistency of the estimator are verified for three important statistical problems a simulation study is also included

hellinger consistency of certain nonparametric maximum likelihood estimators consider a class mathscr p p theta theta in theta of probability measures on a measurable space mathscr x mathscr a dominated by a sigma finite measure mu let f theta dp theta d mu theta in theta and let theta n be a maximum likelihood estimator based on n independent observations from p theta 0 theta 0 in theta we use results from empirical process theory to obtain convergence for the hellinger distance h f hat theta n f theta 0 under certain entropy conditions on the class of densities f theta theta in theta the examples we present are a model with interval censored observations smooth densities monotone densities and convolution models in most examples the convexity of the class of densities is of special importance

fast solution methods for fredholm integral equations of the second kind the main purpose of this paper is to describe a fast solution method for one dimensional fredholm integral equations of the second kind with a smooth kernel and a non smooth right hand side function let the integral equation be defined on the interval 1 1 we discretize by a nystrom method with nodes cos j n j 0 n this yields a linear system of algebraic equations with an n 1 n 1 matrixa generallyn has to be chosen fairly large in order to obtain an accurate approximate solution of the integral equation we show by fourier analysis thata can be approximated well by mathop a limits sim a low rank modification of the identity matrix replacinga by mathop a limits sim in the linear system of algebraic equations yields a new linear system of equations whose elements and whose solution tilde x can be computed ino n logn arithmetic operations if the kernel has two more derivatives than the right hand side function then tilde x is shown to converge optimally to the solution of the integral equation asn increases we also consider iterative solution of the linear system of algebraic equations the iterative schemes use botha anda they yield the solution ino n 2 arithmetic operations under mild restrictions on the kernel and the right hand side function finally we discuss discretization by the chebyshev galerkin method the techniques developed for the nystrom method carry over to this discretization method and we develop solution schemes that are faster than those previously presented in the literature the schemes presented carry over in a straightforward manner to fredholm integral equations of the second kind defined on a hypercube

adaptive bayesian estimation using a gaussian random field with inverse gamma bandwidth we consider nonparametric bayesian estimation inference using a rescaled smooth gaussian field as a prior for a multidimensional function the rescaling is achieved using a gamma variable and the procedure can be viewed as choosing an inverse gamma bandwidth the procedure is studied from a frequentist perspective in three statistical settings involving replicated observations density estimation regression and classification we prove that the resulting posterior distribution shrinks to the distribution that generates the data at a speed which is minimax optimal up to a logarithmic factor whatever the regularity level of the data generating distribution thus the hierachical bayesian procedure with a fixed prior is shown to be fully adaptive

general empirical bayes wavelet methods and exactly adaptive minimax estimation in many statistical problems stochastic signals can be represented as a sequence of noisy wavelet coefficients in this paper we develop general empirical bayes methods for the estimation of true signal our estimators approximate certain oracle separable rules and achieve adaptation to ideal risks and exact minimax risks in broad collections of classes of signals in particular our estimators are uniformly adaptive to the minimum risk of separable estimators and the exact minimax risks simultaneously in besov balls of all smoothness and shape indices and they are uniformly superefficient in convergence rates in all compact sets in besov spaces with a finite secondary shape parameter furthermore in classes nested between besov balls of the same smoothness index our estimators dominate threshold and james stein estimators within an infinitesimal fraction of the minimax risks more general block empirical bayes estimators are developed both white noise with drift and nonparametric regression are considered

posterior consistency of dirichlet mixtures in density estimation a dirichlet mixture of normal densities is a useful choice for a prior distribution on densities in the problem of bayesian density estimation in the recent years efficient markov chain monte carlo method for the computation of the posterior distribution has been developed the method has been applied to data arising from different fields of interest the important issue of consistency was however left open in this paper we settle this issue in affirmative

sure estimates for a heteroscedastic hierarchical model hierarchical models are extensively studied and widely used in statistics and many other scientific areas they provide an effective tool for combining information from similar resources and achieving partial pooling of inference since the seminal work by james and stein 1961 and stein 1962 shrinkage estimation has become one major focus for hierarchical models for the homoscedastic normal model it is well known that shrinkage estimators especially the james stein estimator have good risk properties the heteroscedastic model though more appropriate for practical applications is less well studied and it is unclear what types of shrinkage estimators are superior in terms of the risk we propose in this article a class of shrinkage estimators based on stein s unbiased estimate of risk sure we study asymptotic properties of various common estimators as the number of means to be estimated grows p we establish the asymptotic optimality property for the sure estimators we then extend our

convergence rates for bayesian density estimation of infinite dimensional exponential families we study the rate of convergence of posterior distributions in density estimation problems for log densities in periodic sobolev classes characterized by a smoothness parameter p the posterior expected density provides a nonparametric estimation procedure attaining the optimal minimax rate of convergence under hellinger loss if the posterior distribution achieves the optimal rate over certain uniformity classes a prior on the density class of interest is induced by a prior on the coefficients of the trigonometric series expansion of the log density we show that when p is known the posterior distribution of a gaussian prior achieves the optimal rate provided the prior variances die off sufficiently rapidly for a mixture of normal distributions the mixing weights on the dimension of the exponential family are assumed to be bounded below by an exponentially decreasing sequence to avoid the use of infinite bases we develop priors that cut off the series at a sample size dependent truncation point when the degree of smoothness is unknown a finite mixture of normal priors indexed by the smoothness parameter which is also assigned a prior produces the best rate a rate adaptive estimator is derived

rates of contraction of posterior distributions based on gaussian process priors we derive rates of contraction of posterior distributions on nonparametric or semiparametric models based on gaussian processes the rate of contraction is shown to depend on the position of the true parameter relative to the reproducing kernel hilbert space of the gaussian process and the small ball probabilities of the gaussian process we determine these quantities for a range of examples of gaussian priors and in several statistical settings for instance we consider the rate of contraction of the posterior distribution based on sampling from a smooth density model when the prior models the log density as a fractionally integrated brownian motion we also consider regression with gaussian errors and smooth classification under a logistic or probit link function combined with various priors

switching observer design for uncertain nonlinear systems the problem of designing a switching observer for a finite or infinite set of nonlinear time varying systems with unknown parameters and or varying structure is studied in this paper under a compactness condition and a lyapunov detectability assumption it is shown that a switching observer with asymptotically stable error dynamics can be constructed different from the variable structure control method only a finite number of switches occur before the asymptotic stability is achieved the lyapunov direct method is used to construct both the switching gain and the switching index the special case where the error dynamics becomes exponentially convergent with quadratic lyapunov detectability is also examined to illustrate the proposed observer design procedure

convergence rate of sieve estimates in this paper we develop a general theory for the convergence rate of sieve estimates maximum likelihood estimates mle s and related estimates obtained by optimizing certain empirical criteria in general parameter spaces in many cases especially when the parameter space is infinite dimensional maximization over the whole parameter space is undesirable in such cases one has to perform maximization over an approximating space sieve of the original parameter space and allow the size of the approximating space to grow as the sample size increases this method is called the method of sieves in the case of the maximum likelihood estimation an mle based on a sieve is called a sieve mle we found that the convergence rate of a sieve estimate is governed by a the local expected values variances and l 2 entropy of the criterion differences and b the approximation error of the sieve a robust nonparametric regression problem a mixture problem and a nonparametric regression problem are discussed as illustrations of the theory we also found that when the underlying space is too large the estimate based on optimizing over the whole parameter space may not achieve the best possible rates of convergence whereas the sieve estimate typically does not suffer from this difficulty

an algorithm for maximizing expected log investment return let the random stock market vector x geq 0 be drawn according to a known distribution function f x x in r m a log optimal portfolio b ast is any portfolio b achieving maximal expected log return w ast sup b e ln b t x where the supremum is over the simplex b geq 0 sum i 1 m b i 1 an algorithm is presented for finding b ast the algorithm consists of replacing the portfolio b by the expected portfolio b b i e b i x i b t x corresponding to the expected proportion of holdings in each stock after one market period the improvement in w b after each iteration is lower bounded by the kullback leibler information number d b b between the current and updated portfolios thus the algorithm monotonically improves the return w an upper bound on w ast is given in terms of the current portfolio and the gradient and the convergence of the algorithm is established

meta analysis with fixed unknown study specific parameters meta analysis is a valuable tool for combining information from independent studies however most common meta analysis techniques rely on distributional assumptions that are difficult if not impossible to verify for instance in the commonly used fixed effects and random effects models we take for granted that the underlying study level parameters are either exactly the same across individual studies or that they are realizations of a random sample from a population often under a parametric distributional assumption in this article we present a new framework for summarizing information obtained from multiple studies and make inference that is not dependent on any distributional assumption for the study level parameters specifically we assume the study level parameters are unknown fixed parameters and draw inferences about for example the quantiles of this set of parameters using study specific summary statistics this type of problem is known to be quite challenging see hall and miller we

rates of contraction for posterior distributions in lr metrics 1 r the frequentist behavior of nonparametric bayes estimates more specifically rates of contraction of the posterior distributions to shrinking lr norm neighborhoods 1 r of the unknown parameter are studied a theorem for nonparametric density estimation is proved under general approximation theoretic assumptions on the prior the result is applied to a variety of common examples including gaussian process wavelet series normal mixture and histogram priors the rates of contraction are minimax optimal for 1 r 2 but deteriorate as r increases beyond 2 in the case of gaussian nonparametric regression a gaussian prior is devised for which the posterior contracts at the optimal rate in all lr norms 1 r

on rates of convergence for posterior distributions in infinite dimensional models this paper introduces a new approach to the study of rates of convergence for posterior distributions it is a natural extension of a recent approach to the study of bayesian consistency in particular we improve on current rates of convergence for models including the mixture of dirichlet process model and the random bernstein polynomial model

bayesian regression tree ensembles that adapt to smoothness and sparsity ensembles of decision trees are a useful tool for obtaining flexible estimates of regression functions examples of these methods include gradient boosted decision trees random forests and bayesian classification and regression trees two potential shortcomings of tree ensembles are their lack of smoothness and their vulnerability to the curse of dimensionality we show that these issues can be overcome by instead considering sparsity inducing soft decision trees in which the decisions are treated as probabilistic we implement this in the context of the bayesian additive regression trees framework and illustrate its promising performance through testing on benchmark data sets we provide strong theoretical support for our methodology by showing that the posterior distribution concentrates at the minimax rate up to a logarithmic factor for sparse functions and functions with additive structures in the high dimensional regime where the dimensionality of the covariate space is allowed to grow nearly exponentially in the sample size our method also adapts to the unknown smoothness and sparsity levels and can be implemented by making minimal modifications to existing bayesian additive regression tree algorithms

rates of convergence of posterior distributions we compute the rate at which the posterior distribution concentrates around the true parameter value the spaces we work in are quite general and include infinite dimensional cases the rates are driven by two quantities the size of the space as measured by bracketing entropy and the degree to which the prior concentrates in a small ball around the true parameter we consider two examples

numerical methods based on rational variable substitution for wiener hopf equations of the second kind this paper considers numerical methods for wiener hopf equations of the second kind y t 0 k t s y s ds g t 0 t by applying rational variable substitution to integrals on the semi infinite interval 0 and using the well known clenshaw curtis quadrature to the resulted integral we get a clenshaw curtis rational ccr quadrature rule we then apply the ccr quadrature to wiener hopf equations the reduction of singularities in the transformed equation is considered numerical examples are given to illustrate the efficiency of the numerical methods proposed in this paper

optimal shrinkage estimation of mean parameters in family of distributions with quadratic variance this paper discusses the simultaneous inference of mean parameters in a family of distributions with quadratic variance function we first introduce a class of semi parametric parametric shrinkage estimators and establish their asymptotic optimality properties two specific cases the location scale family and the natural exponential family with quadratic variance function are then studied in detail we conduct a comprehensive simulation study to compare the performance of the proposed methods with existing shrinkage estimators we also apply the method to real data and obtain encouraging results

semiparametric estimation of normal mixture densities a semiparametric method for estimating densities of normal mean mixtures is presented this consistent data driven method of estimation is based on probability spacings the estimation technique involves iteratively fixing the standard deviation of the normal kernel that serves as a smoothing parameter and then maximizing a function of the probability spacings over all mixing distributions based on the distribution of uniform spacings a distribution free goodness of fit criterion is developed to guide the selection of the smoothing parameter the result is a set of consistent estimators indexed by a range of smoothing parameters empirical process results are used to prove consistency

parametric empirical bayes inference theory and applications abstract this article reviews the state of multiparameter shrinkage estimators with emphasis on the empirical bayes viewpoint particularly in the case of parametric prior distributions some successful applications of major importance are considered recent results concerning estimates of error and confidence intervals are described and illustrated with data

high order corrected trapezoidal quadrature rules for singular functions a class of quadrature formulae is presented applicable to both nonsingular and singular functions generalizing the classical endpoint corrected trapezoidal quadrature rules while the latter rules are usually derived by means of the euler maclaurin formula their generalizations are obtained as solutions of certain systems of linear algebraic equations a procedure is developed for the construction of very high order quadrature rules applicable to functions with a priori specified singularities and relaxing the requirements on the distribution of nodes the scheme applies not only to nonsingular functions but also to a wide class of functions with monotonic singularities numerical experiments are presented demonstrating the practical usefulness of the new class of quadratures tables of quadrature weights are included for singularities of the form s x x lambda for a variety of values of lambda and s x log x

convex optimization shape constraints compound decisions and empirical bayes rules estimation of mixture densities for the classical gaussian compound decision problem and their associated empirical bayes rules is considered from two new perspectives the first motivated by brown and greenshtein introduces a nonparametric maximum likelihood estimator of the mixture density subject to a monotonicity constraint on the resulting bayes rule the second motivated by jiang and zhang proposes a new approach to computing the kiefer wolfowitz nonparametric maximum likelihood estimator for mixtures in contrast to prior methods for these problems our new approaches are cast as convex optimization problems that can be efficiently solved by modern interior point methods in particular we show that the reformulation of the kiefer wolfowitz estimator as a convex optimization problem reduces the computational effort by several orders of magnitude for typical problems by comparison to prior em algorithm based methods and thus greatly expands the practical applicability of the resulting method

on sure type double shrinkage estimation abstractthe article is concerned with empirical bayes shrinkage estimators for the heteroscedastic hierarchical normal model using stein s unbiased estimate of risk sure recently xie kou and brown proposed a class of estimators for this type of problems and established their asymptotic optimality properties under the assumption of known but unequal variances in this article we consider this problem with unequal and unknown variances which may be more appropriate in real situations by placing priors for both means and variances we propose novel sure type double shrinkage estimators that shrink both means and variances optimal properties for these estimators are derived under certain regularity conditions extensive simulation studies are conducted to compare the newly developed methods with other shrinkage techniques finally the methods are applied to the well known baseball dataset and a gene expression dataset supplementary materials for this article are available online

observer design for nonlinear triangular systems with unobservable linearisation the article deals with the observer design problem for a wide class of triangular time varying nonlinear systems with unobservable linearisation sufficient conditions are derived for the existence of a luenberger type observer when it is a priori known that the initial state of the system belongs to a given nonempty bounded subset of the state space for the general case the state estimation is exhibited by means of a switching sequence of time varying dynamics

probability inequalities for likelihood ratios and convergence rates of sieve mles let y 1 y n be independent identically distributed with density p 0 and let f be a space of densities we show that the supremum of the likelihood ratios i 1 n p y i p 0 y i where the supremum is over p f with p l 2 p 0 1 2 2 e is exponentially small with probability exponentially close to 1 the exponent is proportional to ne 2 the only condition required for this to hold is that e exceeds a value determined by the bracketing hellinger entropy of f a similar inequality also holds if we replace f by f n and p 0 by q n where q n is an approximation to p 0 in a suitable sense these results are applied to establish rates of convergence of sieve mles furthermore weak conditions are given under which the optimal rate e n defined by h e n f ne n 2 where h f is the hellinger entropy of f is nearly achievable by sieve estimators

the local geometry of finite mixtures we introduce a technique to obtain local bracketing metric en tropy bounds for subsets of a normed vector space from global entropy bounds using this method we establish that for q 1 the class of convex combina tions of q translates of a probability density has finite local doubling dimension under a smoothness assumption the proof requires a detailed investigation of the local geometry of mixture classes which is of independent interest

the risk inflation criterion for multiple regression a new criterion is proposed for the evaluation of variable selection procedures in multiple regression this criterion which we call the risk inflation is based on an adjustment to the risk essentially the risk inflation is the maximum increase in risk due to selecting rather than knowing the correct predictors a new variable selection procedure is obtained which in the case of orthogonal predictors substantially improves on aic c p and bic and is close to optimal in contrast to aic c p and bic which use dimensionality penalties of 2 2 and log n respectively this new procedure uses a penalty 2 log p where p is the number of available predictors for the case of nonorthogonal predictors bounds for the optimal penalty are obtained

quadrature rule for indefinite integral of algebraic logarithmic singular integrands an automatic quadrature method is presented for approximating the indefinite integral of functions having algebraic logarithmic singularities q x y c f email protected x yf t t c alog t c dt 1 1 within a finite range 1 1 for some smooth function f t that is approximated by a finite sum of chebyshev polynomials we expand the given indefinite integral in terms of chebyshev polynomials by using auxiliary algebraic logarithmic functions present scheme approximates the indefinite integral q x y c f uniformly namely bounds the approximation error independently of the value c as well x and y this fact enables us to evaluate the integral transform q x y c f with varied values of x y and c efficiently some numerical examples illustrate the performance of the present quadrature scheme

application of non parametric empirical bayes estimation to high dimensional classification we consider the problem of classification using high dimensional features space in a paper by bickel and levina 2004 it is recommended to use naive bayes classifiers that is to treat the features as if they are statistically independent consider now a sparse setup where only a few of the features are informative for classification fan and fan 2008 suggested a variable selection and classification method called fair the fair method improves the design of naive bayes classifiers in sparse setups the improvement is due to reducing the noise in estimating the features means this reduction is since that only the means of a few selected variables should be estimated we also consider the design of naive bayes classifiers we show that a good alternative to variable selection is estimation of the means through a certain non parametric empirical bayes procedure in sparse setups the empirical bayes implicitly performs an efficient variable selection it also adapts very well to non sparse setups and has the advantage of making use of the information from many weakly informative variables which variable selection type of classification procedures give up on using we compare our method with fair and other classification methods in simulation for sparse and non sparse setups and in real data examples involving classification of normal versus malignant tissues based on microarray data

minimax risk over l p balls for l p error consider estimating the mean vector from datan n 2 i withl q norm loss q 1 when is known to lie in ann dimensionall p ball p 0 for largen the ratio of minimaxlinear risk to minimax risk can bearbitrarily large ifp q obvious exceptions aside the limiting ratio equals 1 only ifp q 2 our arguments are mostly indirect involving a reduction to a univariate bayes minimax problem whenp q simple non linear co ordinatewise threshold rules are asymptotically minimax at small signal to noise ratios and within a bounded factor of asymptotic minimaxity in general we also give asymptotic evaluations of the minimax linear risk our results are basic to a theory of estimation in besov spaces using wavelet bases to appear elsewhere

on choosing mixture components via non local priors choosing the number of mixture components remains an elusive challenge model selection criteria can be either overly liberal or conservative and return poorly separated components of limited practical use we formalize non local priors nlps for mixtures and show how they lead to well separated components with non negligible weight interpretable as distinct subpopulations we also propose an estimator for posterior model probabilities under local priors and nlps showing that bayes factors are ratios of posterior to prior empty cluster probabilities the estimator is widely applicable and helps to set thresholds to drop unoccupied components in overfitted mixtures we suggest default prior parameters based on multimodality for normal t mixtures and minimal informativeness for categorical outcomes we characterize theoretically the nlp induced sparsity derive tractable expressions and algorithms we fully develop normal binomial and product binomial mixtures but the theory computation and principles hold more generally we observed a serious lack of sensitivity of the bayesian information criterion insufficient parsimony of the akaike information criterion and a local prior and a mixed behaviour of the singular bayesian information criterion we also considered overfitted mixtures their performance was competitive but depended on tuning parameters under our default prior elicitation nlps offered a good compromise between sparsity and power to detect meaningfully separated components

from image deblurring to optimal investments maximum likelihood solutions for positive linear inverse problems the problem of recovering an input signal from a blurred output in an input output system with linear distortion is ubiquitous in science and technology when the blurred output is not degraded by statistical noise the problem is entirely deterministic and amounts to a mathematical inversion of a linear system with positive parameters subject to positivity constraints on the solution we show that all such linear inverse problems with positivity restrictions lininpos problems for short can be interpreted as statistical estimation problems from incomplete data based on infinitely large samples and that maximum likelihood ml estimation and the em algorithm provide a straightforward method of solution for such problems

bayesian estimation of the spectral density of a time series this article describes a bayesian approach to estimating the spectral density of a stationary time series a nonparametric prior on the spectral density is described through bernstein polynomials because the actual likelihood is very complicated a pseudoposterior distribution is obtained by updating the prior using the whittle likelihood a markov chain monte carlo algorithm for sampling from this posterior distribution is described that is used for computing the posterior mean variance and other statistics a consistency result is established for this pseudoposterior distribution that holds for a short memory gaussian time series and under some conditions on the prior to prove this asymptotic result a general consistency theorem of schwartz is extended for a triangular array of independent nonidentically distributed observations this extension is also of independent interest a simulation study is conducted to compare the proposed method with some existing methods the method is illustrated with

modeling unobserved sources of heterogeneity in animal abundance using a dirichlet process prior in surveys of natural populations of animals a sampling protocol is often spatially replicated to collect a representative sample of the population in these surveys differences in abundance of animals among sample locations may induce spatial heterogeneity in the counts associated with a particular sampling protocol for some species the sources of heterogeneity in abundance may be unknown or unmeasurable leading one to specify the variation in abundance among sample locations stochastically however choosing a parametric model for the distribution of unmeasured heterogeneity is potentially subject to error and can have profound effects on predictions of abundance at unsampled locations in this article we develop an alternative approach wherein a dirichlet process prior is assumed for the distribution of latent abundances this approach allows for uncertainty in model specification and for natural clustering in the distribution of abundances in a data adaptive way we apply this approach in an analysis of counts based on removal samples of an endangered fish species the okaloosa darter results of our data analysis and simulation studies suggest that our implementation of the dirichlet process prior has several attractive features not shared by conventional fully parametric alternatives

general maximum likelihood empirical bayes estimation of normal means we propose a general maximum likelihood empirical bayes gmleb method for the estimation of a mean vector based on observations with i i d normal errors we prove that under mild moment conditions on the unknown means the average mean squared error mse of the gmleb is within an infinitesimal fraction of the minimum average mse among all separable estimators which use a single deterministic estimating function on individual observations provided that the risk is of greater order than log n 5 n we also prove that the gmleb is uniformly approximately minimax in regular and weak l p balls when the order of the length normalized norm of the unknown means is between log n k1 n 1 p 2 and n log n k2 simulation experiments demonstrate that the gmleb outperforms the james stein and several state of the art threshold estimators in a wide range of settings without much down side

volterra type integral equation method for the radial schrodinger equation single channel case a new volterra type integral equation method for the numerical solution of the single channel radial schrodinger equation is investigated the method carried out in configuration space is based on the conversion of differential equations into a system of volterra type integral equations together with the application of a spectral type clenshaw curtis quadrature through numerical examples the volterra type integral equation method is shown to be more efficient and more accurate than the integral equation method based on the fredholm formulation 1 and better than finite difference methods accompanying c code for the volterra type method is available upon request

adaptive bayesian multivariate density estimation with dirichlet mixtures summary we show that rate adaptive multivariate density estimation can be performed using bayesian methods based on dirichlet mixtures of normal kernels with a prior distribution on the kernel s covariance matrix parameter we derive sufficient conditions on the prior specification that guarantee convergence to a true density at a rate that is minimax optimal for the smoothness class to which the true density belongs no prior knowledge of smoothness is assumed the sufficient conditions are shown to hold for the dirichlet location mixture of normals prior with a gaussian base measure and an inverse wishart prior on the covariance matrix parameter locally h older

adapting to unknown smoothness via wavelet shrinkage abstract we attempt to recover a function of unknown smoothness from noisy sampled data we introduce a procedure sureshrink that suppresses noise by thresholding the empirical wavelet coefficients the thresholding is adaptive a threshold level is assigned to each dyadic resolution level by the principle of minimizing the stein unbiased estimate of risk sure for threshold estimates the computational effort of the overall procedure is order n log n as a function of the sample size n sureshrink is smoothness adaptive if the unknown function contains jumps then the reconstruction essentially does also if the unknown function has a smooth piece then the reconstruction is essentially as smooth as the mother wavelet will allow the procedure is in a sense optimally smoothness adaptive it is near minimax simultaneously over a whole interval of the besov scale the size of this interval depends on the choice of mother wavelet we know from a previous paper by the authors that traditional smoot

a dirichlet process gaussian state machine model for change detection in transient processes abstractthe ability to detect incipient and critical changes in real world process esessential for system integrity assurance is currently impeded by the mismatch between the key assumption of stationarity underlying most change detection methods and the nonlinear and nonstationary transient dynamics of most real world processes the current approaches are slow or outright unable to detect qualitative changes in the behaviors that lead to anomalies we present a dirichlet process gaussian state machine dpgsm model to represent dynamic intermittency which is one of the most ubiquitous real world transient behaviors the dpgsm model treats a signal as a random walk among a dirichlet process mixture of gaussian clusters hypothesis tests and a numerical scheme based on this nonparametric representation were developed to detect subtle changes in the transient intermittent dynamics experimental investigations suggest that the dpgsm approach can consistently detect incipient critical changes in intermi

bayesian density estimation and inference using mixtures abstract we describe and illustrate bayesian inference in models for density estimation using mixtures of dirichlet processes these models provide natural settings for density estimation and are exemplified by special cases where data are modeled as a sample from mixtures of normal distributions efficient simulation methods are used to approximate various prior posterior and predictive distributions this allows for direct inference on a variety of practical issues including problems of local versus global smoothing uncertainty about density estimates assessment of modality and the inference on the numbers of components also convergence results are established for a general class of normal mixture models

bayesian repulsive gaussian mixture model we develop a general class of bayesian repulsive gaussian mixture models that encourage well separated clusters aiming at reducing potentially redundant components produced by independent priors f

the consistency of posterior distributions in nonparametric problems we give conditions that guarantee that the posterior probability of every hellinger neighborhood of the true distribution tends to 1 almost surely the conditions are 1 a requirement that the prior not put high mass near distributions with very rough densities and 2 a requirement that the prior put positive mass in kullback leibler neighborhoods of the true distribution the results are based on the idea of approximating the set of distributions with a finite dimensional set of distributions with sufficiently small hellinger bracketing metric entropy we apply the results to some examples

practical bayesian density estimation using mixtures of normals abstract mixtures of normals provide a flexible model for estimating densities in a bayesian framework there are some difficulties with this model however first standard reference priors yield improper posteriors second the posterior for the number of components in the mixture is not well defined if the reference prior is used third posterior simulation does not provide a direct estimate of the posterior for the number of components we present some practical methods for coping with these problems finally we give some results on the consistency of the method when the maximum number of components is allowed to grow with the sample size

irrational exuberance correcting bias in probability estimates we consider the common setting where one observes probability estimates for a large number of events such as default risks for numerous bonds unfortunately even with unbiased estimates selectin

locally corrected multidimensional quadrature rules for singular functions accurate numerical integration of singular functions usually requires either adaptivity or product integration both interfere with fast summation techniques and thus hamper large scale computations this paper presents a method for computing highly accurate quadrature formulas for singular functions which combine well with fast summation methods given the singularity and the n nodes we first construct weights which integrate smooth functions with order k accuracy then we locally correct a small number of weights near the singularity to achieve order k accuracy on singular functions as well the method is highly efficient and runs in o nk 2d n log 2 n time and o k 2d n space we derive precise error bounds and time estimates and confirm them with numerical results which demonstrate the accuracy and efficiency of the method in large scale computations as part of our implementation we also construct a new adaptive multidimensional product gauss quadrature routine with an effective error

posterior convergence rates of dirichlet mixtures at smooth densities we study the rates of convergence of the posterior distribution for bayesian density estimation with dirichlet mixtures of normal distributions as the prior the true density is assumed to be twice continuously differentiable the bandwidth is given a sequence of priors which is obtained by scaling a single prior by an appropriate order in order to handle this problem we derive a new general rate theorem by considering a countable covering of the parameter space whose prior probabilities satisfy a summability condition together with certain individual bounds on the hellinger metric entropy we apply this new general theorem on posterior convergence rates by computing bounds for hellinger bracketing entropy numbers for the involved class of densities the error in the approximation of a smooth density by normal mixtures and the concentration rate of the prior the best obtainable rate of convergence of the posterior turns out to be equivalent to the well known frequentist rate for integrated mean squared error n 2 5 up to a logarithmic factor

an empirical bayes method for chi squared data in a thought provoking paper efron investigated the merit and limitation of an empirical bayes method to correct selection bias based on tweedie s formula first reported in the study by robbins t

the poisson compound decision problem revisited the compound decision problem for a vector of independent poisson random variables with possibly different means has a half century old solution however it appears that the classical solution needs smoothing adjustment we discuss three such adjustments we also present another approach that first transforms the problem into the normal compound decision problem a simulation study shows the effectiveness of the procedures in improving the performance over that of the classical procedure a real data example is also provided the procedures depend on a smoothness parameter that can be selected using a nonstandard cross validation step which is of independent interest finally we mention some asymptotic results

nystrom clenshaw curtis quadrature for integral equations with discontinuous kernels a new highly accurate numerical approximation scheme based on a gauss type clenshaw curtis quadrature for fredholm integral equations of the second kind x t ab k t s x s ds y t whose kernel k t s is either discontinuous or not smooth along the main diagonal is presented this scheme is of spectral accuracy when k t s is infinitely differentiable away from the diagonal t s relation to the singular value decomposition is indicated application to integro differential schrodinger equations with nonlocal potentials is given

asymptotics of discrete mdl for online prediction minimum description length mdl is an important principle for induction and prediction with strong relations to optimal bayesian learning this paper deals with learning processes which are independent and identically distributed i i d by means of two part mdl where the underlying model class is countable we consider the online learning framework i e observations come in one by one and the predictor is allowed to update its state of mind after each time step we identify two ways of predicting by mdl for this setup namely a static and a dynamic one a third variant hybrid mdl will turn out inferior we will prove that under the only assumption that the data is generated by a distribution contained in the model class the mdl predictions converge to the true values almost surely this is accomplished by proving finite bounds on the quadratic the hellinger and the kullback leibler loss of the mdl learner which are however exponentially worse than for bayesian prediction we demonstrate that these bounds are sharp even for model classes containing only bernoulli distributions we show how these bounds imply regret bounds for arbitrary loss functions our results apply to a wide range of setups namely sequence prediction pattern classification regression and universal induction in the sense of algorithmic information theory among others

on nonlinear observers in this note we make some remarks about the general observer problem for nonlinear systems we show how the existence of a partial observer without stability requirements is locally equivalent to the existence of a conditioned invariant distribution we argue that by considering not only identity observers we can put the stability issue in a broader perspective

controlling the false discovery rate a practical and powerful approach to multiple testing summary the common approach to the multiplicity problem calls for controlling the familywise error rate fwer this approach though has faults and we point out a few a different approach to problems of multiple significance testing is presented it calls for controlling the expected proportion of falsely rejected hypotheses the false discovery rate this error rate is equivalent to the fwer when all hypotheses are true but is smaller otherwise therefore in problems where the control of the false discovery rate rather than that of the fwer is desired there is potential for a gain in power a simple sequential bonferronitype procedure is proved to control the false discovery rate for independent test statistics and a simulation study shows that the gain in power is substantial the use of the new procedure and the appropriateness of the criterion are illustrated with examples

nonparametric maximum likelihood estimation by the method of sieves maximum likelihood estimation often fails when the parameter takes values in an infinite dimensional space for example the maximum likelihood method cannot be applied to the completely nonparametric estimation of a density function from an operatorname iid sample the maximum of the likelihood is not attained by any density in this example as in many other examples the parameter space positive functions with area one is too big but the likelihood method can often be salvaged if we first maximize over a constrained subspace of the parameter space and then relax the constraint as the sample size grows this is grenander s method of sieves application of the method sometimes leads to new estimators for familiar problems or to a new motivation for an already well studied technique we will establish some general consistency results for the method and then we will focus on three applications

convergence rates of posterior distributions we consider the asymptotic behavior of posterior distributions and bayes estimators for infinite dimensional statistical models we give general results on the rate of convergence of the posterior measure these are applied to several examples including priors on finite sieves log spline models dirichlet processes and interval censoring

on the bayes optimality of f measure maximizers the f measure which has originally been introduced in information retrieval is nowadays routinely used as a performance metric for problems such as binary classification multi label classification and structured output prediction optimizing this measure is a statistically and computationally challenging problem since no closed form solution exists adopting a decision theoretic perspective this article provides a formal and experimental analysis of different approaches for maximizing the f measure we start with a bayes risk analysis of related loss functions such as hamming loss and subset zero one loss showing that optimizing such losses as a surrogate of the f measure leads to a high worst case regret subsequently we perform a similar type of analysis for f measure maximizing algorithms showing that such algorithms are approximate while relying on additional assumptions regarding the statistical distribution of the binary response variables furthermore we present a new algorithm which is not only computationally efficient but also bayes optimal regardless of the underlying distribution to this end the algorithm requires only a quadratic with respect to the number of binary responses number of parameters of the joint distribution we illustrate the practical performance of all analyzed methods by means of experiments with multi label classification problems

rates of convergence for the gaussian mixture sieve gaussian mixtures provide a convenient method of densityestimation that lies somewhere between parametric models and kernel densityestimators when the number of components of the mixture is allowed to increase as sample size increases the model is called a mixture sieve we establish a bound on the rate of convergence in hellinger distance for densityestimation using the gaussian mixture sieve assuming that the true density is itself a mixture of gaussians the underlying mixing measure of the true densityis not necessarilyassumed to have finite support computing the rate involves some delicate calculations since the size of the sieve as measured bybracketing entropy and the saturation rate cannot be found using standard methods when the mixing measure has compact support using kn n 2 3 log n 1 3 components in the mixture yields a rate of order log n 1 6 n1 6 for every 0 the rates depend heavilyon the tail behavior of the true density the sensitivity to the tail behavior is diminished byusing a robust sieve which includes a long tailed component in the

convergence rates for density estimation with bernstein polynomials mixture models for density estimation provide a very useful set up for the bayesian or the maximum likelihood approach for a density on the unit interval mixtures of beta densities form a flexible model the class of bernstein densities is a much smaller subclass of the beta mixtures defined by bernstein polynomials which can approximate any continuous density a bernstein polynomial prior is obtained by putting a prior distribution on the class of bernstein densities the posterior distribution of a bernstein polynomial prior is consistent under very general conditions in this article we present some results on the rate of convergence of the posterior distribution if the underlying distribution generating the data is itself a bernstein density then we show that the posterior distribution converges at nearly parametric rate log n sqrt n for the hellinger distance if the true density is not of the bernstein type we show that the posterior converges at a rate n 1 3 log n 5 6 provided that the true density is twice differentiable and bounded away from 0 similar rates are also obtained for sieve maximum likelihood estimates these rates are inferior to the pointwise convergence rate of a kernel type estimator we show that the bayesian bootstrap method gives a proxy for the posterior distribution and has a convergence rate at par with that of the kernel estimator

convergence rates of posterior distributions for non i i d observations we consider the asymptotic behavior of posterior distributions and bayes estimators based on observations which are required to be neither independent nor identically distributed we give general results on the rate of convergence of the posterior measure relative to distances derived from a testing criterion we then specialize our results to independent nonidentically distributed observations markov processes stationary gaussian time series and the white noise model we apply our general results to several examples of infinite dimensional statistical models including nonparametric regression with normal errors binary regression poisson regression an interval censoring model whittle estimation of the spectral density of a time series and a nonlinear autoregressive model

integral equation method for the continuous spectrum radial schrodinger equation a new approach to the numerical solution of boundary value problems for differential equations which originated in recent papers by greengard and rokhlin is improved and adapted to the numerical solution of the radial schrodinger equation the approach is based on the conversion of the differential equation into an integral equation together with the application of a spectral type clenshaw curtis quadrature method through numerical examples the integral equation method is shown to be superior to finite difference methods