information theoretic determination of minimax rates of convergence we present some general results determining minimax bounds on statistical risk for density estimation based on certain information theoretic considerations these bounds depend only on metric entropy conditions and are used to identify the minimax rates of convergence

introduction to matrix analytic methods in stochastic modeling preface part i quasi birth and death processes 1 examples part ii the method of phases 2 ph distributions 3 markovian point processes part iii the matrix geometric distribution 4 birth and death processes 5 processes under a taboo 6 homogeneous qbds 7 stability condition part iv algorithms 8 algorithms for the rate matrix 9 spectral analysis 10 finite qbds 11 first passage times part v beyond simple qbds 12 nonhomogeneous qbds 13 processes skip free in one direction 14 tree processes 15 product form networks 16 nondenumerable states bibliography index

nonparametric least squares estimation of a multivariate convex regression function this paper deals with the consistency of the least squares estimator of a convex regression function when the predictor is multidimensional we characterize and discuss the computation of such an estimator via the solution of certain quadratic and linear programs mild sufficient conditions for the consistency of this estimator and its subdifferentials in fixed and stochastic design regression settings are provided we also consider a regression function which is known to be convex and componentwise nonincreasing and discuss the characterization computation and consistency of its least squares estimator

nonparametric estimation of multivariate convex transformed densities we study estimation of multivariate densities p of the form p x h g x for d and for a fixed monotone function h and an unknown convex function g the canonical example is h y e y for y r in this case the resulting class of densities p e y p exp g g is convex is well known as the class of log concave densities other functions h allow for classes of densities with heavier tails than the log concave class we first investigate when the maximum likelihood estimator p exists for the class p h for various choices of monotone transformations h including decreasing and increasing functions h the resulting models for increasing transformations h extend the classes of log convex densities studied previously in the econometrics literature corresponding to h y exp y we then establish consistency of the maximum likelihood estimator for fairly general functions h including the log concave class p e y and many others in a final section we provide asymptotic minimax lower bounds for the estimation of p and its vector of derivatives at a fixed point x 0 under natural smoothness hypotheses on h and g the proofs rely heavily on results from convex analysis

global risk bounds and adaptation in univariate convex regression we consider the problem of nonparametric estimation of a convex regression function phi 0 we study the risk of the least squares estimator lse under the natural squared error loss we show that the risk is always bounded from above by n 4 5 modulo logarithmic factors while being much smaller when phi 0 is well approximable by a piecewise affine convex function with not too many affine pieces in which case the risk is at most 1 n up to logarithmic factors on the other hand when phi 0 has curvature we show that no estimator can have risk smaller than a constant multiple of n 4 5 in a very strong sense by proving a local minimax lower bound we also study the case of model misspecification where we show that the lse exhibits the same global behavior provided the loss is measured from the closest convex projection of the true regression function in the process of deriving our risk bounds we prove new results for the metric entropy of local neighborhoods of the space of univariate convex functions these results which may be of independent interest demonstrate the non uniform nature of the space of univariate convex functions in sharp contrast to classical function spaces based on smoothness constraints

semiconvex regression for metamodeling based optimization stochastic search involves finding a set of controllable parameters that minimizes an unknown objective function using a set of noisy observations we consider the case when the unknown function is convex and a metamodel is used as a surrogate objective function often the data are non i i d and include an observable state variable such as applicant information in a loan rate decision problem state information is difficult to incorporate into convex models we propose a new semiconvex regression method that is used to produce a convex metamodel in the presence of a state variable we show consistency for this method we demonstrate its effectiveness for metamodeling on a set of synthetic inventory management problems and a large real life auto loan dataset

maximum likelihood estimation of a multi dimensional log concave density fn is shown to have smaller mean integrated squared error compared with kernel based methods even when we allow the use of a theoret ical optimal fixed bandwidth for the kernel estimator that would not be available in practice we also present a real data clustering example which shows that our methodology can be used in conjunction with the expectation maximization algorithm to fit finite mixtures of log concave densities

rates of convergence for minimum contrast estimators we shall present here a general study of minimum contrast estimators in a nonparametric setting although our results are also valid in the classical parametric case for independent observations these estimators include many of the most popular estimators in various situations such as maximum likelihood estimators least squares and other estimators of the regression function estimators for mixture models or deconvolution the main theorem relates the rate of convergence of those estimators to the entropy structure of the space of parameters optimal rates depending on entropy conditions are already known at least for some of the models involved and they agree with what we get for minimum contrast estimators as long as the entropy counts are not too large but under some circumstances large entropies or changes in the entropy structure due to local perturbations the resulting the rates are only suboptimal counterexamples are constructed which show that the phenomenon is real for non parametric maximum likelihood or regression this proves that under purely metric assumptions our theorem is optimal and that minimum contrast estimators happen to be suboptimal

approximations of semicontinuous functions with applications to stochastic optimization and statistical estimation upper semicontinuous usc functions arise in the analysis of maximization problems distributionally robust optimization and function identification which includes many problems of nonparametric statistics we establish that every usc function is the limit of a hypo converging sequence of piecewise affine functions of the difference of max type and illustrate resulting algorithmic possibilities in the context of approximate solution of infinite dimensional optimization problems in an effort to quantify the ease with which classes of usc functions can be approximated by finite collections we provide upper and lower bounds on covering numbers for bounded sets of usc functions under the attouch wets distance the result is applied in the context of stochastic optimization problems defined over spaces of usc functions we establish confidence regions for optimal solutions based on sample average approximations and examine the accompanying rates of convergence examples from nonparametric statistics illustrate the results

on density estimation in the view of kolmogorov s ideas in approximation theory the paper deals with upper and lower bounds for the quality of probability density estimation connections are established between these problems and the theory of approximation of functions particularly it is demonstrated how some of kolmogorov s concepts work

approximation dans les espaces metriques et theorie de l estimation we investigate the relations between the speed of estimation and the metric structure of the parameter space especially in the case when its metric dimension is infinite given some distance d on generally hellinger distance in the case of n i i d variables we consider the minimax risk for n observations r n q mathop inf sup limits t n theta in theta text ie theta d q theta t n t n being any estimate of we shall look for functions r such that for positive constants c 1 q and c 2 q c 1 r q n r n q c2 r q n r n is the speed of estimation and we shall show under fairly general conditions including i i d variables and regular cases of markov chains and stationnary gaussian processes that r n is determined up to multiplicative constants by the metric structure of we shall also give a construction for some sort of universal estimates the risk of which is bounded by c 2 r q n in all cases where the preceding theory applies

estimation of integral functionals of a density let be a smooth function of k 2 variables we shall investigate in this paper the rates of convergence of estimators of t f f f x f x f k x x dx when f belongs to some class of densities of smoothness s we prove that when 2k 1 4 one can define an estimator t n of t f based on n i i d observations of density f on the real line which converges at the semiparametric rate 1 n on the other hand when s 2k 1 4 t f cannot be estimated at a rate faster than n with 4 s k 4s 1 we shall also provide some extensions to the multidimensional case those results extend previous works of levit of bickel and ritov and of donoho and nussbaum on estimation of quadratic functionals

lower bounds for the minimax risk using f divergences and applications lower bounds involving f divergences between the underlying probability measures are proved for the minimax risk in estimation problems our proofs just use simple convexity facts special cases and straightforward corollaries of our bounds include well known inequalities for establishing minimax lower bounds such as fano s inequality pinsker s inequality and inequalities based on global entropy conditions two applications are provided a new minimax lower bound for the reconstruction of convex bodies from noisy support function measurements and a different proof of a recent minimax lower bound for the estimation of a covariance matrix

risk bounds for model selection via penalization performance bounds for criteria for model selection are devel oped using recent theory for sieves the model selection criteria are based on an empirical loss or contrast function with an added penalty term moti vated by empirical process theory and roughly proportional to the number of parameters needed to describe the model divided by the number of ob servations most of our examples involve density or regression estimation settings and we focus on the problem of estimating the unknown density or regression function we show that the quadratic risk of the minimum penal ized empirical contrast estimatoris bounded by an index of the accuracy of the sieve this accuracy index quanties the trade off among the candidate models between the approximation error and parameter dimension relative to sample size if we choose a list of models which exhibit good approximation prop erties with respect to different classes of smoothness the estimator can be simultaneously minimax rate optimal in each of those classes this is what is usually called adaptation the type of classes of smoothness in which one gets adaptation depends heavily on the list of models if too many models are involved in order to get accurate approximation of many wide classes of functions simultaneously it may happen that the estimator is only approx

convergence of pure and relaxed newton methods for solving a matrix polynomial equation arising in stochastic models abstract we consider a matrix polynomial equation which has the form of a n x n a n 1 x n 1 a 0 0 where a n a n 1 a 0 and x are square matrices assuming the positivity of coefficients from stochastic models the monotone convergence of newton s method for solving the equation is considered and we show that the elementwise minimal nonnegative solution can be found by the method with the zero starting matrix moreover the relaxed newton method preserving the monotonicity result is provided finally numerical experiments show that our method reduces the number of iterations significantly

on the joint signature of several coherent systems with some shared components the concept of joint signature js introduced by navarro samaniego and balakrishnan 2010 is a useful tool for investigating the joint reliability of two coherent systems with shared components in this article by considering several coherent systems which share some components with independent and identically distributed lifetimes we obtain a pseudo mixture representation for the joint distribution of the systems lifetimes based on a general notion of joint signature which is referred to as generalized joint signature gjs it is shown how the gjs is separated from the effect of the components lifetime distribution and this relationship helps us to represent the gjs as a two dimensional matrix instead of a high dimensional one based on the gjs some ordering results are obtained for comparing two clusters of coherent systems with some shared components several examples are provided to illustrate the results established here

probability inequalities for likelihood ratios and convergence rates of sieve mles let y 1 y n be independent identically distributed with density p 0 and let f be a space of densities we show that the supremum of the likelihood ratios i 1 n p y i p 0 y i where the supremum is over p f with p l 2 p 0 1 2 2 e is exponentially small with probability exponentially close to 1 the exponent is proportional to ne 2 the only condition required for this to hold is that e exceeds a value determined by the bracketing hellinger entropy of f a similar inequality also holds if we replace f by f n and p 0 by q n where q n is an approximation to p 0 in a suitable sense these results are applied to establish rates of convergence of sieve mles furthermore weak conditions are given under which the optimal rate e n defined by h e n f ne n 2 where h f is the hellinger entropy of f is nearly achievable by sieve estimators

on the mean residual life of a k out of n g system with a single cold standby component abstract the concept of mean residual life is one of the most important characteristics that has been widely used in dynamic reliability analysis it is a useful tool for investigating ageing properties of technical systems in this paper we define and study three different mean residual life functions for k out of n g system with a single cold standby component in particular we obtain explicit expressions for the corresponding functions using distributions of order statistics we also provide some stochastic ordering results associated with the lifetime of a system we illustrate the results for various lifetime distributions

approximation of density functions by sequences of exponential families 1 introduction consider the estimation of a probability density func tion p x defined on a bounded interval we approximate the logarithm of the density by a basis function expansion consisting of polynomials splines or trigonometric series the expansion yields a regular exponential family within which we estimate the density by the method of maximum likelihood this method of density estimation arises by application of the principle of maxi mum entropy or minimum relative entropy subject to empirical constraints we show that if the logarithm of the density has r square integrable deriva tives f idr log p12 ox then the sequence of density estimators p converges to p in the sense of relative entropy kullback leibler fp log p pn at rate opr 1 m2r m n as m x c and m2 n 0 in the spline and trigonometric cases and m3 n o 0 in the polynomial case where m is the dimension of the family and n is the sample size boundary conditions are assumed for the density in the trigonometric case this convergence rate specializes to op r n 2r 2r 1 by setting m nl 2r 1 when the log density is known to have degree of smoothness at least r analogous convergence results for the relative entropy are shown to hold in general for any class of log density functions and sequence of finite dimensional linear spaces having l2 and l approximation properties the approximation of log densities using polynomials has previously been considered by neyman 1937 to define alternatives for goodness of fit tests by good 1963 as an application of the method of maximum entropy or minimum relative entropy by crain 1974 1976a b 1977 who demonstrates existence and consistency of the maximum likelihood estimator and by mead and

a survey of network reliability and domination theory we present a brief survey of the current state of the art in network reliability we survey only exact methods and do not consider monte carlo methods most network reliability problems are in the worst case np hard and are in a sense more difficult than many standard combinatorial optimization problems nevertheless there are in fact linear and polynomial time algorithms for network reliability problems of special structure we review general methods for network reliability computation and discuss the central role played by domination theory in network reliability computational complexity we also point out the connection with the more general problem of computing the reliability of coherent structures the class of coherent structures contains both directed and undirected networks as well as logic or fault trees without not gates this topic is a rich area for further research

covering numbers for bounded variation functions abstract in this paper we provide upper and lower estimates for the minimal number of functions needed to represent a bounded variation function with an accuracy of epsilon with respect to l 1 distance

approximation of least squares regression on nested subspaces on presente une methode generale pour approximer le biais et la variance dans une echelle de normes hilbertiennes

approximation by log concave distributions with applications to regression we study the approximation of arbitrary distributions p on d dimensional space by distributions with log concave density approximation means minimizing a kullback leibler type functional we show that such an approximation exists if and only if p has finite first moments and is not supported by some hyperplane furthermore we show that this approximation depends continuously on p with respect to mallows distance d 1 this result implies consistency of the maximum likelihood estimator of a log concave density under fairly general conditions it also allows us to prove existence and consistency of estimators in regression models with a response y x e where x and e are independent belongs to a certain class of regression functions while e is a random error with log concave density and mean zero

dynamic network reliability modeling under nonhomogeneous poisson processes abstract in this paper we consider a two state up and down network consisting of n links we study the d spectrum based dynamic reliability of the network under the assumption that the links are subject to failure according to a nonhomogeneous poisson process several mixture representations are provided for the reliability function of residual lifetime of used networks under different conditions on the status of the network or its links these representations enable us to explore the residual reliability of operating networks in terms of the reliability functions of residual lifetimes of upper record values the distribution function of inactivity time of a network is examined under the condition that the network has failed by inspection time t stochastic ordering properties of the residual lifetimes of networks under conditional d spectra are investigated several examples and graphs are also provided to illustrate the established results

global rates of convergence in log concave density estimation the estimation of a log concave density on mathbb r d represents a central problem in the area of nonparametric inference under shape constraints in this paper we study the performance of log concave density estimators with respect to global loss functions and adopt a minimax approach we first show that no statistical procedure based on a sample of size n can estimate a log concave density with respect to the squared hellinger loss function with supremum risk smaller than order n 4 5 when d 1 and order n 2 d 1 when d geq2 in particular this reveals a sense in which when d geq3 log concave density estimation is fundamentally more challenging than the estimation of a density with two bounded derivatives a problem to which it has been compared second we show that for d leq3 the hellinger varepsilon bracketing entropy of a class of log concave densities with small mean and covariance matrix close to the identity grows like max varepsilon d 2 varepsilon d 1 up to a logarithmic factor when d 2 this enables us to prove that when d leq3 the log concave maximum likelihood estimator achieves the minimax optimal rate up to logarithmic factors when d 2 3 with respect to squared hellinger loss

mutual information metric entropy and cumulative relative entropy risk assume p theta theta epsilon theta is a set of probability distributions with a common dominating measure on a complete separable metric space y a state theta epsilon theta is chosen by nature a statistician obtains n independent observations y 1 dots y n from y distributed according to p theta for each time t between 1 and n based on the observations y 1 dots y t 1 the statistician produces an estimated distribution hat p t for p theta and suffers a loss l p theta hat p t the cumulative risk for the statistician is the average total loss up to time n of special interest in information theory data compression mathematical finance computational learning theory and statistical mechanics is the special case when the loss l p theta hat p t is the relative entropy between the true distribution p theta and the estimated distribution hat p t here the cumulative bayes risk from time 1 to n is the mutual information between the random parameter theta and the observations y 1 dots y n new bounds on this mutual information are given in terms of the laplace transform of the hellinger distance between pairs of distributions indexed by parameters in theta from these bounds on the cumulative minimax risk are given in terms of the metric entropy of theta with respect to the hellinger distance the assumptions required for these bounds are very general and do not depend on the choice of the dominating measure they apply to both finite and infinite dimensional theta they apply in some cases where y is infinite dimensional in some cases where y is not compact in some cases where the distributions are not smooth and in some parametric cases where asymptotic normality of the posterior distribution fails

minimax lower bound and optimal estimation of convex functions in the sup norm estimation of convex functions finds broad applications in science and engineering however the convex shape constraint complicates the asymptotic performance analysis of such estimators this technical note is devoted to the minimax optimal estimation of univariate convex functions in a given holder class particularly a minimax lower bound in the supremum norm or simply sup norm is established by constructing a novel family of piecewise quadratic convex functions in the holder class this result along with a recent result on the minimax upper bound gives rise to the optimal rate of convergence for the minimax sup norm risk of convex functions with the holder order between one and two the present technical note provides the first rigorous justification of the optimal minimax risk for convex estimation on the entire interval of interest in the sup norm

convergence of algorithms for reconstructing convex bodies and directional measures we investigate algorithms for reconstructing a convex body k in r n from noisy measurements of its support function or its brightness function in k directions u 1 u k the key idea of these algorithms is to construct a convex polytope p k whose support function or brightness function best approximates the given measurements in the directions u 1 u k in the least squares sense the measurement errors are assumed to be stochastically independent and gaussian it is shown that this procedure is strongly consistent meaning that almost surely p k tends to k in the hausdorff metric as k here some mild assumptions on the sequence u i of directions are needed using results from the theory of empirical processes estimates of rates of convergence are derived which are first obtained in the l 2 metric and then transferred to the hausdorff metric along the way a new estimate is obtained for the metric entropy of the class of origin symmetric zonoids contained in the unit ball similar results are obtained for the convergence of an algorithm that reconstructs an approximating measure to the directional measure of a stationary fiber process from noisy measurements of its rose of intersections in k directions u 1 u k here the dudley and prohorov metrics are used the methods are linked to those employed for the support and brightness function algorithms via the fact that the rose of intersections is the support function of a projection body

global rates of convergence of the mles of log concave and s concave densities we establish global rates of convergence for the maximum likelihood estimators mles of log concave and s concave densities on the main finding is that the rate of convergence of the mle in the hellinger metric is no worse than n 2 5 when 1 s where s 0 corresponds to the log concave case we also show that the mle does not exist for the classes of s concave densities with s 1

an exact line search method for solving generalized continuous time algebraic riccati equations we present a newton like method for solving algebraic riccati equations that uses an exact line search to improve the sometimes erratic convergence behavior of newton s method it avoids the problem of a disastrously large first step and accelerates convergence when newton steps are too small or too long the additional work to perform the line search is small relative to the work needed to calculate the newton step

covering numbers for convex functions in this paper we study the covering numbers of the space of convex and uniformly bounded functions in multidimension we find optimal upper and lower bounds for the epsilon covering number of cal c a b d b in the l p metric 1 leq p in terms of the relevant constants where d geq 1 a b 0 and cal c a b d b denotes the set of all convex functions on a b d that are uniformly bounded by b we summarize previously known results on covering numbers for convex functions and also provide alternate proofs of some known results our results have direct implications in the study of rates of convergence of empirical minimization procedures as well as optimal convergence rates in the numerous convexity constrained function estimation problems

information theory and superefficiency the asymptotic risk of efficient estimators with kullback leibler loss in smoothly parametrized statistical models is k 2n where k is the parameter dimension and n is the sample size under fairly general conditions we given a simple information theoretic proof that the set of parameter values where any arbitrary estimator is superefficient is negligible the proof is based on a result of rissanen that codes have asymptotic redundancy not smaller than k 2 log n except in a set of measure 0

comparisons and bounds for expected lifetimes of reliability systems sharp bounds on expectations of lifetimes of coherent and mixed systems composed of elements with independent and either identically or non identically distributed lifetimes are expressed in terms of expected lifetimes of components similar evaluations are concluded for the respective mean residual lifetimes in the iid case improved inequalities dependent on a concentration parameter connected to the gini dispersion index are obtained the results can be used to compare systems with component lifetimes ordered in the convex ordering in the inid case some refined bounds are derived in terms of the expected lifetimes of series systems of smaller sizes and the expected lifetime of single unit for the equivalent systems with iid components the latter can be further simplified in the case of weak schur concavity and schur convexity of the system generalized domination polynomial