asymptotically efficient adaptive allocation schemes for controlled markov chains finite parameter space the authors consider a controlled markov chain whose transition probabilities and initial distribution are parametrized by an unknown parameter theta belonging to some known parameter space theta there is a one step reward associated with each pair of control and the following state of the process the objective is to maximize the expected value of the sum of one step rewards over an infinite horizon the loss associated with a control scheme at a parameter value is the function of time giving the difference between the maximum reward that could have been achieved if the parameter were known and the reward achieved by the scheme since it is impossible to minimize the loss uniformly for all parameter values the authors define uniformly good adaptive control schemes and restrict attention to these schemes they develop a lower bound on the loss associated with any uniformly good control scheme they construct an adaptive control scheme whose loss equals the lower bound for every parameter value and is therefore asymptotically efficient

hellinger deviance tests efficiency breakdown points and examples abstract hellinger distance analogs of likelihood ratio tests are proposed for parametric inference the proposed tests are based on minimized hellinger distances between nonparametric density estimates and densities corresponding to null and unconstrained parametric models if the parametric model is correct the hellinger deviance test is asymptotically equivalent to the likelihood ratio test a simulation study examines the relative performance of these tests in finite samples breakdown properties of the hellinger deviance tests and likelihood ratio tests are compared the hellinger tests are generally high breakdown point tests whereas in many instances the likelihood ratio tests have breakdown points of 0 two numerical examples illustrate the implementation and performance of the proposed tests likelihood based methods are widely used in applications they provide a routine method for generating efficient estimates and inference statements in many instances however these methods are known to

rescaling entropy and divergence rates based on rescaling by some suitable sequence instead of the number of time units the usual notion of divergence rate is here extended to define and determine meaningful generalized divergence rates rescaling entropy rates appears as a special case suitable rescaling is naturally induced by the asymptotic behavior of the marginal divergences closed form formulas are obtained as soon as the marginal divergences behave like powers of some analytical functions a wide class of countable markov chains is proved to satisfy this property most divergence and entropy functionals defined in the literature are concerned e g the classical shannon kullback leibler renyi and tsallis for illustration purposes ferreri or basu harris hjort jones among others are also considered

breakdown points of affine equivariant estimators of multivariate location and covariance matrices finite sample replacement breakdown points are derived for different types of estimators of multivariate location and covariance matrices the role of various equivariance properties is illustrated the breakdown point is related to a measure of performance based on large deviations probabilities finally we show that one step reweighting preserves the breakdown point

a variable neighborhood search based algorithm for finite horizon markov decision processes abstract this paper considers the application of a variable neighborhood search vns algorithm for finite horizon h stages markov decision processes mdps for the purpose of alleviating the curse of dimensionality phenomenon in searching for the global optimum the main idea behind the vnsmdp algorithm is that based on the result of the stage just considered the search for the optimal solution action of state x in stage t is conducted systematically in variable neighborhood sets of the current action thus the vnsmdp algorithm is capable of searching for the optimum within some subsets of the action space rather than over the whole action set analysis on complexity and convergence attributes of the vnsmdp algorithm are conducted in the paper it is shown by theoretical and computational analysis that the vnsmdp algorithm succeeds in searching for the global optimum in an efficient way

breakdown robustness of tests abstract power and level breakdown functions of a test statistic are introduced to obtain a unified analysis of local stability as typified by the influence function and global stability as illustrated by the breakdown point the power breakdown function gives the amount of contamination of each alternative distribution that can carry the test statistic to a null value the level breakdown function gives the amount of contamination of a null distribution that can carry the test statistic to each value in the alternative space in many cases the breakdown function bears an inverse relationship to the maximum bias as a function of the contamination as the maximum bias is not invariant to one to one transformations however equivalent test statistics can have different scales for bias a distinct advantage of the inverse scale is the invariance of the breakdown functions to such transformations the resulting stability curves depend on the parameters in a way analogous to the power curve the slopes o

least squares quantization in pcm it has long been realized that in pulse code modulation pcm with a given ensemble of signals to handle the quantum values should be spaced more closely in the voltage regions where the signal amplitude is more likely to fall it has been shown by panter and dite that in the limit as the number of quanta becomes infinite the asymptotic fractional density of quanta per unit voltage should vary as the one third power of the probability density per unit voltage of signal amplitudes in this paper the corresponding result for any finite number of quanta is derived that is necessary conditions are found that the quanta and associated quantization intervals of an optimum finite quantization scheme must satisfy the optimization criterion used is that the average quantization noise power be a minimum it is shown that the result obtained here goes over into the panter and dite result as the number of quanta become large the optimum quautization schemes for 2 b quanta b 1 2 cdots 7 are given numerically for gaussian and for laplacian distribution of signal amplitudes

general entropy criteria for inverse problems with applications to data compression pattern classification and cluster analysis minimum distance approaches are considered for the reconstruction of a real function from finitely many linear functional values an optimal class of distances satisfying an orthogonality condition analogous to that enjoyed by linear projections in hilbert space is derived these optimal distances are related to measures of distances between probability distributions recently introduced by c r rao and t k nayak 1985 and possess the geometric properties of cross entropy useful in speech and image compression pattern classification and cluster analysis several examples from spectrum estimation and image processing are discussed

on divergences and informations in statistics and information theory the paper deals with the f divergences of csiszar generalizing the discrimination information of kullback the total variation distance the hellinger divergence and the pearson divergence all basic properties of f divergences including relations to the decision errors are proved in a new manner replacing the classical jensen inequality by a new generalized taylor expansion of convex functions some new properties are proved too e g relations to the statistical sufficiency and deficiency the generalized taylor expansion also shows very easily that all f divergences are average statistical informations differences between prior and posterior bayes errors mutually differing only in the weights imposed on various prior distributions the statistical information introduced by de groot and the classical information of shannon are shown to be extremal cases corresponding to alpha 0 and alpha 1 in the class of the so called arimoto alpha informations introduced in this paper for 0 alpha 1 by means of the arimoto alpha entropies some new examples of f divergences are introduced as well namely the shannon divergences and the arimoto alpha divergences leading for alphauarr1 to the shannon divergences square roots of all these divergences are shown to be metrics satisfying the triangle inequality the last section introduces statistical tests and estimators based on the minimal f divergence with the empirical distribution achieved in the families of hypothetic distributions for the kullback divergence this leads to the classical likelihood ratio test and estimator

recent developments in nonparametric density estimation advances in computation and the fast and cheap computational facilities now available to statisticians have had a significant impact upon statistical research and especially the development of nonparametric data analysis procedures in particular theoretical and applied research on nonparametric density estimation has had a noticeable influence on related topics such as nonparametric regression nonparametric discrimination and nonparametric pattern recognition this article reviews recent developments in nonparametric density estimation and includes topics that have been omitted from review articles and books on the subject the early density estimation methods such as the histogram kernel estimators and orthogonal series estimators are still very popular and recent research on them is described different types of restricted maximum likelihood density estimators including order restricted estimators maximum penalized likelihood estimators and sieve estimators are discussed where restrictions are imposed upon the class of densities or on the form of the likelihood function nonparametric density estimators that are data adaptive and lead to locally smoothed estimators are also discussed these include variable partition histograms estimators based on statistically equivalent blocks nearest neighbor estimators variable kernel estimators and adaptive kernel estimators for the multivariate case extensions of methods of univariate density estimation are usually straightforward but can be computationally expensive a method of multivariate density estimation that did not spring from a univariate generalization is described namely projection pursuit density estimation in which both dimensionality reduction and density estimation can be pursued at the same time finally some areas of related research are mentioned such as nonparametric estimation of functionals of a density robust parametric estimation semiparametric models and density estimation for censored and incomplete data directional and spherical data and density estimation for dependent sequences of observations

structures and numerical ranges of power partial isometries abstract we derive a matrix model under unitary similarity of an n by n matrix a such that a a 2 a k k 1 are all partial isometries which generalizes the known fact that if a is a partial isometry then it is unitarily similar to a matrix of the form 0 b 0 c with b b c c i using this model we show that if a has ascent k and a a 2 a k 1 are partial isometries then the numerical range w a of a is a circular disc centered at the origin if and only if a is unitarily similar to a direct sum of jordan blocks whose largest size is k as an application this yields that for any s n matrix a w a resp w a a is a circular disc centered at the origin if and only if a is unitarily similar to the jordan block j n finally examples are given to show that for a general matrix a the conditions that w a and w a a are circular discs at 0 are independent of each other

asymptotically efficient adaptive choice of control laws incontrolled markov chains we consider a controlled markov chain on a general state space whose transition probabilities are parameterized by an unknown parameter belonging to a compact metric space there is a one step reward associated with each pair of control and the following state of the process given a finite set of stationary control laws under each of which the markov chain is uniformly recurrent an optimal control law is this set is one that maximizes the long run average reward in ignorance of the parameter value we construct an adaptive control rule which uses the optimal control law s at a relative frequency of 1 o n 1 log n and show that this relative frequency gives an asymptotically optimal balance between the control objective and the amount of information needed to learn about the unknown parameter the basic idea underlying this construction is to introduce suitable uncertainty adjustments via sequential testing theory into the certainty equivalence rule thus resolving the apparent dilemma between control and information

pathologies of some minimum distance estimators on etudie des estimateurs de distance minimale dans le modele n 0 1 on montre que la consistance automatique n est pas necessaire lorsque le modele est seulement approximativement vrai

the automatic robustness of minimum distance functionals on montre que les estimateurs de distance minimale forment une classe d estimateurs automatiquement robustes dans le meme sens que l estimateur du maximum de vraisemblance

on the optimality of conditional expectation as a bregman predictor we consider the problem of predicting a random variable x from observations denoted by a random variable z it is well known that the conditional expectation e x z is the optimal l sup 2 predictor also known as the least mean square error predictor of x among all borel measurable functions of z in this orrespondence we provide necessary and sufficient conditions for the general loss functions under which the conditional expectation is the unique optimal predictor we show that e x z is the optimal predictor for all bregman loss functions blfs of which the l sup 2 loss function is a special case moreover under mild conditions we show that the blfs are exhaustive i e if for every random variable x the infimum of e f x y over all constants y is attained by the expectation e x then f is a blf

minimum hellinger distance estimation for finite mixtures of poisson regression models and its applications il a ete montre que l estimation au minimum de la distance de hellinger mhde reduit des points irreguliers par effet de lissage avec une efficacite du premier ordre pour un modele correctement specifie on propose une approche d estimation adaptee aux melanges finis de modeles de regression de poisson basee sur la mhde les simulations de monte carlo montrent a l evidence que la mhde est une alternative viable a l estimation au maximum de vraisemblance lorsque les composants du melange ne sont pas bien separes ou lorsque les parametres du modele sont proches de zero des applications de biometrie illustrent egalement l utilite pratique de la methode mhde

weighted likelihood equations with bootstrap root search abstract we discuss a method of weighting likelihood equations with the aim of obtaining fully efficient and robust estimators we discuss the case of continuous probability models using unimodal weighting functions these weighting functions downweight observations that are inconsistent with the assumed model at the true model therefore the proposed estimating equations behave like the ordinary likelihood equations we investigate the number of solutions of the estimating equations via a bootstrap root search the estimators obtained are consistent and asymptotically normal and have desirable robustness properties an extensive simulation study and real data examples illustrate the operating characteristics of the proposed methodology

stability analysis of multiplicative update algorithms and application to nonnegative matrix factorization multiplicative update algorithms have proved to be a great success in solving optimization problems with nonnegativity constraints such as the famous nonnegative matrix factorization nmf and its many variants however despite several years of research on the topic the understanding of their convergence properties is still to be improved in this paper we show that lyapunov s stability theory provides a very enlightening viewpoint on the problem we prove the exponential or asymptotic stability of the solutions to general optimization problems with nonnegative constraints including the particular case of supervised nmf and finally study the more difficult case of unsupervised nmf the theoretical results presented in this paper are confirmed by numerical simulations involving both supervised and unsupervised nmf and the convergence speed of nmf multiplicative updates is investigated

performance guarantees for empirical markov decision processes with applications to multiperiod inventory models we consider markov decision processes with unknown transition probabilities and unknown single period expected cost functions and we study a method for estimating these quantities from historical or simulated data the method requires knowledge of the system equations that govern state transitions as well as the single period cost functions but not the single period expected cost functions the estimation procedure is based upon taking expectations with respect to the empirical distribution functions of such data once the estimates are in place the method computes a policy by solving the obtained empirical markov decision process as if the estimates were correct for mdps that satisfy some conditions we provide explicit easily computed expressions for the probability that the procedure will produce a policy whose true expected cost is within any specified absolute distance of the actual optimal expected cost of the true markov decision process we also provide expressions for the number of historical or simulated data values that is sufficient for the procedure to produce a policy whose true expected cost is with a prescribed probability within a prescribed absolute distance of the actual optimal expected cost of the true markov decision process we apply our results to multiperiod inventory models in addition we provide a specialized analysis of such inventory models that also yields relative rather than absolute accuracy guarantees we make comparisons with related results that have recently appeared and we provide numerical examples

minimum distance estimators of scale with censored data abstract this article examines how the level of censoring affects the performance of estimators of scale that minimize a weighted cramervon mises distance between the kaplan meier product limit estimator of the survival distribution function and an assumed model the article shows that under certain conditions these estimators are asymptotically normal if the survival function is a member of the assumed scale family weights that minimize the asymptotic variance under these conditions also are found under stronger conditions the minimum distance estimators mde s are asymptotically normal even if the survival function is not a member of the assumed scale family asymptotic coverage probabilities of confidence intervals are found for percentiles constructed from exponential mde s of scale under departures from the exponential distribution and what happens in the limiting cases of no censoring and complete censoring is examined using the asymptotic results and monte carlo experiments several exponenti

minimum hellinger distance estimation for multivariate location and covariance abstract the hellinger distance between a nonparametric density estimator and a model family is minimized to produce estimates of location and covariance in multivariate data with suitable restrictions on the density estimators and the model family these minimum hellinger distance estimators mhde s are shown to be affine invariant consistent and asymptotically normal the robustness of the mhde as measured by the breakdown point compares favorably against the previously studied m estimators monte carlo results suggest that the mhde s are an attractive robust alternative to the usual sample means and covariance matrix

convexity classification and risk bounds many of the classification algorithms developed in the machine learning literature including the support vector machine and boosting can be viewed as minimum contrast methods that minimize a convex surrogate of the 0 1 loss function the convexity makes these algorithms computationally efficient the use of a surrogate however has statistical consequences that must be balanced against the computational virtues of convexity to study these issues we provide a general quantitative relationship between the risk as assessed using the 0 1 loss and the risk as assessed using any nonnegative surrogate loss function we show that this relationship gives nontrivial upper bounds on excess risk under the weakest possible condition on the loss function that it satisfies a pointwise form of fisher consistency for classification the relationship is based on a simple variational transformation of the loss function that is easy to compute in many applications we also present a refined version of this result in the

power partial isometry index and ascent of a finite matrix we give a complete characterization of nonnegative integers j and k and a positive integer n for which there is an n by n matrix with its power partial isometry index equal to j and its ascent equal to k recall that the power partial isometry index p a of a matrix a is the supremum possibly infinity of nonnegative integers j such that i a a2 aj are all partial isometries while the ascent a a of a is the smallest integer k 0 for which ker ak equals ker ak 1 it was known before that for any matrix a either p a min a a n 1 or p a in this paper we prove more precisely that there is an n by n matrix a such that p a j and a a k if and only if one of the following conditions holds a j k n 1 b j k 1 and j k n 1 or c j k 2 and j k n this answers a question we asked in a previous paper

wavelet based texture retrieval using generalized gaussian density and kullback leibler distance we present a statistical view of the texture retrieval problem by combining the two related tasks namely feature extraction fe and similarity measurement sm into a joint modeling and classification scheme we show that using a consistent estimator of texture model parameters for the fe step followed by computing the kullback leibler distance kld between estimated models for the sm step is asymptotically optimal in term of retrieval error probability the statistical scheme leads to a new wavelet based texture retrieval method that is based on the accurate modeling of the marginal distribution of wavelet coefficients using generalized gaussian density ggd and on the existence a closed form for the kld between ggds the proposed method provides greater accuracy and flexibility in capturing texture information while its simplified form has a close resemblance with the existing methods which uses energy distribution in the frequency domain to identify textures experimental results on a database of 640 texture images indicate that the new method significantly improves retrieval rates e g from 65 to 77 compared with traditional approaches while it retains comparable levels of computational complexity

about distances of discrete distributions satisfying the data processing theorem of information theory the distances of discrete probability distributions are considered necessary and sufficient conditions for validity of the data processing theorem of information theory are established these conditions are applied to the burbea rao 1982 divergences and bregman 1967 distances

nonparametric density estimation the l 1 view differentiation of integrals consistency lower bounds for rates of convergence rates of convergence in l1 the automatic kernel estimate l1 and pointwise convergence estimates related to the kernel estimate and the histogram estimate simulation inequalities and random variate generation the transformed kernel estimate applications in discrimination operations on density estimates estimators based on orthogonal series index

iterative regularization and nonlinear inverse scale space applied to wavelet based denoising in this paper we generalize the iterative regularization method and the inverse scale space method recently developed for total variation tv based image restoration to wavelet based image restoration this continues our earlier joint work with others where we applied these techniques to variational based image restoration obtaining significant improvement over the rudin osher fatemi tv based restoration here we apply these techniques to soft shrinkage and obtain the somewhat surprising result that a the iterative procedure applied to soft shrinkage gives firm shrinkage and converges to hard shrinkage and b that these procedures enhance the noise removal capability both theoretically in the sense of generalized bregman distance and for some examples experimentally in terms of the signal to noise ratio leaving less signal in the residual

a computationally compact divergence measure for speech processing the directed divergence which is a measure based on the discrimination information between two signal classes is investigated a simplified expression for computing the directed divergence is derived for comparing two gaussian autoregressive processes such as those found in speech this expression alleviates both the computational cost reduced by two thirds and the numerical problems encountered in computing the directed divergence in addition the simplified expression is compared with the itakura saito distance which asymptotically approaches the directed divergence although the expressions for these two distances closely resemble each other only moderate correlations between the two were found on a set of actual speech data

minimum hellinger distance estimation for the analysis of count data abstract minimum hellinger distance mhd estimation is studied in the context of discrete data the mhd estimator is shown to provide an effective treatment of anomalous data points and its properties are illustrated using short term mutagenicity test data asymptotic normality for a discrete distribution with countable support is derived under a readily verified condition on the model breakdown properties of the mhd estimator and an outlier screen are compared count data occur frequently in statistical applications for instance in chemical mutagenicity studies which comprise an important step in the identification of environmental carcinogens much of the resultant data are counts woodruff mason valencia and zimmering 1984 reported anomalous counts in the sex linked recessive lethal test in drosophila these outliers can have a substantial impact on the experimental conclusions mhd estimation provides a means for reliable inference when modeling count data that are prone to outliers the mh

bregman divergences and surrogates for learning bartlett et al 2006 recently proved that a ground condition for surrogates classification calibration ties up their consistent minimization to that of the classification risk and left as an important problem the algorithmic questions about their minimization in this paper we address this problem for a wide set which lies at the intersection of classification calibrated surrogates and those of murata et al 2004 this set coincides with those satisfying three common assumptions about surrogates equivalent expressions for the members sometimes well known follow for convex and concave surrogates frequently used in the induction of linear separators and decision trees most notably they share remarkable algorithmic features for each of these two types of classifiers we give a minimization algorithm provably converging to the minimum of any such surrogate while seemingly different we show that these algorithms are offshoots of the same ldquomasterrdquo algorithm this provides a new and broad unified account of different popular algorithms including additive regression with the squared loss the logistic loss and the top down induction performed in cart c4 5 moreover we show that the induction enjoys the most popular boosting features regardless of the surrogate experiments are provided on 40 readily available domains

parametric statistical modeling by minimum integrated square error the likelihood function plays a central role in parametric and bayesian estimation as well as in nonparametric function estimation via local polynomial modeling however integrated square error has enjoyed a long tradition as the goodness of fit criterion of choice in nonparametric density estimation in this article i investigate the use of integrated square error or l2 distance as a theoretical and practical estimation tool for a variety of parametric statistical models i show that the asymptotic inefficiency of the parameters estimated by minimizing the integrated square error or l2estimation l2e criterion versus the maximum likelihood estimator is roughly that of the median versus the mean i demonstrate by example the well known result that minimum distance estimators including l2e are inherently robust however l2e does not require specification of any tuning factors found in robust likelihood algorithms l2e is particularly appropriate for analyzing massive datasets in which data cleanin

robust estimation via robust gradient estimation we provide a new computationally efficient class of estimators for risk minimization we show that these estimators are robust for general statistical models under varied robustness settings including in the classical huber e contamination model and in heavy tailed settings our workhorse is a novel robust variant of gradient descent and we provide conditions under which our gradient descent variant provides accurate estimators in a general convex risk minimization problem we provide specific consequences of our theory for linear regression and logistic regression and for canonical parameter estimation in an exponential family these results provide some of the first computationally tractable and provably robust estimators for these canonical statistical models finally we study the empirical performance of our proposed methods on synthetic and real data sets and we find that our methods convincingly outperform a variety of baselines

rerandomizing the median in matched pairs designs on montre que la re randomisation de la mediane a la place de la moyenne transforme un gros probleme de calcul en un probleme convenable pour une calculatrice

semiparametric inference in the proportional odds regression model abstract for fitting the proportional odds regression model with right censored survival times we introduce some weighted empirical odds functions these functions are solutions of some self consistency equations and have a nice martingale representation from these functions several classes of new regression estimators such as the pseudo maximum likelihood estimator martingale residual based estimators and minimum distance estimators are derived these estimators have desirable properties such as easy computation asymptotic normality via a martingale analysis and reliable asymptotic covariance estimation in closed form extensive numerical studies show that the minimum l 2 distance estimators have very good finite sample behaviors compared to existing methods results of some simulation studies and applications to a real dataset are given the weighted odds function based approach also provides inference on the baseline odds function and some measures for lack of fit analysis

robust estimation of mixture complexity in many applications it is important to find the mixture with fewest number of components known as the mixture complexity that provides a satisfactory fit to the data this article focuses on developing an estimator of mixture complexity that is consistent when the form of component densities are unknown but are postulated to be members of some parametric family and is simultaneously robust against model misspecification we treat the estimation of mixture complexity as a model selection problem and construct an estimator of mixture complexity as a byproduct of minimizing a hellinger information criterion this estimator is shown to be consistent for any parametric family of mixtures when the model is correctly specified monte carlo simulations for a wide variety of normal mixtures show that our estimator is very competitive with several others in the literature in correctly identifying the true mixture complexity the basic construction being firmly rooted in the minimum hellinger distance approach

review papers recent developments in nonparametric density estimation abstract advances in computation and the fast and cheap computational facilities now available to statisticians have had a significant impact upon statistical research and especially the development of nonparametric data analysis procedures in particular theoretical and applied research on nonparametric density estimation has had a noticeable influence on related topics such as nonparametric regression nonparametric discrimination and nonparametric pattern recognition this article reviews recent developments in nonparametric density estimation and includes topics that have been omitted from review articles and books on the subject the early density estimation methods such as the histogram kernel estimators and orthogonal series estimators are still very popular and recent research on them is described different types of restricted maximum likelihood density estimators including order restricted estimators maximum penalized likelihood estimators and sieve estimators are discussed where re

higher order infinitesimal robustness using the von mises expansion we study the higher order infinitesimal robustness of a general m functional and characterize its second order properties we show that second order robustness is equivalent to the boundedness of both the estimator s estimating function and its derivative with respect to the parameter it implies at the same time i variance robustness and ii robustness of higher order saddlepoint approximations to the estimator s finite sample density the proposed construction of second order robust m estimators is fairly general and potentially useful in a variety of relevant settings besides the theoretical contributions we discuss the main computational issues and provide an algorithm for the implementation of second order robust m estimators finally we illustrate our theory by monte carlo simulation and in a real data estimation of the maximal losses of nikkei 225 index returns our findings indicate that second order robust estimators can improve on other widely applied robus

robust and efficient estimation by minimising a density power divergence a minimum divergence estimation method is developed for robust parameter estimation the proposed approach uses new density based divergences which unlike existing methods of this type such as minimum hellinger distance estimation avoid the use of nonparametric density estimation and associated complications such as bandwidth selection the proposed class of density power divergences is indexed by a single parameter which controls the trade off between robustness and efficiency the methodology affords a robust extension of maximum likelihood estimation for which 0 choices of near zero afford considerable robustness while retaining efficiency close to that of maximum likelihood

bias robust estimation of scale in this paper we consider the problem of robust estimation of the scale of the location residuals when the underlying distribution of the data belongs to a contamination neighborhood of a parametric location scale family we define the class of m estimates of scale with general location and show that under certain regularity assumptions these scale estimates converge to their asymptotic functionals uniformly with respect to the underlying distribution and with respect to the m estimate defining score function chi we establish expressions for the maximum asymptotic bias of m estimates of scale over the contamination neighborhood as a function of the fraction of contamination using these expressions we construct asymptotically min max bias robust estimates of scale in particular we show that a scaled version of the madm median of absolute residuals about the median is approximately min max bias robust within the class of huber s proposal 2 joint estimates of location and scale we also consider the larger class of m estimates of scale with general location and show that a scaled version of the shorth the shortest half of the data is approximately min max bias robust in this class finally we present the results of a monte carlo study showing that the shorth has attractive finite sample size mean squared error properties for contaminated gaussian data

error bounds for rolling horizon policies in discrete time markov control processes error bounds are presented for rolling horizon rh policies in general stationary and nonstationary borel markov control problems with both discounted and average reward criteria in each of these cases conditions are given under which the reward of the rolling horizon policy converges geometrically to the optimal reward function uniformly in the initial state as the length of the rolling horizon increases a description of the control model and the general assumptions are given the approach is based on extending the results of j m alden and a r l smith 1988 on nonstationary processes with finite state and action spaces however the proofs presented are simpler this is because when stationary models are analyzed first the error bounds follow more or less directly from well known value iteration results the corresponding error bounds for nonstationary models are obtained by reducing these models to stationary ones

of quantiles and expectiles consistent scoring functions choquet representations and forecast rankings summary in the practice of point prediction it is desirable that forecasters receive a directive in the form of a statistical functional for example forecasters might be asked to report the mean or a quantile of their predictive distributions when evaluating and comparing competing forecasts it is then critical that the scoring function used for these purposes be consistent for the functional at hand in the sense that the expected score is minimized when following the directive we show that any scoring function that is consistent for a quantile or an expectile functional can be represented as a mixture of elementary or extremal scoring functions that form a linearly parameterized family scoring functions for the mean value and probability forecasts of binary events constitute important examples the extremal scoring functions admit appealing economic interpretations of quantiles and expectiles in the context of betting and investment problems the choquet type mixture representations give rise to simple checks of whether a forecast dominates another in the sense that it is preferable under any consistent scoring function in empirical settings it suffices to compare the average scores for only a finite number of extremal elements plots of the average scores with respect to the extremal scoring functions which we call murphy diagrams permit detailed comparisons of the relative merits of competing forecasts

on asymptotic properties of information theoretic divergences mutual asymptotic equivalence is established within three classes of information theoretic divergences of discrete probability distributions namely f divergences of csiszar f divergences of bregman and f divergences of burbea rao these equivalences are used to find asymptotic distributions of the corresponding divergence statistics for testing the goodness of fit when the hypothetic distribution is uniform all results are based on standard expansion techniques and on a new relation between the bregman and burbea rao divergences formulated in lemma 2 of the paper

minimum hellinger distance estimation for k component poisson mixture with random effects the k component poisson regression mixture with random effects is an effective model in describing the heterogeneity for clustered count data arising from several latent subpopulations however the residual maximum likelihood estimation reml of regression coefficients and variance component parameters tend to be unstable and may result in misleading inferences in the presence of outliers or extreme contamination in the literature the minimum hellinger distance mhd estimation has been investigated to obtain robust estimation for finite poisson mixtures this article aims to develop a robust mhd estimation approach for k component poisson mixtures with normally distributed random effects by applying the gaussian quadrature technique to approximate the integrals involved in the marginal distribution the marginal probability function of the k component poisson mixture with random effects can be approximated by the summation of a set of finite poisson mixtures simulation study shows that the mhd estimates perform satisfactorily for data without outlying observation s and outperform the reml estimates when data are contaminated application to a data set of recurrent urinary tract infections uti with random institution effects demonstrates the practical use of the robust mhd estimation method

nonnegative matrix factorization with the itakura saito divergence with application to music analysis this letter presents theoretical algorithmic and experimental results about nonnegative matrix factorization nmf with the itakura saito is divergence we describe how is nmf is underlaid by a well defined statistical model of superimposed gaussian components and is equivalent to maximum likelihood estimation of variance parameters this setting can accommodate regularization constraints on the factors through bayesian priors in particular inverse gamma and gamma markov chain priors are considered in this work estimation can be carried out using a space alternating generalized expectation maximization sage algorithm this leads to a novel type of nmf algorithm whose convergence to a stationary point of the is cost function is guaranteed we also discuss the links between the is divergence and other cost functions used in nmf in particular the euclidean distance and the generalized kullback leibler kl divergence as such we describe how is nmf can also be performed using a gradient multiplicative algorithm a standard algorithm structure in nmf whose convergence is observed in practice though not proven finally we report a furnished experimental comparative study of euclidean nmf kl nmf and is nmf algorithms applied to the power spectrogram of a short piano sequence recorded in real conditions with various initializations and model orders then we show how is nmf can successfully be employed for denoising and upmix mono to stereo conversion of an original piece of early jazz music these experiments indicate that is nmf correctly captures the semantics of audio and is better suited to the representation of music signals than nmf with the usual euclidean and kl costs

lower bounds for contamination bias globally minimax versus locally linear estimation we study how robust estimators can be in parametric families obtaining a lower bound on the contamination bias of an estimator that holds for a wide class of parametric families this lower bound includes as a special case the bound used to establish that the median is bias minimax among location equivariant estimators and it is tight or nearly tight in a variety of other settings such as scale estimation discrete exponential families and multiple linear regression the minimum variation distance estimator has contamination bias within a dimension free factor of this bound a second lower bound applies to locally linear estimates and implies that such estimates cannot be bias minimax among all fisher consistent estimates in higher dimensions in linear regression this class of estimates includes the familiar m estimates gm estimates and s estimates in discrete exponential families yet another lower bound implies that the proportion of zeros estimate has minimax bias if the median of the distribution is zero a common situation in some fields this bound also implies that the information standardized sensitivity of every fisher consistent estimate of the poisson mean and of the binomial proportion is unbounded

an adaptive sampling algorithm for solving markov decision processes based on recent results for multiarmed bandit problems we propose an adaptive sampling algorithm that approximates the optimal value of a finite horizon markov decision process mdp with finite state and action spaces the algorithm adaptively chooses which action to sample as the sampling process proceeds and generates an asymptotically unbiased estimator whose bias is bounded by a quantity that converges to zero at rate lnn n wheren is the total number of samples that are used per state sampled in each stage the worst case running time complexity of the algorithm iso a n h independent of the size of the state space where a is the size of the action space andh is the horizon length the algorithm can be used to create an approximate receding horizon control to solve infinite horizon mdps to illustrate the algorithm computational results are reported on simple examples from inventory control

m estimation for discrete data asymptotic distribution theory and implications abstract the asymptotic distribution of an m estimator is studies when the underlying distribution is discrete asymptotic normality is shown to hold quite generally within the assumed parametric family when the specification of the model is inexact however it is demonstrated that an m estimator with a non smooth score function e g a huber estimator has a non normal limiting distribution at certain distributions resulting in unstable inference in the neighborhood of such distributions consequently smooth score functions are proposed for discrete data keywords robust estimation and discrete parametric models author

information geometry of u boost and bregman divergence we aim at an extension of adaboost to u boost in the paradigm to build a stronger classification machine from a set of weak learning machines a geometric understanding of the bregman divergence defined by a generic convex function u leads to the u boost method in the framework of information geometry extended to the space of the finite measures over a label set we propose two versions of u boost learning algorithms by taking account of whether the domain is restricted to the space of probability functions in the sequential step we observe that the two adjacent and the initial classifiers are associated with a right triangle in the scale via the bregman divergence called the pythagorean relation this leads to a mild convergence property of the u boost algorithm as seen in the expectation maximization algorithm statistical discussions for consistency and robustness elucidate the properties of the u boost methods based on a stochastic assumption for training data

rendering parametric procedures more robust by empirically tilting the model summary we suggest methods for tilting a likelihood so as to enhance the robustness of maximum likelihood procedures from the viewpoint of computation tilting amounts to choosing unequal weights for the score function in such a way as to maximise likelihood subject to moving a given distance from equally weighted scores empirical methods based on standard parametric q q plots are used to determine the appropriate amount of tilting distance may be measured in a variety of ways and we devote particular attention to power divergence approaches in this context one of the two kullback leibler distance measures is shown to be advantageous

an asymptotically efficient simulation based algorithm for finite horizon stochastic dynamic programming we present a simulation based algorithm called simulated annealing multiplicative weights samw for solving large finite horizon stochastic dynamic programming problems at each iteration of the algorithm a probability distribution over candidate policies is updated by a simple multiplicative weight rule and with proper annealing of a control parameter the generated sequence of distributions converges to a distribution concentrated only on the best policies the algorithm is asymptotically efficient in the sense that for the goal of estimating the value of an optimal policy a provably convergent finite time upper bound for the sample mean is obtained

min max bias robust regression abstract this paper considers the problem of minimizing the maximum asymptotic bias of regression estimates over epsilon contamination neighborhoods for the joint distribution of the response and carriers two classes of estimates are treated 1 m estimates with bounded function rho applied to the scaled residuals using a very general class of scale estimates and 2 bounded influence function type generalized m estimates estimates in the first class are obtained as the solution of a minimization problem while estimates in the second class are specified by an estimating equation the first class of m estimates is sufficiently general to include both huber proposal 2 simultaneous estimates of regression coefficients and residuals scale and rousseeuw yohai s estimates of regression it is shown than an s estimate based on a jump function type rho solves the min max bias problem for the class of m estimates with very general scale this estimate is obtained by the minimization of the alpha quantile of the squared residuals where alpha epsilon depends on the fraction of contamination epsilon when epsilon approaches limit of 5 alpha epsilon approaches limit of 5 and the min max estimator approaches the least median of squared residuals estimator introduced by rousseeuw for the bounded influence class of gm estimates it is shown the a sign type nonlinearity yields the min max estimate this estimate coincides with the minimum gross error sensitivity gm estimate

robustness infinitesimal neighborhoods and moment restrictions this paper is concerned with robust estimation under moment restrictions a moment restriction model is semiparametric and distribution free therefore it imposes mild assumptions yet it is reasonable to expect that the probability law of observations may have some deviations from the ideal distribution being modeled due to various factors such as measurement errors it is then sensible to seek an estimation procedure that is robust against slight perturbation in the probability measure that generates observations this paper considers local deviations within shrinking topological neighborhoods to develop its large sample theory so that both bias and variance matter asymptotically the main result shows that there exists a computationally convenient estimator that achieves optimal minimax robust properties it is semiparametrically efficient when the model assumption holds and at the same time it enjoys desirable robust properties when it does not

minimum hellinger distance estimation for finite mixture models abstract minimum hellinger distance estimates are considered for finite mixture models when the exact forms of the component densities are unknown in detail but are thought to be close to members of some parametric family minimum hellinger distance estimates are asymptotically efficient if the data come from a member of the parametric family and are robust to certain departures from the parametric family a new algorithm is introduced that is similar to the em algorithm a specialized adaptive density estimate is also introduced standard measures of robustness are discussed some difficulties are noted the robustness and asymptotic efficiency of the estimators are illustrated using simulations

a comparison of related density based minimum divergence estimators this paper compares the minimum divergence estimator of basu et al 1998 to a competing minimum divergence estimator which turns out to be equivalent to a method proposed from a different perspective by windham 1995 both methods can be applied for any parametric model and contain maximum likelihood as a special case efficiencies are compared under model conditions and robustness properties are studied overall the two methods are found to perform quite similarly some relatively small advantages of the former method over the latter are identified

a characterization of all single integral non kernel divergence estimators divergence measures have been used for a long time for different purposes in information theory and statistics in particular density based minimum divergence estimation is a popular tool in the statistical literature given the sampled data and a parametric model we estimate the model parameter by choosing the member of the model family that is closest to the data distribution in terms of the given divergence in the absolutely continuous set up when the distributions from the model family and the unknown data generating distribution are assumed to have densities the application of kernel based non parametric smoothing is sometimes unavoidable to get an estimate of the true data density the use of kernels or other non parametric smoothing techniques makes the estimation process considerably more complex as now one has to impose necessary conditions not just on the model but also on the kernel and its bandwidth in higher dimensions the efficiency of the kernel density estimator kde often becomes too low for the minimum divergence procedure to be practically useful it can therefore lead to a significant advantage to have a divergence which allows minimum divergence estimation bypassing the use of non parametric smoothing for the same reason characterizing the class of such divergences would be a notable achievement in this work we provide a characterization of the class of divergences that bypasses the use of non parametric smoothing in the construction of divergences providing a solution to this very important problem

condition for the numerical range to contain an elliptic disc for an n by n matrix a and an elliptic disc e in the plane we show that the sum of the number of common supporting lines and the number of common intersection points to e and the numerical range w a of a should be at least 2n 1 in order to guarantee that e be contained in w a this generalizes previous results of anderson and thompson as an application our result is used to verify a special case of the poncelet property conjecture 2003 elsevier science inc all rights reserved

approximate stochastic annealing for online control of infinite horizon markov decision processes we present an online simulation based algorithm called approximate stochastic annealing asa for solving infinite horizon finite state action space markov decision processes mdps the algorithm estimates the optimal policy by sampling at each iteration from a probability distribution function over the policy space which is updated iteratively based on the q function estimates obtained via a recursion of q learning type by exploiting a novel connection of asa to the stochastic approximation method we show that the sequence of distribution functions generated by the algorithm converges to a degenerated distribution that concentrates only on the optimal policy numerical examples are also provided to illustrate the algorithm

a unified continuous optimization framework for center based clustering methods center based partitioning clustering algorithms rely on minimizing an appropriately formulated objective function and different formulations suggest different possible algorithms in this paper we start with the standard nonconvex and nonsmooth formulation of the partitioning clustering problem we demonstrate that within this elementary formulation convex analysis tools and optimization theory provide a unifying language and framework to design analyze and extend hard and soft center based clustering algorithms through a generic algorithm which retains the computational simplicity of the popular k means scheme we show that several well known and more recent center based clustering algorithms which have been derived either heuristically or and have emerged from intuitive analogies in physics statistical techniques and information theoretic perspectives can be recovered as special cases of the proposed analysis and we streamline their relationships

qualitative robustness of tests abstract a test is qualitatively robust by definition if its sequence of n 1 log transformed p values n being a measure of the sample size is continuous as a point function of the observations and weakly equicontinuous as a function of discrete probability measures this definition is applitable both to unconditional and to conditional tests under weak regularity conditions an unconditional test is qualitatively robust if and only if its test statistic is continuous a counterexample shows that conditional tests do not share this property the sample mean student s t and ybar xbar permutation tests are not qualitatively robust the sign wilcoxon huber censored likelihood and normal scores tests are qualitatively robust

on bregman distances and divergences of probability measures this paper introduces scaled bregman distances of probability distributions which admit nonuniform contributions of observed events they are introduced in a general form covering not only the distances of discrete and continuous stochastic observations but also the distances of random processes and signals it is shown that the scaled bregman distances extend not only the classical ones studied in the previous literature but also the information divergence and the related wider class of convex divergences of probability measures an information processing theorem is established too but only in the sense of invariance w r t statistically sufficient transformations and not in the sense of universal monotonicity pathological situations where coding can increase the classical bregman distance are illustrated by a concrete example in addition to the classical areas of application of the bregman distances and convex divergences such as recognition classification learning and evaluation of proximity of various features and signals the paper mentions a new application in 3 d exploratory data analysis explicit expressions for the scaled bregman distances are obtained in general exponential families with concrete applications in the binomial poisson and rayleigh families and in the families of exponential processes such as the poisson and diffusion processes including the classical examples of the wiener process and geometric brownian motion

recursive learning automata approach to markov decision processes in this note we present a sampling algorithm called recursive automata sampling algorithm rasa for control of finite horizon markov decision processes mdps by extending in a recursive manner sastry s learning automata pursuit algorithm designed for solving nonsequential stochastic optimization problems rasa returns an estimate of both the optimal action from a given state and the corresponding optimal value based on the finite time analysis of the pursuit algorithm by rajaraman and sastry we provide an analysis for the finite time behavior of rasa specifically for a given initial state we derive the following probability bounds as a function of the number of samples 1 a lower bound on the probability that rasa will sample the optimal action and 2 an upper bound on the probability that the deviation between the true optimal value and the rasa estimate exceeds a given error

geometrizing rates of convergence ii consider estimating a functional t f of an unknown distribution f in mathbf f from data x 1 cdots x n i i d f let omega varepsilon denote the modulus of continuity of the functional t over mathbf f computed with respect to hellinger distance for well behaved loss functions l t we show that inf t n sup mathbf f e fl t n t f is equivalent to l omega n 1 2 to within constants whenever t is linear and mathbf f is convex the same conclusion holds in three nonlinear cases estimating the rate of decay of a density estimating the mode and robust nonparametric regression we study the difficulty of testing between the composite infinite dimensional hypotheses h 0 t f leq t and h 1 t f geq t delta our results hold in the cases studied because the difficulty of the full infinite dimensional composite testing problem is comparable to the difficulty of the hardest simple two point testing subproblem

a nonparametric asymptotic analysis of inventory planning with censored demand we study stochastic inventory planning with lost sales and instantaneous replenishment where contrary to the classical inventory theory knowledge of the demand distribution is not available furthermore we observe only the sales quantity in each period and lost sales are unobservable that is demand data are censored the manager must make an ordering decision in each period based only on historical sales data excess inventory is either perishable or carried over to the next period in this setting we propose nonparametric adaptive policies that generate ordering decisions over time we show that the t period average expected cost of our policy differs from the benchmark newsvendor cost the minimum expected cost that would have incurred if the manager had known the underlying demand distribution by at most o 1 t0 5

integration of stochastic models by minimizing α divergence when there are a number of stochastic models in the form of probability distributions one needs to integrate them mixtures of distributions are frequently used but exponential mixtures also provide a good means of integration this letter proposes a one parameter family of integration called integration which includes all of these well known integrations these are generalizations of various averages of numbers such as arithmetic geometric and harmonic averages there are psychophysical experiments that suggest that integrations are used in the brain the divergence between two distributions is defined which is a natural generalization of kullback leibler divergence and hellinger distance and it is proved that integration is optimal in the sense of minimizing divergence the theory is applied to generalize the mixture of experts and the product of experts to the mixture of experts the predictive distribution is also stated in the bayesian framework

an rkhs approach to robust l sup 2 estimation and signal detection the application of rkhs reproducing kernel hilbert space theory to the problems of robust signal detection and estimation is investigated it is shown that this approach provides a general and unified framework in which to analyze the problems of l sup 2 estimation matched filtering and quadratic detection in the presence of uncertainties regarding the second order structure of the random processes involved