a unified treatment of integer parameter models abstract it is shown that there is an analog to the score function the derivative of the log likelihood with respect to the parameter here called the difference score which plays an important theoretical role in integer parameter models a unified treatment of integer parameter models can be obtained by recognizing that many commonly used models have a linear difference score lds it is established that many of the exact results available for exponential family models also hold for lds models in particular we find an information lower bound show the uniqueness of the maximum likelihood estimator establish monotone likelihood ratio and establish optimality properties in two sided tests it is also shown that if one has a multiparameter likelihood in which there is a conditional likelihood depending on n alone then these exact results apply there as well it is also established when and why the usual likelihood methodology will apply to a class of nonstandard models in which the natural approxima

non logarithmic jensen shannon divergence abstract the jensen shannon divergence is a symmetrized and smoothed version of the kullback leibler divergence recently it has been widely applied to the analysis and characterization of symbolic sequences in this paper we investigate a generalization of the jensen shannon divergence this generalization is done in the framework of the non extensive tsallis statistics we study its basic properties and we investigate its applicability as a tool for segmentating symbolic sequences

some inequalities for information divergence and related measures of discrimination inequalities which connect information divergence with other measures of discrimination or distance between probability distributions are used in information theory and its applications to mathematical statistics ergodic theory and other scientific fields we suggest new inequalities of this type often based on underlying identities as a consequence we obtain certain improvements of the well known pinsker inequality our study depends on two measures of discrimination called capacitory discrimination and triangular discrimination the discussion contains references to related research and comparison with other measures of discrimination e g ali silvey csiszar 1996 1966 divergences and in particular the hellinger distance

easily computable facets of the knapsack polytope it is known that facets and valid inequalities for the knapsack polytope p can be obtained by lifting a simple inequality derived from each minimal cover we study the computational complexity of such lifting in particular we show that the task of computing a lifted facet can be accomplished in o ns where s n is the cardinality of the minimal cover also for a lifted inequality with integer coefficients we show that the dual tasks of recognizing whether the inequality is valid for p or is a facet of p can be done within the same time bound

a monoparametric family of metrics for statistical mechanics abstract in a recent paper we have studied a generalization of the jensen shannon divergence jsd physica a 329 2003 81 this generalization was made in the context of tsallis statistical mechanics the present work is devoted to an investigation of the metric character of the jsd generalization

quantum covariance quantum fisher information and the uncertainty relations in this paper the relation between quantum covariances and quantum fisher informations is studied this study is applied to generalize a recently proved uncertainty relation based on quantum fisher information the proof given here considerably simplifies the previously proposed proofs and leads to more general inequalities

o n log n procedures for tightening cover inequalities abstract we present two procedures for tightening cover induced inequalities in 0 1 programs by using knapsack constraints plus some other information from the program the tightening is obtained by solving successive knapsack problems with all 0 1 objective function coefficients and using dominance criteria to avoid the explicit solving of some knapsack problems the new constraints are 0 1 equivalent to and lp tighter than the original ones both procedures have o n log n complexity where n is the number of variables with nonzero coefficients in the knapsack constraint however one of them can strongly reduce the computational effort some computational experience is reported

the geometry of mixture likelihoods a general theory in this paper certain fundamental properties of the maximum likelihood estimator of a mixing distribution are shown to be geometric properties of the likelihood set the existence support size likelihood equations and uniqueness of the estimator are revealed to be directly related to the properties of the convex hull of the likelihood set and the support hyperplanes of that hull it is shown using geometric techniques that the estimator exists under quite general conditions with a support size no larger than the number of distinct observations analysis of the convex dual of the likelihood set leads to a dual maximization problem a convergent algorithm is described the defining equations for the estimator are compared with the usual parametric likelihood equations for finite mixtures sufficient conditions for uniqueness are given part ii will deal with a special theory for exponential family mixtures

binomial mixtures geometric estimation of the mixing distribution given a mixture of binomial distributions how do we estimate the unknown mixing distribution we build on earlier work of lindsay and further elucidate the geometry underlying this question exploring the approximating role played by cyclic polytopes convergence of a resulting maximum likelihood fitting algorithm is proved and numerical examples given problems over the lack of identifiability of the mixing distribution in part disappear

universal bounds for the holevo quantity coherent information and the jensen shannon divergence the mutual information between the sender of a classical message encoded in quantum carriers and a receiver is fundamentally limited by the holevo quantity using strong subadditivity of entropy we prove that the holevo quantity is not larger than an exchange entropy this implies an upper bound for coherent information moreover restricting our attention to classical information we bound the transmission distance between probability distributions by their entropic distance which is a concave function of their hellinger distance

a branch and cut algorithm for the resolution of large scale symmetric traveling salesman problems an algorithm is described for solving large scale instances of the symmetric traveling salesman problem stsp to optimality the core of the algorithm is a polyhedral cutting plane procedure that exploits a subset of the system of linear inequalities defining the convex hull of the incidence vectors of the hamiltonian cycles of a complete graph the cuts are generated by several identification procedures that have been described in a companion paper whenever the cutting plane procedure does not terminate with an optimal solution the algorithm uses a tree search strategy that as opposed to branch and bound keeps on producing cuts after branching the algorithm has been implemented in fortran two different linear programming lp packages have been used as the lp solver the implementation of the algorithm and the interface with one of the lp solvers is described in sufficient detail to permit the replication of our experiments computational results are reported with up to 42 stsps with sizes rangin

estimating the number of classes estimating the unknown number of classes in a population has numerous important applications in a poisson mixture model the problem is reduced to estimating the odds that a class is undetected in a sample the discontinuity of the odds prevents the existence of locally unbiased and informative estimators and restricts confidence intervals to be one sided confidence intervals for the number of classes are also necessarily one sided a sequence of lower bounds to the odds is developed and used to define pseudo maximum likelihood estimators for the number of classes

estimating the number of classes via sample coverage abstract assume that a random sample is drawn from a population with unknown number of classes and possibly unequal class probabilities a nonparametric estimation technique is proposed to estimate the number of classes using the idea of sample coverage which is defined as the sum of the cell probabilities of the observed classes since expected sample coverage can be well estimated we were motivated to find its role in the estimation of the number of classes this work generalizes the result of esty to a nonparametric approach and extends darroch and ratcliff to incorporate the heterogeneity of the class probabilities the coefficient of variation of the class sizes is shown to play an important role in the recommended estimation procedures the performance of the proposed estimators is investigated by means of monte carlo simulations

predicting the conditional probability of discovering a new class consider a population comprising disjoint classes an important problem arising from various fields is prediction of the random conditional probability of discovering a new class the asymptotic normality of the discovery probability is established in a poisson model where the number of individuals from each class is a poisson process with a class specific rate a new derivation is presented for the well known good toulmin predictor as a moment based estimator for the asymptotic limit of the discovery probability the good toulmin predictor is also shown to be a nonparametric empirical bayes estimator for the expectation of the discovery probability given the rates of the poisson processes and an approximation to an unbiased estimator only for the identifiable part of the expectation of the discovery probability in a multinomial model the properties of the moment based estimator are investigated so that confidence and prediction intervals can be constructed the good toulmin predictor and the discovery

jensen shannon divergence in conjugate spaces the entropy excess of atomic systems and sets with respect to their constituents abstract the disorder of a composite system or of a set of different systems is always higher than the mere sum of the internal disorder of its constituents one of the most widely used functionals employed for measuring the randomness of a single distribution is the shannon entropy from which the increase of disorder within the composite system with respect to those of the constituents is quantified by means of the jensen shannon divergence jsd in this work two different applications of the jsd in the study of the information content of atomic electron densities are carried out i finding the contribution for a given atom of its composing subshells to the total atomic information ii and similarly for selected sets of atoms such as periods and groups throughout the periodic table as well as isoelectronic series in both cases the analysis is performed in the two conjugate position and momentum spaces and the results are interpreted according to physically relevant quantities such as the ionization potential

estimating the number of species in a stochastic abundance model consider a stochastic abundance model in which the species arrive in the sample according to independent poisson processes where the abundance parameters of the processes follow a gamma distribution we propose a new estimator of the number of species for this model the estimator takes the form of the number of duplicated species i e species represented by two or more individuals divided by an estimated duplication fraction the duplication fraction is estimated from all frequencies including singleton information the new estimator is closely related to the sample coverage estimator presented by chao and lee 1992 journal of the american statistical association 87 210 217 we illustrate the procedure using the malayan butterfly data discussed by fisher corbet and williams 1943 journal of animal ecology 12 42 58 and a 1989 christmas bird count dataset collected in florida u s a simulation studies show that this estimator compares well with maximum likelihood estimators i e empirical bayes estimators from the bayesian viewpoint for which an iterative numerical procedure is needed and may be infeasible

on population size estimators in the poisson mixture model summary estimating population sizes via capture recapture experiments has enormous applications the poisson mixture model can be adopted for those applications with a single list in which individuals appear one or more times we compare several nonparametric estimators including the chao estimator the zelterman estimator two jackknife estimators and the bootstrap estimator the target parameter of the chao estimator is a lower bound of the population size those of the other four estimators are not lower bounds and they may produce lower confidence limits for the population size with poor coverage probabilities a simulation study is reported and two examples are investigated

on the jensen shannon divergence and variational distance we study the distance measures between two probability distributions via two different distance metrics a new metric induced from jensen shannon divergence and the well known l sub 1 metric we show that several important results and constructions in computational complexity under the l sub 1 metric carry over to the new metric such as yao s next bit predictor the existence of extractors the leftover hash lemma and the construction of expander graph based extractor finally we show that the useful parity lemma in studying pseudorandomness does not hold in the new metric

solving 0 1 integer programming problems arising from large scale planning models we present methods that are useful in solving some large scale hierarchical planning models involving 0 1 variables these 0 1 programming problems initially could not be solved with any standard techniques we employed several approaches to take advantage of the hierarchical structure of variables ordered by importance and other structures present in the models critical but not sufficient for success was a strong linear programming formulation we describe methods for strengthening the linear programs as well as other techniques necessary for a commercial branch and bound code to be successful in solving these problems

divergence measures based on the shannon entropy a novel class of information theoretic divergence measures based on the shannon entropy is introduced unlike the well known kullback divergences the new measures do not require the condition of absolute continuity to be satisfied by the probability distributions involved more importantly their close relationship with the variational distance and the probability of misclassification error are established in terms of bounds these bounds are crucial in many applications of divergence measures the measures are also well characterized by the properties of nonnegativity finiteness semiboundedness and boundedness

estimating the number of species a review how many kinds are there suppose that a population is partitioned into c classes in many situations interest focuses not on estimation of the relative sizes of the classes but on estimation of c itself for example biologists and ecologists may be interested in estimating the number of species in a population of plants or animals numismatists may be concemed with estimating the number of dies used to produce an ancient coin issue and linguists may be interested in estimating the size of an author s vocabulary in this article we review the problem of statistical estimation of c many approaches have been proposed some purely data analytic and others based in sampling theory in the latter case numerous variations have been considered the population may be finite or infinite if finite samples may be taken with replacement multinomial sampling or without replacement hypergeometric sampling or by bernoulli sampling if infinite sampling may be multinomial or bernoulli or the sample may be th

on the dietrich escudero approach for solving the 0 1 knapsack problem with a 0 1 objective function abstract in this brief note we demonstrate that the dietrich escudero procedure for solving the 0 1 knapsack problem with all 0 1 objective function coefficients is precisely an application of the dantzig algorithm for solving the 0 1 lp knapsack problem plus an appropriate rounding of the solution

on tightening cover induced inequalities abstract here we describe computationally efficient procedures for tightening cover induced inequalities by using 0 1 knapsack constraints and if available cliques whose variables are included in the cover an interesting application is the case where the cover is implied by the knapsack constraint the tightening is achieved by increasing the coefficients of the cover inequality the new constraint is 0 1 equivalent to and lp tighter than the original one the computational complexity of the procedures is o n log n where n is the number of variables in the cover

spreading lengths of hermite polynomials the renyi shannon and fisher spreading lengths of the classical or hypergeometric orthogonal polynomials which are quantifiers of their distribution all over the orthogonality interval are defined and investigated these information theoretic measures of the associated rakhmanov probability density which are direct measures of the polynomial spreading in the sense of having the same units as the variable share interesting properties invariance under translations and reflections linear scaling and vanishing in the limit that the variable tends towards a given definite value the expressions of the renyi and fisher lengths for the hermite polynomials are computed in terms of the polynomial degree the combinatorial multivariable bell polynomials which are shown to characterize the finite power of an arbitrary polynomial play a relevant role for the computation of these information theoretic lengths indeed these polynomials allow us to design an error free computing approach for the entropic moments weighted l q norms of hermite polynomials and subsequently for the renyi and tsallis entropies as well as for the renyi spreading lengths sharp bounds for the shannon length of these polynomials are also given by means of an information theoretic based optimization procedure moreover the existence of a linear correlation between the shannon length as well as the second order renyi length and the standard deviation is computationally proved finally the application to the most popular quantum mechanical prototype system the harmonic oscillator is discussed and some relevant asymptotical open issues related to the entropic moments mentioned previously are posed

pseudo maximum likelihood estimation theory and applications abstract pseudo maximum likelihood estimation easily extends to k parameter models and is of interest in problems in which the likelihood surface is ill behaved in higher dimensions but well behaved in lower dimensions several signal plus noise or convolution models are examined which exhibit such behavior and satisfy the regularity conditions of the asymptotic theory for specific models a numerical comparison of asymptotic variances suggests that a psuedo maximum likelihood estimate of the signal parameter is uniformly more efficient than estimators that have been advanced by previous authors a number of other potential applications are noted

on the convexity of some divergence measures based on entropy functions three measures of divergence between vectors in a convex set of a n dimensional real vector space are defined in terms of certain types of entropy functions and their convexity property is studied among other results a classification of the entropies of degree alpha is obtained by the convexity of these measures these results have applications in information theory and biological studies

a new metric for probability distributions we introduce a metric for probability distributions which is bounded information theoretically motivated and has a natural bayesian interpretation the square root of the well known spl chi sup 2 distance is an asymptotic approximation to it moreover it is a close relative of the capacitory discrimination and jensen shannon divergence

inference on the number of species through geometric lower bounds estimating the number of species in a population from a sample of individuals is investigated in a nonparametric poisson mixture model a sequence of lower bounds to the odds that a species is unseen in the sample are proposed from a geometric perspective a lower bound and its representing mixing distribution can be computed by linear programming with guaranteed convergence these lower bounds can be estimated by the maximum likelihood method and used to construct lower confidence limits for the number of species by the bootstrap method computing the nonparametric maximum likelihood estimator is discussed simulation is used to assess the performance of estimated lower bounds and compare them with several existing estimators a genomic application is investigated

jensen divergence based on fisher s information the measure of jensen fisher divergence between probability distributions is introduced and its theoretical grounds set up this quantity in contrast to the remaining jensen divergences grasps the fluctuations of the probability distributions because it is controlled by the local fisher information which is a gradient functional of the distribution so it is appropriate and informative when studying the similarity of distributions mainly for those having oscillatory character the new jensen fisher divergence shares with the jensen shannon divergence the following properties non negativity additivity when applied to an arbitrary number of probability densities symmetry under exchange of these densities vanishing under certain conditions and definiteness even when these densities present non common zeros moreover the jensen fisher divergence is shown to be expressed in terms of the relative fisher information as the jensen shannon divergence does in terms of the kullback leibler or relative shannon entropy finally the jensen shannon and jensen fisher divergences are compared for the following three large non trivial and qualitatively different families of probability distributions the sinusoidal generalized gamma like and rakhmanov hermite distributions which are closely related to the quantum mechanical probability densities of numerous physical systems