estimating the species accumulation curve using mixtures as a significant tool in ecological studies the species accumulation curve or the collector s curve is the graph of the expected number of detected species as a function of sampling effort the problem of estimating the species accumulation curve based on an empirical data set arising from quadrat sampling is studied in a nonparametric binomial mixture model it will be shown that estimating the species accumulation curve not only is independent of the unknown number of species but also includes estimating the number of species as a limiting case for the purpose of interpolation moment based estimators associated with asymptotic confidence intervals are developed from several points of view a likelihood based procedure is developed for the purpose of extrapolation associated with bootstrap confidence intervals the proposed methods are illustrated by ecological data sets

momentary information transfer as a coupling measure of time series we propose a method to analyze couplings between two simultaneously measured time series our approach is based on conditional mutual sorting information it is related to other concepts for detecting coupling directions the old idea of marko for directed information and the more recent concept of schreiber s transfer entropy by setting suitable conditions we first of all consider momentary information in both time series this enables the detection not only of coupling directions but also delays sorting information refers to ordinal properties of time series which makes the analysis robust with respect to strictly monotonous distortions and thus very useful in the analysis of proxy data in climatology fortunately ordinal analysis is easy and fast to compute we consider also the problem of reliable estimation from finite time series

estimating diversity via frequency ratios we wish to estimate the total number of classes in a population based on sample counts especially in the presence of high latent diversity drawing on probability theory that characterizes distributions on the integers by ratios of consecutive probabilities we construct a nonlinear regression model for the ratios of consecutive frequency counts this allows us to predict the unobserved count and hence estimate the total diversity we believe that this is the first approach to depart from the classical mixed poisson model in this problem our method is geometrically intuitive and yields good fits to data with reasonable standard errors it is especially well suited to analyzing high diversity datasets derived from next generation sequencing in microbial ecology we demonstrate the method s performance in this context and via simulation and we present a dataset for which our method outperforms all competitors

comparing species assemblages via species accumulation curves comparing species assemblages given incidence based data is of importance in ecological studies often done by a visual inspection of estimated species accumulation curves or by an ad hoc use of 95 pointwise confidence bands of these curves it is shown that comparing species assemblages is a challenging problem a chi 2 test is proposed an adjustment using an eigenvalue decomposition is proposed to overcome computational difficulties the bootstrap method is also suggested to approximate the distribution of the proposed test statistic the eigenvalue adjusted eva chi 2 test and the eva bootstrap test are assessed by a simulation study both the eva chi 2 and the eva bootstrap tests are applied to a study that involves two woody seedling species assemblages

chebyshev polynomials moment matching and optimal estimation of the unseen we consider the problem of estimating the support size of a discrete distribution whose minimum non zero mass is at least 1 k using independent samples we show that the minimax sample complexity to achieve an additive error of k with probability at least 0 5 is within universal constant factors of k log k log 2 1 which improves the state of the art result k 2 log k due to valiant and valiant the optimal procedure is a linear estimator based on the chebyshev polynomial and its approximation theoretic properties we also study the closely related species problem where the goal is to estimate the number of distinct colors in an urn containing k balls from repeated draws while achieving an additive error proportional to k still requires k log k samples we show that with k samples one can strictly outperform a general support size estimator using interpolating polynomials

a martingale estimating equation for a capture recapture experiment in discrete time we use martingale theory and a method of moments technique to derive a class of estimators for the size of a closed population in a capture recapture experiment in discrete time where the capture probabilities on the successive capture occasions are allowed to be different the same capture probability is assumed to apply to all animals in the population on the same occasion explicit expressions are given for these estimators and their associated standard errors which involve only simple computation an optimal estimator for the population size is found asymptotic results are readily obtained by an application of a martingale central limit theorem two examples are given to compare the results

parameter orthogonality for a family of discrete distributions abstract the standard contagious distributions see douglas 1980 have been used in such varied fields as biology and automobile insurance often to model various physical phenomena as well as provide a good fit to count data when other models are inadequate unfortunately the parameterizations often used when working with these distributions normally lead to extremely high correlations of the maximum likelihood estimators mle s this tends to lead to mathematical complexities and causes difficulty or even errors in their interpretation furthermore numerical difficulties may arise when using numerical procedures to locate the estimates some of these difficulties were discussed by douglas 1980 pp 171 204 205 who suggested that a reparameterization to reduce or even eliminate such correlation is desirable if the mle s are asymptotically uncorrelated the parameterization is orthogonal philpot 1964 derived an orthogonal parameterization for the neyman type a distribution stein zucchini an

estimating entropy rates with bayesian confidence intervals the entropy rate quantifies the amount of uncertainty or disorder produced by any dynamical system in a spiking neuron this uncertainty translates into the amount of information potentially encoded and thus the subject of intense theoretical and experimental investigation estimating this quantity in observed experimental data is difficult and requires a judicious selection of probabilistic models balancing between two opposing biases we use a model weighting principle originally developed for lossless data compression following the minimum description length principle this weighting yields a direct estimator of the entropy rate which compared to existing methods exhibits significantly less bias and converges faster in simulation with monte carlo techinques we estimate a bayesian confidence interval for the entropy rate in related work we apply these ideas to estimate the information rates between sensory stimuli and neural responses in experimental data shlens kennel abarbanel chichilnisky in preparation

parameter estimation for the sichel distribution and its multivariate extension abstract the sichel distribution is a three parameter compound poisson distribution it is a versatile model for highly skewed frequency distributions of observed counts and has proved useful in fields as diverse as mining engineering linguistics ecology industrial psychology and market research we propose a reparameterization of the sichel distribution and give an algorithm which can be implemented on a typical desktop microcomputer for computing the maximum likelihood estimates of the new parameters the reparameterization has a number of advantages over the old in the important two parameter special case of the sichel distribution known as the inverse gaussian poisson the new parameters are the population mean and a shape parameter and their maximum likelihood estimators are asymptotically uncorrelated the reparameterization also lends itself to the convenient multivariate extension presented here this distribution is well suited for modeling correlated count data whose marginal distribution

the equality of kolmogorov sinai entropy and metric permutation entropy generalized abstract in a 2005 paper the author and collaborators proposed an approach to permutation entropy based on symbolic dynamics this approach allowed us to prove the equality of metric permutation entropy to the conventional metric entropy for symbolic dynamics and as a consequence also for n dimensional interval maps under the assumption of ergodicity in this paper we generalize our approach and extend that equality both to general i e not necessarily ergodic symbolic dynamics and to just measurable maps on not necessarily ordered finite measure spaces arguably the most general setting possible

a statistical estimate of the structure of multi extremal problems the solution of ak extremal problem is defined as the set of pairs xi i i 1 k where xt isith local minimum and i is the volume of the set of attraction of this minimum a bayesian estimate ofk and 1 k is constructed

bayesian estimation of the number of species summary this paper is concerned with a method of estimating the number of species present in a population by considering the number of species obtained in a sample a bayesian approach is used and results are based on a zero truncated negative binomial prior distribution for the number of species in the population results for the posterior model are compared with the prior information and guidelines are suggested in the choice of values for the parameters the estimation of the number of species in a population or the number of species unrepresented by a sample is an inferential problem that occurs frequently in the biological sciences much previous work has stemmed from a parametric model due to fisher corbet and williams 1943 where fisher fitted a theoretical distribution to williams data on macrolepidoptera also good 1953 and later good and toulmin 1956 derived a model based on a set of statistical hypotheses concerning the population frequencies of the individual species however the estimation of the total number of species in the population was not the prime concern of any of the work described above when such estimation has been the main aim it has usually been assumed that the species have equal relative abundances as in for example the maximum likelihood approach of driml and ullrich 1967 clearly this assumption will not be appropriate in many practical situations more recently nonequiprobable models have been derived by efron and thisted 1976 and hill 1979 efron and thisted obtained a method of estimating the number of unseen species by an euler transformation and a linear programming approach they took the number of unseen species to be the number of words shakespeare knew but did not use in print hill provided a model that enables an estimate of the number of species in the population to be made by imposing a zero truncated negative binomial prior distribution for the number of species however hill s final analysis is limited to a uniform prior distribution for the number of species it is stated that this gives a sufficiently close approximation in the present paper it is proposed both to derive and to analyse a model based on the zero truncated negative binomial prior distribution for the number of species a similar generalization of hill s model for zipf s law was given by chen 1980 the form of joint prior density for the unknown species relative abundances given by equation 2 1 is derived under the assumptions used by kempton and wedderburn 1978 they explained that when the frequency distribution of the species abundances can be approximated by a continuous distribution then in practice the gamma distribution provides a reasonable fit hence the abundances of species in the population constitute independent observations from a gamma distribution the relative abundances are then beta variates and their joint density function is as shown in 2 1

estimating the population size for capture recapture data with unequal catchability a point estimator and its associated confidence interval for the size of a closed population are proposed under models that incorporate heterogeneity of capture probability real data sets provided in edwards and eberhardt 1967 journal of wildlife management 31 87 96 and carothers 1973 journal of animal ecology 42 125 146 are used to illustrate this method and to compare it with other estimates the performance of the proposed procedure is also investigated by means of monte carlo experiments the method is especially useful when most of the captured individuals are caught once or twice in the sample for which case the jackknife estimator usually does not work well numerical results also show that the proposed confidence interval performs satisfactorily in maintaining the nominal levels

symbolic diffusion entropy rate of chaotic time series as a surrogate measure for the largest lyapunov exponent existing methods for estimating the largest lyapunov exponent from a time series rely on the rate of separation of initially nearby trajectories reconstructed from the time series in phase space according to ueda chaotic dynamical behavior is viewed as a manifestation of random transitions between unstable periodic orbits in a chaotic attractor which are triggered by perturbations due to experimental observation or the roundoff error characteristic of the computing machine and consequently consists of a sequence of piecewise deterministic processes instead of an entirely deterministic process chaotic trajectories might have no physical reality here we propose a mathematical method for estimating a surrogate measure for the largest lyapunov exponent on the basis of the random diffusion of the symbols generated from a time series in a chaotic attractor without resorting to initially nearby trajectories we apply the proposed method to numerical time series generated by chaotic flow models and verify its validity

distributions on partitions on obtient une famille de distributions a deux parametres sur des partitions par melange d une variable multinomiale avec une distribution de dirichlet symetrique

estimation of sums of random variables examples and information bounds this paper concerns the estimation of sums of functions of observable and unobservable variables lower bounds for the asymptotic variance and a convolution theorem are derived in general finite and infinite dimensional models an explicit relationship is established between efficient influence functions for the estimation of sums of variables and the estimation of their means certain plug in estimators are proved to be asymptotically efficient in finite dimensional models while u v estimators of robbins are proved to be efficient in infinite dimensional mixture models examples include certain species network and data confidentiality problems

a sequential formulation of a logic based on fuzzy modalities abstract a nakamura proposed a logic based on fuzzy modalities the main purpose of this paper is to present a gentzen type sequential formulation of this logic

permutation complexity via duality between values and orderings abstract we study the permutation complexity of finite state stationary stochastic processes based on a duality between values and orderings between values first we establish a duality between the set of all words of a fixed length and the set of all permutations of the same length second on this basis we give an elementary alternative proof of the equality between the permutation entropy rate and the entropy rate for a finite state stationary stochastic processes first proved in j m amigo m b kennel l kocarev the permutation entropy rate equals the metric entropy rate for ergodic information sources and ergodic dynamical systems physica d 210 2005 77 95 third we show that further information on the relationship between the structure of values and the structure of orderings for finite state stationary stochastic processes beyond the entropy rate can be obtained from the established duality in particular we prove that the permutation excess entropy is equal to the excess entropy which is a measure of global correlation present in a stationary stochastic process for finite state stationary ergodic markov processes

what is the real size of a sampled network the case of the internet internet mapping projects generally consist in sampling the network from a limited set of sources by usingtraceroute probes this methodology akin to the merging of spanning trees from the different sources to a set of destinations leads necessarily to a partial incomplete map of the internet accordingly determination of internet topology characteristics from such sampled maps is in part a problem of statistical inference our contribution begins with the observation that the inference of many of the most basic topological quantities including network size and degree characteristics fromtraceroute measurements is in fact a version of the so called species problem in statistics this obse rvation has important implications since species problems are often quite challenging we focus here on the most fundamental example of atraceroute internet species the number of nodes in a network specifica lly

a poisson model for the coverage problem with a genomic application summary suppose a population has infinitely many individuals and is partitioned into unknown n disjoint classes the sample coverage of a random sample from the population is the total proportion of the classes observed in the sample this paper uses a nonparametric poisson mixture model to give new understanding and results for inference on the sample coverage the poisson mixture model provides a simplified framework for inferring any general abundance f coverage the sum of the proportions of those classes that contribute exactly k individuals in the sample for some k in with being a set of nonnegative integers a new moment based derivation of the well known turing estimators is presented as an application a gene categorisation problem in genomic research is addressed since turing s approach is a moment based method maximum likelihood estimation and minimum distance estimation are indicated as alternatives for the coverage problem finally it will be shown that any turing estimator is asymptotically fully efficient

statistically relaxing to generating partitions for observed time series data we introduce a relaxation algorithm to estimate approximations to generating partitions for observed dynamical time series generating partitions preserve dynamical information of a deterministic map in the symbolic representation our method optimizes an essential property of a generating partition avoiding topological degeneracies we construct an energylike functional and use a nonequilibrium stochastic minimization algorithm to search through configuration space for the best assignment of symbols to observed data as each observed point may be assigned a symbol the partitions are not constrained to an arbitrary parametrization we further show how to select particular generating partition solutions which also code low order unstable periodic orbits in a given way hence being able to enumerate through a number of potential generating partition solutions

rediscovery of good turing estimators via bayesian nonparametrics the problem of estimating discovery probabilities originated in the context of statistical ecology and in recent years it has become popular due to its frequent appearance in challenging applications arising in genetics bioinformatics linguistics designs of experiments machine learning etc a full range of statistical approaches parametric and nonparametric as well as frequentist and bayesian has been proposed for estimating discovery probabilities in this article we investigate the relationships between the celebrated good turing approach which is a frequentist nonparametric approach developed in the 1940s and a bayesian nonparametric approach recently introduced in the literature specifically under the assumption of a two parameter poisson dirichlet prior we show that bayesian nonparametric estimators of discovery probabilities are asymptotically equivalent for a large sample size to suitably smoothed good turing estimators as a by product of this result we introduce and investigate a methodology for deriving exact and asymptotic credible intervals to be associated with the bayesian nonparametric estimators of discovery probabilities the proposed methodology is illustrated through a comprehensive simulation study and the analysis of expressed sequence tags data generated by sequencing a benchmark complementary dna library

on the erdos renyi theorem for random fields and sequences and its relationships with the theory of runs and spacings we prove in this paper a law of erdos renyi type for arrays of independent and identically distributed random variables the relationships of our theorem with similar results obtained in the theory of runs and spacings are investigated applications include the evaluation of the rate of convergence of erdos renyi maxima in limiting cases and a generalization of a theorem of erdos and revesz on runs

small sample estimation of species richness applied to forest communities summary many well known methods are available for estimating the number of species in a forest community however most existing methods result in considerable negative bias in applications where field surveys typically represent only a small fraction of sampled communities this article develops a new method based on sampling with replacement to estimate species richness via the generalized jackknife procedure the proposed estimator yields small bias and reasonably accurate interval estimation even with small samples the performance of the proposed estimator is compared with several typical estimators via simulation study using two complete census datasets from panama and malaysia

nonparametric estimation of the probability of discovering a new species abstract a random sample is taken from a population consisting of an unknown number of distinct species a quantity of interest is the probability of discovering a new species when an additional draw from the population is made an estimator of this quantity was introduced by starr 1979 we prove a conjecture of starr s that the estimator is uniformly minimum variance unbiased and give various asymptotic properties of the estimator a nonparametric maximum likelihood estimator that has similar asymptotic properties is introduced a monte carlo study that suggests guidelines for choosing an estimator under various circumstances is given to amplify suppose that if we take a sample of size 1 from a population then the probability of drawing a representative of the ith species is p 1 if n draws are made with replacement the unconditional probability that species i will be observed for the first time on the n 1st draw is given simply as the term p i q n i q i 1 p i from a geometric distributio

an estimator of species overlap using a modified beta binomial model a new species overlap measure is defined as the probability that given a randomly selected species is present in at least one of the two communities it is present in both communities this measure can be thought of as a community analog of the jaccard index for species overlap between two samples standard maximum likelihood methods are used to estimate the parameters of a modified beta binomial distribution this leads directly to estimates of the community overlap measure in an example from decapod communities on coral reefs it is shown that this new estimate of community species overlap is 50 larger than the commonly used jaccard measure of species overlap

nonparametric lower bounds for species richness and shared species richness under sampling without replacement summary a number of species richness estimators have been developed under the model that individuals or sampling units are sampled with replacement however if sampling is done without replacement so that no sampled unit can be repeatedly observed then the traditional estimators for sampling with replacement tend to overestimate richness for relatively high sampling fractions ratio of sample size to the total number of sampling units and do not converge to the true species richness when the sampling fraction approaches one based on abundance data or replicated incidence data we propose a nonparametric lower bound for species richness in a single community and also a lower bound for the number of species shared by multiple communities our proposed lower bounds are derived under very general sampling models they are universally valid for all types of species abundance distributions and species detection probabilities for abundance data individuals detectabilities are allowed to be heterogeneous among species for replicated incidence data the selected sampling units e g quadrats need not be fully censused and species can be spatially aggregated all bounds converge correctly to the true parameters when the sampling fraction approaches one real data sets are used for illustration we also test the proposed bounds by using subsamples generated from large real surveys or censuses and their performance is compared with that of some previous estimators

estimating the number of classes estimating the unknown number of classes in a population has numerous important applications in a poisson mixture model the problem is reduced to estimating the odds that a class is undetected in a sample the discontinuity of the odds prevents the existence of locally unbiased and informative estimators and restricts confidence intervals to be one sided confidence intervals for the number of classes are also necessarily one sided a sequence of lower bounds to the odds is developed and used to define pseudo maximum likelihood estimators for the number of classes

did shakespeare write a newly discovered poem summary the consistency of the word usage in a previously unknown nine stanza poem attributed to shakespeare with that of the shakespearean canon is examined using a nonparametric empirical bayes model we consider also poems by jonson marlowe and donne as well as four poems definitely attributed to shakespeare on balance the poem is found to fit previous shakespearean usage reasonably well

mixture models for estimating the size of a closed population when capture rates vary among individuals nous developpons une parametrisation du melange de lois beta et binomiale qui fournit un cadre inferentiel approprie pour estimer l effectif d une population fermee lorsque les probabilites de capture ou de detection varient entre individus nous ajustons trois classes de modeles de melange beta binomiale logistique normale et modele a classes latentes d une part a des donnees de recapture de lievres d amerique pour en estimer l abondance et d autre part a des comptages d especes d oiseaux pour estimer la richesse specifique dans les deux jeux de donnees les taux de detection varient davantage entre individus lievres ou especes qu entre occasions ou lieux de capture les estimateurs de l effectif de la population et de la richesse specifique sont sensibles aux hypotheses faites sur la distribution sous jacente des probabilites individuelles de detection nous montrons par simulations que les outils habituels d evaluation de la qualite du modele comme la deviance ne sont pas fiables pour choisir entre des modeles de melanges celui qui fournit une inference correcte de l effectif de la population lorsqu elles existent des connaissances a priori sur les causes d heterogeneite des taux de detection doivent etre utilisees pour aider a choisir quelle classe de modeles de melanges utiliser pour l inference

an extended gentzen type formulation of a many valued modal propositinal logic based on zadeh s similarity relation the main purpose of this paper is to present an extended gentzen type formulation of a many valued modal propositional logic based on zadeh s similarity relation

nonexistence of informative unbiased estimators in singular problems in many nonparametric problems such as density estimation nonparametric regression and so on all the existing informative estimators are biased asymptotic or finite sample there has long been a suspicion that either informative unbiased estimators do not exist for such problems or they must be quite complicated in this paper we clarify the nonexistence of informative unbiased estimators in all singular problems both for fixed sample size and asymptotically this includes most problems with optimal rate of convergence slower than n1 2 we also discuss situations in regular problems where such nonexistences can occur

predicting the conditional probability of discovering a new class consider a population comprising disjoint classes an important problem arising from various fields is prediction of the random conditional probability of discovering a new class the asymptotic normality of the discovery probability is established in a poisson model where the number of individuals from each class is a poisson process with a class specific rate a new derivation is presented for the well known good toulmin predictor as a moment based estimator for the asymptotic limit of the discovery probability the good toulmin predictor is also shown to be a nonparametric empirical bayes estimator for the expectation of the discovery probability given the rates of the poisson processes and an approximation to an unbiased estimator only for the identifiable part of the expectation of the discovery probability in a multinomial model the properties of the moment based estimator are investigated so that confidence and prediction intervals can be constructed the good toulmin predictor and the discovery

estimating the number of classes via sample coverage abstract assume that a random sample is drawn from a population with unknown number of classes and possibly unequal class probabilities a nonparametric estimation technique is proposed to estimate the number of classes using the idea of sample coverage which is defined as the sum of the cell probabilities of the observed classes since expected sample coverage can be well estimated we were motivated to find its role in the estimation of the number of classes this work generalizes the result of esty to a nonparametric approach and extends darroch and ratcliff to incorporate the heterogeneity of the class probabilities the coefficient of variation of the class sizes is shown to play an important role in the recommended estimation procedures the performance of the proposed estimators is investigated by means of monte carlo simulations

estimating the prediction function and the number of unseen species in sampling with replacement abstract a sample of n units is taken from a population consisting of an unknown number of species we are interested in estimating the number of species and the prediction function for future sampling the prediction function is defined as the expected number of new species that will be found if an additional sample of size tn is taken for any positive real number t in this paper we point out that an estimator suggested by efron and thisted lacks some essential properties of the true prediction function for example the property of alternating copositivity as a result it cannot be used for large values of t we propose an alternative estimator that possesses the essential properties and is easily obtained we illustrate our estimator with two numerical examples and a simulation study

estimating the entropy rate of spike trains via lempel ziv complexity normalized lempel ziv complexity which measures the generation rate of new patterns along a digital sequence is closely related to such important source properties as entropy and compression ratio but in contrast to these it is a property of individual sequences in this article we propose to exploit this concept to estimate or at least to bound from below the entropy of neural discharges spike trains the main advantages of this method include fast convergence of the estimator as supported by numerical simulation and the fact that there is no need to know the probability law of the process generating the signal furthermore we present numerical and experimental comparisons of the new method against the standard method based on word frequencies providing evidence that this new approach is an alternative entropy estimator for binned spike trains

the permutation entropy rate equals the metric entropy rate for ergodic information sources and ergodic dynamical systems abstract permutation entropy quantifies the diversity of possible orderings of the values a random or deterministic system can take as shannon entropy quantifies the diversity of values we show that the metric and permutation entropy rates measures of new disorder per new observed value are equal for ergodic finite alphabet information sources discrete time stationary stochastic processes with this result we then prove that the same holds for deterministic dynamical systems defined by ergodic maps on n dimensional intervals this result generalizes a previous one for piecewise monotone interval maps on the real line c bandt g keller b pompe entropy of interval maps via permutations nonlinearity 15 2002 1595 1602 at the expense of requiring ergodicity and using a definition of permutation entropy rate differing modestly in the order of two limits the case of non ergodic finite alphabet sources is also studied and an inequality developed finally the equality of permutation and metric entropy rates is extended to ergodic non discrete information sources when entropy is replaced by differential entropy in the usual way

on the relation of ks entropy and permutation entropy abstract since bandt et al have shown that the permutation entropy and the kolmogorov sinai entropy coincide for piecewise monotone interval maps the relationship of both entropies for time discrete dynamical systems is of a certain interest the aim of the present paper is a discussion of this relationship on the basis of an ordinal characterization of the kolmogorov sinai entropy recently given

forbidden ordinal patterns in higher dimensional dynamics abstract forbidden ordinal patterns are ordinal patterns or rank blocks that cannot appear in the orbits generated by a map taking values on a linearly ordered space in which case we say that the map has forbidden patterns once a map has a forbidden pattern of a given length l 0 it has forbidden patterns of any length l l 0 and their number grows superexponentially with l using recent results on topological permutation entropy in this paper we study the existence and some basic properties of forbidden ordinal patterns for self maps on n dimensional intervals our most applicable conclusion is that expansive interval maps with finite topological entropy have necessarily forbidden patterns although we conjecture that this is also the case under more general conditions the theoretical results are nicely illustrated for n 2 both using the naive counting estimator for forbidden patterns and chao s estimator for the number of classes in a population the robustness of forbidden ordinal patterns against observational white noise is also illustrated

a normal limit law for a nonparametric estimator of the coverage of a random sample the coverage of a multinomial random sample is the sum of the probabilities of the observed classes a normal limit law is rigorously proved for good s 1953 coverage estimator the result is valid under very general conditions and all terms except the coverage itself are observable nevertheless the implied confidence intervals are not much wider than those developed under restrictive assumptions such as in the classical occupancy problem the asymptotic variance is somewhat unexpected the proof utilizes a method of holst 1979

bayesian shrinkage estimation of the relative abundance of mrna transcripts using sage sage est une technique permettant de quantifier l expression d un gene dans un tissu biologique cette technique aboutit a l obtention de donnees de comptage modelisables par une loi multinomiale possedant les deux caracteristiques suivantes asymetrie de la distribution des frequences relatives et echantillon de petite taille par rapport a la dimension de la loi multinomiale en raison de ces caracteristiques un echantillon sage peut manquer un grand nombre de types d arnm pourtant presents dans le tissu des lors qu ils ignorent ce probleme des types manquants des estimateurs empiriques de la frequence relative des types d arnm tendent naturellement a surestimer la frequence des types rares trouves au moins une fois lesquels forment la grande majorite des types observes nous avons donc developpe une nouvelle procedure d estimation bayesienne comportant une information a priori tenant compte des caracteristiques precitees cette procedure aboutit a des estimateurs non lineaires a retrecisseur shrinkage plus efficaces que les estimateurs du maximum de vraisemblance la distribution a priori est constituee d un melange de lois de dirichlet les types arnm etant stochastiquement classes en types rares et abondants une distribution multivariee a priori est calculee separement pour chacune des deux classes differentes simulations montrent que sur les scenarios sage envisages cet estimateur a un meilleur lmse somme des ecarts quadratiques moyens des erreurs sur la frequence cumulee sur l ensemble des types que l estimateur du maximum de vraisemblance c est a dire qu il propose pour tous les echantillons simules des profils de frequences relatives plus proches de la realite au sens de la distance euclidienne nous appliquons cette methode a une bibliotheque sage de tissus issus de colons normaux et discutons de ses implications quant a l evaluation de l expression differentielle

estimating the number of species in a stochastic abundance model consider a stochastic abundance model in which the species arrive in the sample according to independent poisson processes where the abundance parameters of the processes follow a gamma distribution we propose a new estimator of the number of species for this model the estimator takes the form of the number of duplicated species i e species represented by two or more individuals divided by an estimated duplication fraction the duplication fraction is estimated from all frequencies including singleton information the new estimator is closely related to the sample coverage estimator presented by chao and lee 1992 journal of the american statistical association 87 210 217 we illustrate the procedure using the malayan butterfly data discussed by fisher corbet and williams 1943 journal of animal ecology 12 42 58 and a 1989 christmas bird count dataset collected in florida u s a simulation studies show that this estimator compares well with maximum likelihood estimators i e empirical bayes estimators from the bayesian viewpoint for which an iterative numerical procedure is needed and may be infeasible

on population size estimators in the poisson mixture model summary estimating population sizes via capture recapture experiments has enormous applications the poisson mixture model can be adopted for those applications with a single list in which individuals appear one or more times we compare several nonparametric estimators including the chao estimator the zelterman estimator two jackknife estimators and the bootstrap estimator the target parameter of the chao estimator is a lower bound of the population size those of the other four estimators are not lower bounds and they may produce lower confidence limits for the population size with poor coverage probabilities a simulation study is reported and two examples are investigated

context tree modeling of observed symbolic dynamics modern techniques invented for data compression provide efficient automated algorithms for the modeling of the observed symbolic dynamics we demonstrate the relationship between coding and modeling motivating the well known minimum description length mdl principle and give concrete demonstrations of the context tree weighting and context tree maximizing algorithms the predictive modeling technique obviates many of the technical difficulties traditionally associated with the correct mdl analyses these symbolic models representing the symbol generating process as a finite state automaton with probabilistic emission probabilities provide excellent and reliable entropy estimations the resimulations of estimated tree models satisfying the mdl model selection criterion are faithful to the original in a number of measures the modeling suggests that the automated context tree model construction could replace fixed order word lengths in many traditional forms of empirical symbolic analysis of the data we provide an explicit pseudocode for implementation of the context tree weighting and maximizing algorithms as well as for the conversion to an equivalent markov chain

resampling inference with complex survey data abstract methods for standard errors and confidence intervals for nonlinear statistics such as ratios regression and correlation coefficients have been extensively studied for stratified multistage designs in which the clusters are sampled with replacement in particular the important special case of two sampled clusters per stratum these methods include the customary linearization or taylor method and resampling methods based on the jackknife and balanced repeated replication brr unlike the jackknife or the brr the linearization method is applicable to general sampling designs but it involves a separate variance formula for each nonlinear statistic thereby requiring additional programming efforts both the jackknife and the brr use a single variance formula for all nonlinear statistics but they are more computing intensive the resampling methods developed here retain these features of the jackknife and the brr yet permit extension to more complex designs involving sampling without replace

extending the lincoln petersen estimator for multiple identifications in one source the lincoln petersen estimator is one of the most popular estimators used in capture recapture studies it was developed for a sampling situation in which two sources independently identify members of a target population for each of the two sources it is determined if a unit of the target population is identified or not this leads to a 2 2 table with frequencies f11 f10 f01 f00 indicating the number of units identified by both sources by the first but not the second source by the second but not the first source and not identified by any of the two sources respectively however f00 is unobserved so that the 2 2 table is incomplete and the lincoln petersen estimator provides an estimate for f00 in this paper we consider a generalization of this situation for which one source provides not only a binary identification outcome but also a count outcome of how many times a unit has been identified using a truncated poisson count model truncating multiple identifications larger than two we propose a maximum likelihood estimator of the poisson parameter and ultimately of the population size this estimator shows benefits in comparison with lincoln petersen s in terms of bias and efficiency it is possible to test the homogeneity assumption that is not testable in the lincoln petersen framework the approach is applied to surveillance data on syphilis from izmir turkey

a multinomial bayesian approach to the estimation of population and vocabulary size we approach estimation of the size of a population or a vocabulary through a bayesian analysis of the multinomial distribution we view the sample as being generated from such a distribution with an unknown number of cells and unknown cell probabilities and develop a bayesian procedure to estimate the number of cells and the coverage of the sample the prior distribution of the number of cells is arbitrary given that number the cell probabilities are assumed to follow a symmetric dirichlet prior a two stage approach is developed for use when the flattening constant of the latter prior cannot be specified in advance our procedures are applied to samples of butterflies insect species and alleles to the works of shakespeare and joyce and to eldridge s sample of english words

diversity as a concept and its measurement abstract this paper puts forth the view that diversity is an average property of a community and identifies that property as species rarity an intrinsic diversity ordering of communities is defined and is shown to be equivalent to stochastic ordering also the sensitivity of an index to rare species is developed culminating in a crossing point theorem and a response theory to perturbations diversity decompositions analogous to the analysis of variance are discussed for two way classifications and mixtures the paper concludes with a brief survey of genetic diversity linguistic diversity industrial concentration and income inequality

a measure of disclosure risk for microdata protection against disclosure is important for statistical agencies releasing microdata files from sample surveys simple measures of disclosure risk can provide useful evidence to support decisions about release we propose a new measure of disclosure risk the probability that a unique match between a microdata record and a population unit is correct we argue that this measure has at least two advantages first we suggest that it may be a more realistic measure of risk than two measures that are currently used with census data second we show that consistent inference in a specified sense may be made about this measure from sample data without strong modelling assumptions this is a surprising finding in its contrast with the properties of the two similar established measures as a result this measure has potentially useful applications to sample surveys in addition to obtaining a simple consistent predictor of the measure we propose a simple variance estimator and show that it is consistent we also consider the extension of inference to allow for certain complex sampling schemes we present a numerical study based on 1991 census data for about 450 000 enumerated individuals in one area of great britain we show that the theoretical results on the properties of the point predictor of the measure of risk and its variance estimator hold to a good approximation for these data

nonparametric entropy estimation for stationary processes and random fields with applications to english text we discuss a family of estimators for the entropy rate of a stationary ergodic process and prove their pointwise and mean consistency under a doeblin type mixing condition the estimators are cesaro averages of longest match lengths and their consistency follows from a generalized ergodic theorem due to maker 1940 we provide examples of their performance on english text and we generalize our results to countable alphabet processes and to random fields

topological permutation entropy permutation entropy quantifies the diversity of possible ordering of the successively observed values a random or deterministic system can take just as shannon entropy quantifies the diversity of the values themselves when the observable or state variable has a natural order relation making permutation entropy possible to compute then the asymptotic rate of growth in permutation entropy with word length forms an alternative means of describing the intrinsic entropy rate of a source herein extending a previous result on metric entropy rate we show that the topological permutation entropy rate for expansive maps equals the conventional topological entropy rate familiar from symbolic dynamics this result is not limited to one dimensional maps

general large deviations of longest gaps in homogeneous poisson processes abstract let n t t 0 be a homogeneous poisson process with unit intensity the longest gap l t of this process is the maximal time subinterval between epochs of arrival times up to time t and it finds applications in the theory of reliability it has been known since the 1980s that the longest gap obeys the laws of large numbers l t ln t 1 as t almost surely however large deviation probabilities of the longest gap have not been well developed yet and in this paper we derive such large deviations with a very general speed the main tool is to establish a global estimation for the distribution function of the longest gap which is of independent interest

on the analysis of accumulation curves summary identifying and counting the total number of biological species observed when plotted against a measure of the effort used to record them gives rise to a species accumulation curve we investigate estimation of the total number of species and other relevant properties of accumulation by elaborating on the multinomial model nakamura and peraza 1998 journal of agricultural biological and environmental statistics 3 17 36 that includes specification of a beta density for recording probabilities we consider a unified description including complete and incomplete aggregated curves a more general scheme for recording and a bayesian framework to allow for inclusion of the biologist s knowledge with regard to typical recording probabilities and the total number of species the beta distribution is used as a prior but recording probabilities are not restrained to be beta distributed the methods yield either closed analytical expressions for inference or relatively simple numerical procedures predictive distributions of future recordings that would eventually lead to an optimal decision theoretical rule for stopping collection effort may be easily calculated a case study regarding species of bats is considered including some guidelines for elicitation of an informative prior

estimating the unseen improved estimators for entropy and other properties we show that a class of statistical properties of distributions which includes such practically relevant properties as entropy the number of distinct elements and distance metrics between pairs of distributions can be estimated given a sublinear sized sample specifically given a sample consisting of independent draws from any distribution over at most k distinct elements these properties can be estimated accurately using a sample of size o k log k for these estimation tasks this performance is optimal to constant factors complementing these theoretical results we also demonstrate that our estimators perform exceptionally well in practice for a variety of estimation tasks on a variety of natural distributions for a wide range of parameters the key step in our approach is to first use the sample to characterize the unseen portion of the distribution effectively reconstructing this portion of the distribution as accurately as if one had a logarithmic factor larger sample this goes beyond such tools as the good turing frequency estimation scheme which estimates the total probability mass of the unobserved portion of the distribution we seek to estimate the shape of the unobserved portion of the distribution this work can be seen as introducing a robust general and theoretically principled framework that for many practical applications essentially amplifies the sample size by a logarithmic factor we expect that it may be fruitfully used as a component within larger machine learning and statistical analysis systems

on the number of observed classes from a multinomial distribution calcul du nombre de classes observees afin de determiner le nombre de loci sur les chromosomes de drosophila

bayesian nonparametric estimation of the probability of discovering new species summary we consider the problem of evaluating the probability of discovering a certain number of new species in a new sample of population units conditional on the number of species recorded in a basic sample we use a bayesian nonparametric approach the different species proportions are assumed to be random and the observations from the population exchangeable we provide a bayesian estimator under quadratic loss for the probability of discovering new species which can be compared with well known frequentist estimators the results we obtain are illustrated through a numerical example and an application to a genomic dataset concerning the discovery of new genes by sequencing additional single read sequences of cdna fragments

balanced repeated replications based on mixed orthogonal arrays summary for stratified samples with arbitrary numbers of primary sampling units per stratum one can divide the units in the stratum into two groups and then apply the balanced half samples method to the two groups thus formed the efficiency loss of the grouped balanced half samples variance estimator can be substantial use of a mixed orthogonal array whose numbers of symbols correspond to the numbers of units per stratum gives a set of balanced subsamples unlike that of gupta nigam 1987 the proposed method of analysis gives a consistent variance estimator and bias reduction for nonlinear statistics to further reduce the number of replicates use of nearly orthogonal arrays is considered the balanced half samples method mccarthy 1966 1969 is commonly used for estimating the variance of nonlinear statistics from stratified samples with two primary sampling units per stratum these samples arise naturally if deep stratification is desired as in some large scale surveys the method is extended gurney jewett 1975 to the case of p primary sampling units per stratum for prime p this extension for p 2 is not adopted in common practice since it requires a much larger number of replicates also in many survey designs the numbers of primary sampling units are unequal a common practice is to divide the units in the stratum into two groups and then to apply the balanced half samples method to the two groups thus formed this has gained popularity because the software for obtaining balanced half samples is readily available and the number of replicates required is more economical as shown in 2 the resulting variance estimators can be quite inefficient and their bias based on a conditional analysis can be considerable the conditional analysis also suggests a method of grouping to reduce the bias the degree of grouping depends on the tradeoff between computational economy and statistical efficiency if the efficiency loss from grouping is not justifiable one may draw replicates by using mixed orthogonal arrays with variable numbers of symbols to accommodate the different numbers of units or groups of units in the strata as long as an orthogonal array is used for drawing replicates we call the method balanced repeated replications in 3 we show that the analysis method of gupta nigam 1987 based on mixed orthogonal arrays leads to an inconsistent variance estimator for nonlinear statistics and propose a method that is valid for variance estimation and bias reduction

a logic for fuzzy data analysis abstract the aim of this paper is to present new ideas on how to analyse fuzzy data using logic tools we propose a modal logic based on fuzzy relations special attention is paid to similarity relations on a set of objects we discuss its relationships to rough sets then we give useful properties for transformation of formulas

a bayesian analysis of multiple recapture sampling for a closed population summary this paper considers from a bayesian viewpoint inferences about the size of a closed animal population from data obtained by a multiple recapture sampling scheme the method developed enables prior information about the population size and the catch probabilities to be utilized to produce considerable improvements in certain cases on ordinary maximum likelihood methods several ways of expressing such prior information are explored and a practical example of the uses of these ways is given the main result of the paper is an approximation to the posterior distribution of sample size that exhibits the contributions made by the likelihood and the prior ideas the multiple recapture sampling scheme involves taking samples from a population of animals at each stage counting the number of marked animals in the sample marking the previously unmarked animals and returning the sample to the population the literature is reviewed by cormack 1968 and the papers most relevant to our problem are those of chapman 1952 and darroch 1958 in these papers maximum likelihood estimates for the population parameters and variances for these estimates are given our approach differs from previous ones by introducing prior information about the population size and about the propensities of the animals to be captured called catch probabilities by some previous authors the incorporation of these propensities is a feature not shared by many previous approaches darroch 1958 found their introduction into his model does not change the maximum likelihood estimates of the population size our approach will show that strong prior knowledge about the catch probabilities can greatly affect inference about the population size and it is in this respect that the greatest difference from previous approaches lies although it is recognized that models of an open population incorporating death and immigration such as those of jolly 1965 and seber 1965 are more practically realistic we feel that our method which deals only with a closed population is worth investigating as a step towards providing a bayesian treatment of the open population problem

composition law of cardinal ordering permutations abstract in this paper the theorems that determine composition laws both cardinal ordering permutations and their inverses are proven so the relative position of points in an h s periodic orbit is completely known as well as which order those points are visited no matter how the h s periodic orbit emerges be it through a period doubling cascade s 2 n of as the h periodic orbit or a primary window like saddle node bifurcation cascade with h 2 n or a secondary window the birth of an s periodic window inside the h periodic one certainly period doubling cascade orbits are particular cases with h 2 and s 2 n both composition laws are also shown in algorithmic way for easy use

equivalence of truncated count mixture distributions and mixtures of truncated count distributions summary this article is about modeling count data with zero truncation a parametric count density family is considered the truncated mixture of densities from this family is different from the mixture of truncated densities from the same family whereas the former model is more natural to formulate and to interpret the latter model is theoretically easier to treat it is shown that for any mixing distribution leading to a truncated mixture a usually different mixing distribution can be found so that the associated mixture of truncated densities equals the truncated mixture and vice versa this implies that the likelihood surfaces for both situations agree and in this sense both models are equivalent zero truncated count data models are used frequently in the capture recapture setting to estimate population size and it can be shown that the two horvitz thompson estimators associated with the two models agree in particular it is possible to achieve strong results for mixtures of truncated poisson densities including reliable global construction of the unique npmle nonparametric maximum likelihood estimator of the mixing distribution implying a unique estimator for the population size the benefit of these results lies in the fact that it is valid to work with the mixture of truncated count densities which is less appealing for the practitioner but theoretically easier mixtures of truncated count densities form a convex linear model for which a developed theory exists including global maximum likelihood theory as well as algorithmic approaches once the problem has been solved in this class it might readily be transformed back to the original problem by means of an explicitly given mapping applications of these ideas are given particularly in the case of the truncated poisson family

estimating the number of species a review how many kinds are there suppose that a population is partitioned into c classes in many situations interest focuses not on estimation of the relative sizes of the classes but on estimation of c itself for example biologists and ecologists may be interested in estimating the number of species in a population of plants or animals numismatists may be concemed with estimating the number of dies used to produce an ancient coin issue and linguists may be interested in estimating the size of an author s vocabulary in this article we review the problem of statistical estimation of c many approaches have been proposed some purely data analytic and others based in sampling theory in the latter case numerous variations have been considered the population may be finite or infinite if finite samples may be taken with replacement multinomial sampling or without replacement hypergeometric sampling or by bernoulli sampling if infinite sampling may be multinomial or bernoulli or the sample may be th

the efficiency of good s nonparametric coverage estimator the asymptotic efficiency of good s nonparametric coverage estimator is obtained relative to the best estimator derived under the assumption that all classes are equally likely even when that assumption is true good s estimator is quite efficient with an asymptotic relative efficiency of greater than 85 in all cases and greater than 95 if the expected coverage is less than one half

general type token distribution the problem of estimating the number of unique types or dis tinct species in a group occurs in many fields but is not a straightforward one given an arbitrary probabilistic distribution of entries to be sam pled this study shows in exact and approximated forms the conditional probabilistic distribution of unique entries sampled independently given the number of samples the numerical evaluation validates the proposed ap proximation finally we discuss a theory that unifies the related approaches regarding estimation of the number of classes ams 2000 subject classifications primary 60c05 62p99 secondary 62f10

asymptotic behavior of total times for jobs that must start over if a failure occurs many processes must complete in the presence of failures different systems respond to task failure in different ways the system may resume a failed task from the failure point or a saved checkpoint shortly before the failure point it may give up on the task and select a replacement task from the ready queue or it may restart the task the behavior of systems under the first two scenarios is well documented but the third restart has resisted detailed analysis in this paper we derive tight asymptotic relations between the distribution of task times without failures and the total time when including failures for any failure distribution in particular we show that if the task time distribution has an unbounded support then the total time distribution h is always heavy tailed asymptotic expressions are given for the tail of h in various scenarios the key ingredients of the analysis are the cramer lundberg asymptotics for geometric sums and integral asymptotics which in some cases are obtained

deterministic dynamics of neural activity during absence seizures in rats the study of brain electrical activities in terms of deterministic nonlinear dynamics has recently received much attention forbidden ordinal patterns fop is a recently proposed method to investigate the determinism of a dynamical system through the analysis of intrinsic ordinal properties of a nonstationary time series the advantages of this method in comparison to others include simplicity and low complexity in computation without further model assumptions in this paper the fop of the eeg series of genetic absence epilepsy rats from strasbourg was examined to demonstrate evidence of deterministic dynamics during epileptic states experiments showed that the number of fop of the eeg series grew significantly from an interictal to an ictal state via a preictal state these findings indicated that the deterministic dynamics of neural networks increased significantly in the transition from the interictal to the ictal states and also suggested that the fop measures of the eeg series could be considered as a predictor of absence seizures

inference for sichel s compound poisson distribution abstract in a long tailed compound poisson distribution introduced by sichel the probabilities can only readily be calculated if one of the parameters has half integer values that is m 1 2 m integer the maximum likelihood estimation of this parameter is compared with an approximate method derived from a grid of half integer values it is found that the approximate method which avoids excessive special computation yields inferences indistinguishable from those based on maximum likelihood and on the score statistic for testing values of thus a major numerical impediment to use of sichel s distribution is removed

kolmogorov sinai entropy from the ordinal viewpoint abstract in the case of ergodicity much of the structure of a one dimensional time discrete dynamical system is already determined by its ordinal structure we generally discuss this phenomenon by considering the distribution of ordinal patterns which describe the ups and downs in the orbits of a borel measurable map on a borel subset of the real line on this base we give a natural ordinal description of the kolmogorov sinai entropy of one dimensional dynamical systems and relate the kolmogorov sinai entropy to the permutation entropy recently introduced by bandt and pompe

an inference procedure for a capture and recapture experiment with time dependent capture probabilities summary estimates for the size of a closed population are given for multiple recapture studies in discrete time the estimates are derived by a method of moments for martingales an estimate and associated standard error of the population size are derived for a homogeneous population when the capture probabilities are permitted to be time dependent in the discrete case corresponding results are obtained for the continuous case as well explicit expressions are given for these estimates and their standard errors only simple computation is involved

a generalization of chao s estimator for covariate information this note generalizes chao s estimator of population size for closed capture recapture studies if covariates are available chao s estimator was developed under unobserved heterogeneity in which case it represents a lower bound of the population size if observed heterogeneity is available in form of covariates we show how this information can be used to reduce the bias of chao s estimator the key element in this development is the understanding and placement of chao s estimator in a truncated poisson likelihood it is shown that a truncated poisson likelihood with log link with all counts truncated besides ones and twos is equivalent to a binomial likelihood with logit link this enables the development of a generalized chao estimator as the estimated expected value of the frequency of zero counts under a truncated all counts truncated except ones and twos poisson regression model if the regression model accounts for the heterogeneity entirely the generalized chao estimator is asymptotically unbiased a simulation study illustrates the potential in gain of bias reduction comparisons of the generalized chao estimator with the homogeneous zero truncated poisson regression approach are supplied as well the method is applied to a surveillance study on the completeness of farm submissions in great britain