asymptotic normality of a nonparametric estimator of sample coverage this paper establishes a necessary and sufficient condition for the asymptotic normality of the nonparametric estimator of sample coverage proposed by good biometrica 40 1953 237 264 this new necessary and sufficient condition extends the validity of the asymptotic normality beyond the previously proven cases

a proof of simon s theorem on piecewise testable languages abstract we give a new proof of the theorem of i simon that a language is piecewise testable if and only if it is recognized by a finite f trivial monoid our proof is based on representations by certain types of decreasing mappings

infinitely often autoreducible sets a set a is autoreducible if one can compute for all x the value a x by querying a only at places y neq x furthermore a is infinitely often autoreducible if for infinitely many x the value a x can be computed by querying a only at places y neq x for all other x the computation outputs a special symbol to signal that the reduction is undefined it is shown that for polynomial time turing and truth table autoreducibility there are a b c in the class exp of all exponential time computable sets such that a is not infinitely often turing autoreducible b is turing autoreducible but not infinitely often truth table autoreducible and c is truth table autoreducible with g n 1 queries but not infinitely often turing autoreducible with g n queries here n is the length of the input g is nondecreasing and there exists a polynomial p such that p n bounds both the computation time and the value of g at input of length n furthermore connections between notions of infinitely often autoreducibility and notions of approximability are investigated the hausdorff dimension of the class of sets which are not infinitely often autoreducible is shown to be 1

network tomography estimating source destination traffic intensities from link data abstract the problem of estimating the node to node traffic intensity from repeated measurements of traffic on the links of a network is formulated and discussed under poisson assumptions and two types of traffic routing regimens deterministic a fixed known path between each directed pair of nodes and markovian a random path between each directed pair of nodes determined according to a known markov chain fixed for that pair maximum likelihood estimation and related approximations are discussed and computational difficulties are pointed out a detailed methodology is presented for estimates based on the method of moments the estimates are derived algorithmically taking advantage of the fact that the first and second moment equations give rise to a linear inverse problem with positivity restrictions that can be approached by an em algorithm resulting in a particularly simple solution to a hard problem a small simulation study is carried out

generic density and small span theorem we refine the genericity concept of ambos spies by assigning a real number in 0 1 to every generic set called its generic density we construct sets of generic density any e computable real in 0 1 and show a relationship between generic density and lutz resource bounded dimension we also introduce strong generic density and show that it is related to packing dimension we show that all four notions are different we show that whereas dimension notions depend on the underlying probability measure generic density does not which implies that every dimension result proved by generic density arguments simultaneously holds under any biased coin based probability measure we prove such a result we improve the small span theorem of juedes and lutz to the packing dimension setting for k bounded truth table reductions under any biased coin probability measure

chebyshev polynomials moment matching and optimal estimation of the unseen we consider the problem of estimating the support size of a discrete distribution whose minimum non zero mass is at least 1 k using independent samples we show that the minimax sample complexity to achieve an additive error of k with probability at least 0 5 is within universal constant factors of k log k log 2 1 which improves the state of the art result k 2 log k due to valiant and valiant the optimal procedure is a linear estimator based on the chebyshev polynomial and its approximation theoretic properties we also study the closely related species problem where the goal is to estimate the number of distinct colors in an urn containing k balls from repeated draws while achieving an additive error proportional to k still requires k log k samples we show that with k samples one can strictly outperform a general support size estimator using interpolating polynomials

a martingale estimating equation for a capture recapture experiment in discrete time we use martingale theory and a method of moments technique to derive a class of estimators for the size of a closed population in a capture recapture experiment in discrete time where the capture probabilities on the successive capture occasions are allowed to be different the same capture probability is assumed to apply to all animals in the population on the same occasion explicit expressions are given for these estimators and their associated standard errors which involve only simple computation an optimal estimator for the population size is found asymptotic results are readily obtained by an application of a martingale central limit theorem two examples are given to compare the results

parameter orthogonality for a family of discrete distributions abstract the standard contagious distributions see douglas 1980 have been used in such varied fields as biology and automobile insurance often to model various physical phenomena as well as provide a good fit to count data when other models are inadequate unfortunately the parameterizations often used when working with these distributions normally lead to extremely high correlations of the maximum likelihood estimators mle s this tends to lead to mathematical complexities and causes difficulty or even errors in their interpretation furthermore numerical difficulties may arise when using numerical procedures to locate the estimates some of these difficulties were discussed by douglas 1980 pp 171 204 205 who suggested that a reparameterization to reduce or even eliminate such correlation is desirable if the mle s are asymptotically uncorrelated the parameterization is orthogonal philpot 1964 derived an orthogonal parameterization for the neyman type a distribution stein zucchini an

disclosure control of microdata abstract national statistical offices meet an increasing demand for the dissemination of microdata this trend can be at variance with the care taken not to disclose data about individuals therefore the risk of disclosure of each microdata set to be released should be assessed if this risk is too high measures have to be taken to protect the data set this article describes the disclosure problem and explains why it is a real problem using the concept of uniqueness some theory is developed that can help establish the risk of identification it turns out that useful microdata sets can only be released if some of the disclosure risks are dealt with by legal arrangements rather than by restrictions on the data to be released

effective category and measure in abstract complexity theory strong variants of the operator speed up theorem operator gap theorem and compression theorem are obtained using an effective version of baire category theorem it is also shown that all complexity classes of recursive predicates have effective measure zero in the space of recursive predicates and on the other hand the class of predicates with almost everywhere complexity above an arbitrary recursive threshold has recursive measure one in the class of recursive predicates

parameter estimation for the sichel distribution and its multivariate extension abstract the sichel distribution is a three parameter compound poisson distribution it is a versatile model for highly skewed frequency distributions of observed counts and has proved useful in fields as diverse as mining engineering linguistics ecology industrial psychology and market research we propose a reparameterization of the sichel distribution and give an algorithm which can be implemented on a typical desktop microcomputer for computing the maximum likelihood estimates of the new parameters the reparameterization has a number of advantages over the old in the important two parameter special case of the sichel distribution known as the inverse gaussian poisson the new parameters are the population mean and a shape parameter and their maximum likelihood estimators are asymptotically uncorrelated the reparameterization also lends itself to the convenient multivariate extension presented here this distribution is well suited for modeling correlated count data whose marginal distribution

combinatorial results for semigroups of order preserving partial transformations abstract let po n be the semigroup of all order preserving partial transformations of a finite chain it is shown that po n c n satisfies the recurrence 2n 1 n 1 c n 1 4 3n 2 1 c n 2n 1 n 1 c n 1 with initial conditions c0 1 c1 2 it is also shown that e po n e n satisfies the recurrence en 1 5 en en 1 1 with initial conditions e0 1 e1 2 moreover the cardinalities of the green s relations l r and j have been computed

idempotent connected abundant semigroups a general theory for a class of abundant semigroups is developed for a semigroup s in this class let e be its set of idempotents and the subsemigroup of s generated by e when is regular there is a homomorphism with a number of desirable properties from s onto a full subsemigroup of the hall semigroup t from this fact analogues of results in the regular case are obtained for simple and simple abundant semigroups

sparse sets lowness and highness we develop the notions of generalized lowness for sets in ph the union of the polynomial time hierarchy and of generalized highness for arbitrary sets also we develop the notions of extended lowness and extended highness for arbitrary sets these notions extend the decomposition of np into low sets and high sets developed by schoning 15 and studied by ko and schoning 9 we show that either every sparse set in ph is generalized high or no sparse set in ph is generalized high further either every sparse set is extended high or no sparse set is extended high in both situations the former case corresponds to the polynomial time hierarchy having only finitely many levels while the latter case corresponds to the polynomial time hierarchy extending infinitely many levels

a statistical estimate of the structure of multi extremal problems the solution of ak extremal problem is defined as the set of pairs xi i i 1 k where xt isith local minimum and i is the volume of the set of attraction of this minimum a bayesian estimate ofk and 1 k is constructed

bayesian estimation of the number of species summary this paper is concerned with a method of estimating the number of species present in a population by considering the number of species obtained in a sample a bayesian approach is used and results are based on a zero truncated negative binomial prior distribution for the number of species in the population results for the posterior model are compared with the prior information and guidelines are suggested in the choice of values for the parameters the estimation of the number of species in a population or the number of species unrepresented by a sample is an inferential problem that occurs frequently in the biological sciences much previous work has stemmed from a parametric model due to fisher corbet and williams 1943 where fisher fitted a theoretical distribution to williams data on macrolepidoptera also good 1953 and later good and toulmin 1956 derived a model based on a set of statistical hypotheses concerning the population frequencies of the individual species however the estimation of the total number of species in the population was not the prime concern of any of the work described above when such estimation has been the main aim it has usually been assumed that the species have equal relative abundances as in for example the maximum likelihood approach of driml and ullrich 1967 clearly this assumption will not be appropriate in many practical situations more recently nonequiprobable models have been derived by efron and thisted 1976 and hill 1979 efron and thisted obtained a method of estimating the number of unseen species by an euler transformation and a linear programming approach they took the number of unseen species to be the number of words shakespeare knew but did not use in print hill provided a model that enables an estimate of the number of species in the population to be made by imposing a zero truncated negative binomial prior distribution for the number of species however hill s final analysis is limited to a uniform prior distribution for the number of species it is stated that this gives a sufficiently close approximation in the present paper it is proposed both to derive and to analyse a model based on the zero truncated negative binomial prior distribution for the number of species a similar generalization of hill s model for zipf s law was given by chen 1980 the form of joint prior density for the unknown species relative abundances given by equation 2 1 is derived under the assumptions used by kempton and wedderburn 1978 they explained that when the frequency distribution of the species abundances can be approximated by a continuous distribution then in practice the gamma distribution provides a reasonable fit hence the abundances of species in the population constitute independent observations from a gamma distribution the relative abundances are then beta variates and their joint density function is as shown in 2 1

estimating the population size for capture recapture data with unequal catchability a point estimator and its associated confidence interval for the size of a closed population are proposed under models that incorporate heterogeneity of capture probability real data sets provided in edwards and eberhardt 1967 journal of wildlife management 31 87 96 and carothers 1973 journal of animal ecology 42 125 146 are used to illustrate this method and to compare it with other estimates the performance of the proposed procedure is also investigated by means of monte carlo experiments the method is especially useful when most of the captured individuals are caught once or twice in the sample for which case the jackknife estimator usually does not work well numerical results also show that the proposed confidence interval performs satisfactorily in maintaining the nominal levels

idempotent rank in finite full transformation semigroups the subsemigroup sing n of singular elements of the full transformation semigroup on a finite set is generated by n n l 2 idempotents of defect one in this paper we extend this result to the subsemigroup k n r consisting of all elements of rank r or less we prove that the idempotent rank defined as the cardinality of a minimal generating set of idempotents of k n r is s n r the stirling number of the second kind

resource bounded randomness and computational complexity the following is a survey of resource bounded randomness concepts and their relations to each other further we introduce several new resource bounded randomness concepts corresponding to the classical randomness concepts and show that the notion of polynomial time bounded ko randomness is independent of the notions of polynomial time bounded lutz schnorr and kurtz randomness lutz has conjectured that for a given time or space bound the corresponding resource bounded lutz randomness is a proper renement of resource bounded schnorr randomness this conjecture is answered for the case of polynomial time bound moreover we will show that polynomial time bounded schnorr randomness is a proper renement of polynomial time bounded kurtz randomness in contrast to this result we show that the notions of polynomial time bounded lutz schnorr and kurtz randomness coincide in the case of recursive sets thus it suces to study the notion of resource bounded lutz randomness in the context of complexity theory the stochastic properties of resource bounded random sequences will be discussed in detail schnorr has already shown that the law of large numbers holds for p random sequences we will show that another important law in probability theory the law of the iterated logarithm holds for p random sequences too hence almost all sets in the exponential time complexity class are hard from the viewpoint of statistics c 2000 elsevier science b v all rights reserved

martingale families and dimension in p we introduce a new measure notion on small complexity classes called f measure based on martingale families that gets rid of some drawbacks of previous measure notions it can be used to define dimension because martingale families can make money on all strings and it yields random sequences with an equal frequency of 0 s and 1 s on larger complexity classes e and above f measure is equivalent to lutz resource bounded measure as applications to f measure we answer a question raised in e allender m strauss measure on small complexity classes with application for bpp in proc of the 35th ann ieee symp on found of comp sci 1994 pp 807 818 by improving their result to for almost every language a decidable in subexponential time p a bpp a we show that almost all languages in pspace do not have small non uniform complexity we compare f measure to previous notions and prove that martingale families are strictly stronger than c measure e allender m strauss measure on small complexity classes with application for bpp in proc of the 35th ann ieee symp on found of comp sci 1994 pp 807 818 we also discuss the limitations of martingale families concerning finite unions we observe that all classes closed under polynomial many one reductions have measure zero in exp iff they have measure zero in subexp we use martingale families to introduce a natural generalization of lutz resource bounded dimension j h lutz dimension in complexity classes in proceedings of the 15th annual ieee conference on computational complexity 2000 pp 158 169 on p which meets the intuition behind lutz s notion we show that p dimension lies between finite state dimension and dimension on e we prove an analogue of a theorem of eggleston in p i e the class of languages whose characteristic sequence contains 1 s with frequency a has dimension the shannon entropy of a in p

a combinatorial property of finite full transformation semigroups let e be the set of idempotents in the semigroup singn of singular self maps of n 1 n let singn then e2 if and only if for every x in im the set x 1 either contains x or contains an element of im write rank for im and fix for x n xa x define x x x 2 to be an admissible triple if x im x 3 x 2 let comp the complexity of be the maximum number of disjoint admissible triples then e3 if and only if

distributions on partitions on obtient une famille de distributions a deux parametres sur des partitions par melange d une variable multinomiale avec une distribution de dirichlet symetrique

estimation of sums of random variables examples and information bounds this paper concerns the estimation of sums of functions of observable and unobservable variables lower bounds for the asymptotic variance and a convolution theorem are derived in general finite and infinite dimensional models an explicit relationship is established between efficient influence functions for the estimation of sums of variables and the estimation of their means certain plug in estimators are proved to be asymptotically efficient in finite dimensional models while u v estimators of robbins are proved to be efficient in infinite dimensional mixture models examples include certain species network and data confidentiality problems

what is the real size of a sampled network the case of the internet internet mapping projects generally consist in sampling the network from a limited set of sources by usingtraceroute probes this methodology akin to the merging of spanning trees from the different sources to a set of destinations leads necessarily to a partial incomplete map of the internet accordingly determination of internet topology characteristics from such sampled maps is in part a problem of statistical inference our contribution begins with the observation that the inference of many of the most basic topological quantities including network size and degree characteristics fromtraceroute measurements is in fact a version of the so called species problem in statistics this obse rvation has important implications since species problems are often quite challenging we focus here on the most fundamental example of atraceroute internet species the number of nodes in a network specifica lly

a poisson model for the coverage problem with a genomic application summary suppose a population has infinitely many individuals and is partitioned into unknown n disjoint classes the sample coverage of a random sample from the population is the total proportion of the classes observed in the sample this paper uses a nonparametric poisson mixture model to give new understanding and results for inference on the sample coverage the poisson mixture model provides a simplified framework for inferring any general abundance f coverage the sum of the proportions of those classes that contribute exactly k individuals in the sample for some k in with being a set of nonnegative integers a new moment based derivation of the well known turing estimators is presented as an application a gene categorisation problem in genomic research is addressed since turing s approach is a moment based method maximum likelihood estimation and minimum distance estimation are indicated as alternatives for the coverage problem finally it will be shown that any turing estimator is asymptotically fully efficient

almost every set in exponential time is p bi immune a set a is p bi immune if neither a nor its complement has an infinite subset in p we investigate here the abundance of p bi immune languages in linear exponential time e using resource bounded measure we prove that the class of p bi immune sets has measure 1 in e this implies that almost every language in e is p bi immune a bit further we show that every p random pseudorandom language is e bi immune regarding the existence of p bi immune sets in np we show that if np does not have measure 0 in e then np contains a p bi immune set another consequence is that the class of 1 tt p complete languages for e has measure 0 in e in contrast using the approach of resource bounded category it is shown that in e the class of p bi immune languages lacks the property of baire the baire category analogue of lebesgue measurability

recursive computational depth in the 1980 s bennett introduced computational depth as a formal measure of the amount of computational history that is evident in an object s structure in particular bennett identified the classes of weakly deep and strongly deep sequences and showed that the halting problem is strongly deep juedes lathrop and lutz subsequently extended this result by defining the class of weakly useful sequences and proving that every weakly useful sequence is strongly deep

a note on dimensions of polynomial size circuits in this paper we use resource bounded dimension theory to investigate polynomial size circuits we show that for every i 0 p poly has ith order scaled p3 strong dimension 0 we also show that p polyi o has p3 dimension 1 2 and p3 strong dimension 1 our results improve previous measure results of lutz almost everywhere high nonuniform complexity j comput syst sci 44 2 1992 220 258 and dimension results of hitchcock and vinodchandran dimension entropy rates and compression in proc 19th ieee conf computational complexity 2004 pp 174 183 j comput syst sci to appear additionally we establish a supergale dilation theorem which extends the martingale dilation technique introduced implicitly by ambos spies terwijn and zheng resource bounded randomness and weakly complete problems theoret comput sci 172 1 2 1997 195 207 and made explicit by juedes and lutz weak completeness in e and e2 theoret comput sci 143 1 1995 149 158

estimation and inference of quantile regression for survival data under biased sampling abstractbiased sampling occurs frequently in economics epidemiology and medical studies either by design or due to data collecting mechanism failing to take into account the sampling bias usually leads to incorrect inference we propose a unified estimation procedure and a computationally fast resampling method to make statistical inference for quantile regression with survival data under general biased sampling schemes including but not limited to the length biased sampling the case cohort design and variants thereof we establish the uniform consistency and weak convergence of the proposed estimator as a process of the quantile level we also investigate more efficient estimation using the generalized method of moments and derive the asymptotic normality we further propose a new resampling method for inference which differs from alternative procedures in that it does not require to repeatedly solve estimating equations it is proved that the resampling method consistently estimates the asymptoti

genericity randomness and polynomial time approximations polynomial time safe and unsafe approximations for intractable sets were introduced by meyer and paterson technical report tm 126 laboratory for computer science mit cambridge ma 1979 and yesha siam j comput 12 1983 pp 411 425 respectively the question of which sets have optimal safe and unsafe approximations has been investigated extensively duris and rolim lecture notes in comput sci 841 springer verlag berlin new york 1994 pp 38 51 and ambos spies proc 22nd icalp springer verlag berlin new york 1995 pp 384 392 showed that the existence of optimal polynomial time approximations for the safe and unsafe cases is independent using the law of the iterated logarithm for p random sequences which has been recently proven in proc 11th conf computational complexity ieee computer society press piscataway nj 1996 pp 180 189 we extend this observation by showing that both the class of polynomial time delta levelable sets and the class of sets which have optimal polynomial time unsafe approximations have p measure 0 hence typical sets in e in the sense of p measure do not have optimal polynomial time unsafe approximations we will also establish the relationship between resource bounded genericity concepts and the polynomial time safe and unsafe approximation concepts

nonparametric estimation of the probability of discovering a new species abstract a random sample is taken from a population consisting of an unknown number of distinct species a quantity of interest is the probability of discovering a new species when an additional draw from the population is made an estimator of this quantity was introduced by starr 1979 we prove a conjecture of starr s that the estimator is uniformly minimum variance unbiased and give various asymptotic properties of the estimator a nonparametric maximum likelihood estimator that has similar asymptotic properties is introduced a monte carlo study that suggests guidelines for choosing an estimator under various circumstances is given to amplify suppose that if we take a sample of size 1 from a population then the probability of drawing a representative of the ith species is p 1 if n draws are made with replacement the unconditional probability that species i will be observed for the first time on the n 1st draw is given simply as the term p i q n i q i 1 p i from a geometric distributio

estimating a multiplicative treatment effect under biased allocation suppose that given x is poisson with mean while given y is poisson with mean or according as treatment 1 or treatment 2 is used moreover suppose that treatment 1 is used if x

estimating the number of classes estimating the unknown number of classes in a population has numerous important applications in a poisson mixture model the problem is reduced to estimating the odds that a class is undetected in a sample the discontinuity of the odds prevents the existence of locally unbiased and informative estimators and restricts confidence intervals to be one sided confidence intervals for the number of classes are also necessarily one sided a sequence of lower bounds to the odds is developed and used to define pseudo maximum likelihood estimators for the number of classes

did shakespeare write a newly discovered poem summary the consistency of the word usage in a previously unknown nine stanza poem attributed to shakespeare with that of the shakespearean canon is examined using a nonparametric empirical bayes model we consider also poems by jonson marlowe and donne as well as four poems definitely attributed to shakespeare on balance the poem is found to fit previous shakespearean usage reasonably well

estimating the number of classes via sample coverage abstract assume that a random sample is drawn from a population with unknown number of classes and possibly unequal class probabilities a nonparametric estimation technique is proposed to estimate the number of classes using the idea of sample coverage which is defined as the sum of the cell probabilities of the observed classes since expected sample coverage can be well estimated we were motivated to find its role in the estimation of the number of classes this work generalizes the result of esty to a nonparametric approach and extends darroch and ratcliff to incorporate the heterogeneity of the class probabilities the coefficient of variation of the class sizes is shown to play an important role in the recommended estimation procedures the performance of the proposed estimators is investigated by means of monte carlo simulations

assessing identification risk in survey microdata using log linear models this article considers the assessment of the risk of identification of respondents in survey microdata in the context of applications at the united kingdom uk office for national statistics ons the threat comes from the matching of categorical key variables between microdata records and external data sources and from the use of log linear models to facilitate matching while the potential use of such statistical models is well established in the literature little consideration has been given to model specification or to the sensitivity of risk assessment to this specification in numerical work not reported here we have found that standard techniques for selecting log linear models such as chi squared goodness of fit tests provide little guidance regarding the accuracy of risk estimation for the very sparse tables generated by typical applications at ons for example tables with millions of cells formed by cross classifying six key variables with sample sizes of 10 or 100 000 in this articl

on independent random oracles abstract it is shown that p a p b bppp holds for every algorithmically random oracle a b this results extends the corresponding probability one characterization of ambos spies 1986 and kurtz 1987

nonexistence of informative unbiased estimators in singular problems in many nonparametric problems such as density estimation nonparametric regression and so on all the existing informative estimators are biased asymptotic or finite sample there has long been a suspicion that either informative unbiased estimators do not exist for such problems or they must be quite complicated in this paper we clarify the nonexistence of informative unbiased estimators in all singular problems both for fixed sample size and asymptotically this includes most problems with optimal rate of convergence slower than n1 2 we also discuss situations in regular problems where such nonexistences can occur

predicting the conditional probability of discovering a new class consider a population comprising disjoint classes an important problem arising from various fields is prediction of the random conditional probability of discovering a new class the asymptotic normality of the discovery probability is established in a poisson model where the number of individuals from each class is a poisson process with a class specific rate a new derivation is presented for the well known good toulmin predictor as a moment based estimator for the asymptotic limit of the discovery probability the good toulmin predictor is also shown to be a nonparametric empirical bayes estimator for the expectation of the discovery probability given the rates of the poisson processes and an approximation to an unbiased estimator only for the identifiable part of the expectation of the discovery probability in a multinomial model the properties of the moment based estimator are investigated so that confidence and prediction intervals can be constructed the good toulmin predictor and the discovery

a normal limit law for a nonparametric estimator of the coverage of a random sample the coverage of a multinomial random sample is the sum of the probabilities of the observed classes a normal limit law is rigorously proved for good s 1953 coverage estimator the result is valid under very general conditions and all terms except the coverage itself are observable nevertheless the implied confidence intervals are not much wider than those developed under restrictive assumptions such as in the classical occupancy problem the asymptotic variance is somewhat unexpected the proof utilizes a method of holst 1979

estimating a treatment effect under biased sampling methods are presented for the estimation of a treatment effect based on before and after treatment values where for ethical reasons all and only those patients are treated whose before treatment values exceed a given constant

estimating the number of species in a stochastic abundance model consider a stochastic abundance model in which the species arrive in the sample according to independent poisson processes where the abundance parameters of the processes follow a gamma distribution we propose a new estimator of the number of species for this model the estimator takes the form of the number of duplicated species i e species represented by two or more individuals divided by an estimated duplication fraction the duplication fraction is estimated from all frequencies including singleton information the new estimator is closely related to the sample coverage estimator presented by chao and lee 1992 journal of the american statistical association 87 210 217 we illustrate the procedure using the malayan butterfly data discussed by fisher corbet and williams 1943 journal of animal ecology 12 42 58 and a 1989 christmas bird count dataset collected in florida u s a simulation studies show that this estimator compares well with maximum likelihood estimators i e empirical bayes estimators from the bayesian viewpoint for which an iterative numerical procedure is needed and may be infeasible

baire categories on small complexity classes and meager comeager laws we introduce two resource bounded baire category notions on small complexity classes such as p quasipoly subexp and pspace and on probabilistic classes such as bpp which differ on how the corresponding finite extension strategies are computed we give an alternative characterization of small sets via resource bounded banach mazur games as an application of the first notion we show that for almost every language a i e all except a meager class computable in subexponential time p a bpp a we also show that almost all languages in pspace do not have small nonuniform complexity we then switch to the second baire category notion called locally computable and show that the class sparse is meager in p we show that in contrast to the resource bounded measure case meager comeager laws can be obtained for many standard complexity classes relative to locally computable baire category on bpp and pspace another topic where locally computable baire categories differ from resource bounded measure is regarding weak completeness we show that there is no weak completeness notion in p based on locally computable baire categories i e every p weakly complete set is complete for p we also prove that the class of complete sets for p under turing logspace reductions is meager in p if p is not equal to dspace logn and that the same holds unconditionally for quasipoly finally we observe that locally computable baire categories are incomparable with all existing resource bounded measure notions on small complexity classes which might explain why those two settings seem to differ so fundamentally

resource bounded martingales and computable dowd type generic sets martin lof defined algorithmic randomness the use of martingales in this definition and several variants were explored by schnorr and lutz introduced resource bounded randomness ambos spies et al have studied resource bounded randomness under time bounds and have shown that resource bounded randomness implies resource bounded genericity while the genericity of ambos spies is based on time bound on finite extension strategy the genericity of dowd the main topic of this paper is based on an analogy of the forcing theorem for a positive integer r an oracle d is r generic in the sense of dowd r dowd if the following holds if a certain formula f on an exponential sized portion of d is a tautology then a polynomial sized subfunction of d forces f to be a tautology here r is the number of occurrences of query symbols in f the relationship between resource bounded randomness and dowd type genericity has been not clear so far we show that there exists a primitive recursive function t n with the following property every t n random set is r dowd for each positive integer r a proof is done by means of constructing resource bounded martingales in our former paper we left an open problem whether there exists a primitive recursive set which is r dowd for every positive integer r in our recent work we give an affirmative answer to the problem the main theorem of the present paper gives an alternative proof of the affirmative answer

resampling inference with complex survey data abstract methods for standard errors and confidence intervals for nonlinear statistics such as ratios regression and correlation coefficients have been extensively studied for stratified multistage designs in which the clusters are sampled with replacement in particular the important special case of two sampled clusters per stratum these methods include the customary linearization or taylor method and resampling methods based on the jackknife and balanced repeated replication brr unlike the jackknife or the brr the linearization method is applicable to general sampling designs but it involves a separate variance formula for each nonlinear statistic thereby requiring additional programming efforts both the jackknife and the brr use a single variance formula for all nonlinear statistics but they are more computing intensive the resampling methods developed here retain these features of the jackknife and the brr yet permit extension to more complex designs involving sampling without replace

on the topological size of p m complete degrees abstract all polynomial many one degrees are shown to be of second baire category in the superset topology when witness functions are allowed to run in 2loghn time for any h any improvement of this result for the complete p m degrees of re exp or np implies p np or the nonisomorphism of the np complete sets

diversity as a concept and its measurement abstract this paper puts forth the view that diversity is an average property of a community and identifies that property as species rarity an intrinsic diversity ordering of communities is defined and is shown to be equivalent to stochastic ordering also the sensitivity of an index to rare species is developed culminating in a crossing point theorem and a response theory to perturbations diversity decompositions analogous to the analysis of variance are discussed for two way classifications and mixtures the paper concludes with a brief survey of genetic diversity linguistic diversity industrial concentration and income inequality

a multinomial bayesian approach to the estimation of population and vocabulary size we approach estimation of the size of a population or a vocabulary through a bayesian analysis of the multinomial distribution we view the sample as being generated from such a distribution with an unknown number of cells and unknown cell probabilities and develop a bayesian procedure to estimate the number of cells and the coverage of the sample the prior distribution of the number of cells is arbitrary given that number the cell probabilities are assumed to follow a symmetric dirichlet prior a two stage approach is developed for use when the flattening constant of the latter prior cannot be specified in advance our procedures are applied to samples of butterflies insect species and alleles to the works of shakespeare and joyce and to eldridge s sample of english words

if not empty np p is topologically large abstract in the classical cantor topology or in the superset topology np and consequently classes included in np are meagre however in a natural combination of the two topologies we prove that np p if not empty is a second category class while np complete sets form a first category class these results are extended to different levels in the polynomial hierarchy and to the low and high hierarchies p immune sets in np np simple sets p bi immune sets and np effectively simple sets are all second category if not empty it is shown that if c is any of the above second category classes then for all b np there exists an a c such that a is arbitrarily close to b infinitely often

compressibility and resource bounded measure we give a new definition of resource bounded measure based on compressibility of infinite binary strings we prove that the new definition is equivalent to the one commonly used this new characterization offers us a different way to look at resource bounded measure shedding more light on the meaning of measure zero results and providing one more tool to prove such results the main contribution of the paper is the new definition and the proofs leading to the equivalence result we then show how this new characterization can be used to prove that the class of linear autoreducible sets has p measure 0 we also prove that the class of sets that are truth table reducible to a p selective set has p measure 0 and that the class of sets that turing reduce to a subpolynomial dense set has p measure 0 this strengthens various results

a measure of disclosure risk for microdata protection against disclosure is important for statistical agencies releasing microdata files from sample surveys simple measures of disclosure risk can provide useful evidence to support decisions about release we propose a new measure of disclosure risk the probability that a unique match between a microdata record and a population unit is correct we argue that this measure has at least two advantages first we suggest that it may be a more realistic measure of risk than two measures that are currently used with census data second we show that consistent inference in a specified sense may be made about this measure from sample data without strong modelling assumptions this is a surprising finding in its contrast with the properties of the two similar established measures as a result this measure has potentially useful applications to sample surveys in addition to obtaining a simple consistent predictor of the measure we propose a simple variance estimator and show that it is consistent we also consider the extension of inference to allow for certain complex sampling schemes we present a numerical study based on 1991 census data for about 450 000 enumerated individuals in one area of great britain we show that the theoretical results on the properties of the point predictor of the measure of risk and its variance estimator hold to a good approximation for these data

completeness and weak completeness under polynomial size circuits this paper investigates the distribution and nonuniform complexity of problems that are complete or weakly complete for espace under nonuniform many one reductions that are computed by polynomial size circuits p poly many one reductions every weakly p poly many one complete problem is shown to have a dense exponential nonuniform complexity core an exponential lower bound on the space bounded kolmogorov complexities of weakly p poly turing complete problems is established more importantly the p poly many one complete problems are shown to be unusually simple elements of espace in the sense that they obey nontrivial upper bounds on nonuniform complexity size of nonuniform complexity cores and space bounded kolmogorov complexity that are violated by almost every element of espace more generally a small span theorem for p poly many one reducibility in espace is proven and used to show that every p poly many one degree including the complete degree has measure 0 in espace in contrast almost every element of espace is weakly p many one complete all upper and lower bounds are shown to be tight

frequency of frequencies distributions and size dependent exchangeable random partitions abstractmotivated by the fundamental problem of modeling the frequency of frequencies fof distribution this paper introduces the concept of a cluster structure to define a probability function that governs the joint distribution of a random count and its exchangeable random partitions a cluster structure naturally arising from a completely random measure mixed poisson process allows the probability distribution of the random partitions of a subset of a population to be dependent on the population size a distinct and motivated feature that makes it more flexible than a partition structure this allows it to model an entire fof distribution whose structural properties change as the population size varies a fof vector can be simulated by drawing an infinite number of poisson random variables or by a stick breaking construction with a finite random number of steps a generalized negative binomial process model is proposed to generate a cluster structure where in the prior the number of clusters is f

an upward measure separation theorem abstract it is shown that almost every language in espace is very hard to approximate with circuits it follows that p bpp implies that e is a measure 0 subset of espace

on the analysis of accumulation curves summary identifying and counting the total number of biological species observed when plotted against a measure of the effort used to record them gives rise to a species accumulation curve we investigate estimation of the total number of species and other relevant properties of accumulation by elaborating on the multinomial model nakamura and peraza 1998 journal of agricultural biological and environmental statistics 3 17 36 that includes specification of a beta density for recording probabilities we consider a unified description including complete and incomplete aggregated curves a more general scheme for recording and a bayesian framework to allow for inclusion of the biologist s knowledge with regard to typical recording probabilities and the total number of species the beta distribution is used as a prior but recording probabilities are not restrained to be beta distributed the methods yield either closed analytical expressions for inference or relatively simple numerical procedures predictive distributions of future recordings that would eventually lead to an optimal decision theoretical rule for stopping collection effort may be easily calculated a case study regarding species of bats is considered including some guidelines for elicitation of an informative prior

functions that preserve p randomness we show that polynomial time randomness p randomness is preserved under a variety of familiar operations including addition and multiplication by a nonzero polynomial time computable real number these results follow from a general theorem if email protected r is an open interval f i r is a function and email protected i is p random then f r is p random provided1 f is p computable on the dyadic rational points in i and 2 f varies sufficiently at r i e there exists a real constant c 0 such that either email protected i r f x f r x r c or email protected i r f x f r x r c our theorem implies in particular that any analytic function about a p computable point whose power series has uniformly p computable coefficients preserves p randomness in its open interval of absolute convergence such functions include all the familiar functions from first year calculus

on the number of observed classes from a multinomial distribution calcul du nombre de classes observees afin de determiner le nombre de loci sur les chromosomes de drosophila

balanced repeated replications based on mixed orthogonal arrays summary for stratified samples with arbitrary numbers of primary sampling units per stratum one can divide the units in the stratum into two groups and then apply the balanced half samples method to the two groups thus formed the efficiency loss of the grouped balanced half samples variance estimator can be substantial use of a mixed orthogonal array whose numbers of symbols correspond to the numbers of units per stratum gives a set of balanced subsamples unlike that of gupta nigam 1987 the proposed method of analysis gives a consistent variance estimator and bias reduction for nonlinear statistics to further reduce the number of replicates use of nearly orthogonal arrays is considered the balanced half samples method mccarthy 1966 1969 is commonly used for estimating the variance of nonlinear statistics from stratified samples with two primary sampling units per stratum these samples arise naturally if deep stratification is desired as in some large scale surveys the method is extended gurney jewett 1975 to the case of p primary sampling units per stratum for prime p this extension for p 2 is not adopted in common practice since it requires a much larger number of replicates also in many survey designs the numbers of primary sampling units are unequal a common practice is to divide the units in the stratum into two groups and then to apply the balanced half samples method to the two groups thus formed this has gained popularity because the software for obtaining balanced half samples is readily available and the number of replicates required is more economical as shown in 2 the resulting variance estimators can be quite inefficient and their bias based on a conditional analysis can be considerable the conditional analysis also suggests a method of grouping to reduce the bias the degree of grouping depends on the tradeoff between computational economy and statistical efficiency if the efficiency loss from grouping is not justifiable one may draw replicates by using mixed orthogonal arrays with variable numbers of symbols to accommodate the different numbers of units or groups of units in the strata as long as an orthogonal array is used for drawing replicates we call the method balanced repeated replications in 3 we show that the analysis method of gupta nigam 1987 based on mixed orthogonal arrays leads to an inconsistent variance estimator for nonlinear statistics and propose a method that is valid for variance estimation and bias reduction

measure on p we give a notion of measure at p in a paradigm that differs somewhat from the standard theory our new notion overcomes some limitations of earlier formulations specifically concerning closure of null sets under union first we analyze formally some of the difficulties in defining measure at p we then present the new definitions and determine the basic properties of the notion including the density and immunity characteristics of a random language we argue that these results are parallel to previous measure results at exponential time

a bayesian analysis of multiple recapture sampling for a closed population summary this paper considers from a bayesian viewpoint inferences about the size of a closed animal population from data obtained by a multiple recapture sampling scheme the method developed enables prior information about the population size and the catch probabilities to be utilized to produce considerable improvements in certain cases on ordinary maximum likelihood methods several ways of expressing such prior information are explored and a practical example of the uses of these ways is given the main result of the paper is an approximation to the posterior distribution of sample size that exhibits the contributions made by the likelihood and the prior ideas the multiple recapture sampling scheme involves taking samples from a population of animals at each stage counting the number of marked animals in the sample marking the previously unmarked animals and returning the sample to the population the literature is reviewed by cormack 1968 and the papers most relevant to our problem are those of chapman 1952 and darroch 1958 in these papers maximum likelihood estimates for the population parameters and variances for these estimates are given our approach differs from previous ones by introducing prior information about the population size and about the propensities of the animals to be captured called catch probabilities by some previous authors the incorporation of these propensities is a feature not shared by many previous approaches darroch 1958 found their introduction into his model does not change the maximum likelihood estimates of the population size our approach will show that strong prior knowledge about the catch probabilities can greatly affect inference about the population size and it is in this respect that the greatest difference from previous approaches lies although it is recognized that models of an open population incorporating death and immigration such as those of jolly 1965 and seber 1965 are more practically realistic we feel that our method which deals only with a closed population is worth investigating as a step towards providing a bayesian treatment of the open population problem

category and measure in complexity classes this paper presents resource bounded category and resource bounded measure two new tools for computational complexity theory and some applications of these tools to the structure theory of exponent

estimating the number of species a review how many kinds are there suppose that a population is partitioned into c classes in many situations interest focuses not on estimation of the relative sizes of the classes but on estimation of c itself for example biologists and ecologists may be interested in estimating the number of species in a population of plants or animals numismatists may be concemed with estimating the number of dies used to produce an ancient coin issue and linguists may be interested in estimating the size of an author s vocabulary in this article we review the problem of statistical estimation of c many approaches have been proposed some purely data analytic and others based in sampling theory in the latter case numerous variations have been considered the population may be finite or infinite if finite samples may be taken with replacement multinomial sampling or without replacement hypergeometric sampling or by bernoulli sampling if infinite sampling may be multinomial or bernoulli or the sample may be th

efficiency of the u v method of estimation given a pool of motorists how do we estimate the total intensity of those who had a prespecified number of traffic accidents in the past year we previously have proposed the u v method as a solution to estimation problems of this type in this paper we prove that the u v method provides asymptotically efficient estimators in an important special case

general type token distribution the problem of estimating the number of unique types or dis tinct species in a group occurs in many fields but is not a straightforward one given an arbitrary probabilistic distribution of entries to be sam pled this study shows in exact and approximated forms the conditional probabilistic distribution of unique entries sampled independently given the number of samples the numerical evaluation validates the proposed ap proximation finally we discuss a theory that unifies the related approaches regarding estimation of the number of classes ams 2000 subject classifications primary 60c05 62p99 secondary 62f10

the efficiency of good s nonparametric coverage estimator the asymptotic efficiency of good s nonparametric coverage estimator is obtained relative to the best estimator derived under the assumption that all classes are equally likely even when that assumption is true good s estimator is quite efficient with an asymptotic relative efficiency of greater than 85 in all cases and greater than 95 if the expected coverage is less than one half

inference for sichel s compound poisson distribution abstract in a long tailed compound poisson distribution introduced by sichel the probabilities can only readily be calculated if one of the parameters has half integer values that is m 1 2 m integer the maximum likelihood estimation of this parameter is compared with an approximate method derived from a grid of half integer values it is found that the approximate method which avoids excessive special computation yields inferences indistinguishable from those based on maximum likelihood and on the score statistic for testing values of thus a major numerical impediment to use of sichel s distribution is removed

resource bounded randomness and weakly complete problems we introduce and study resource bounded random sets based on lutz s concept of resource bounded measure 5 6 we concentrate on nc randomness c 2 which corresponds to the polynomial time bounded p measure of lutz and which is adequate for studying the internal and quantative structure of e dtime 2lin first we show that the class of nc random sets has p measure 1 this provides a new simplified approach to p measure 1 results next we compare randomness with genericity in the sense of 1 2 and we show that nc 1 random sets are nc generic whereas the converse fails from the former we conclude thatnc random sets are not p btt complete for e our technical main results describe the distribution of the nc random sets under p m reducibility we show that every nc random set in e has nk random predecessors in e for any k 1 whereas the amount of randomness of the successors is bounded we apply this result to answer a question raised by lutz 8 we show that the class of weakly complete sets has measure 1 in e and that there are weakly complete problems which are not p btt complete for e

an inference procedure for a capture and recapture experiment with time dependent capture probabilities summary estimates for the size of a closed population are given for multiple recapture studies in discrete time the estimates are derived by a method of moments for martingales an estimate and associated standard error of the population size are derived for a homogeneous population when the capture probabilities are permitted to be time dependent in the discrete case corresponding results are obtained for the continuous case as well explicit expressions are given for these estimates and their standard errors only simple computation is involved

a generalization of chao s estimator for covariate information this note generalizes chao s estimator of population size for closed capture recapture studies if covariates are available chao s estimator was developed under unobserved heterogeneity in which case it represents a lower bound of the population size if observed heterogeneity is available in form of covariates we show how this information can be used to reduce the bias of chao s estimator the key element in this development is the understanding and placement of chao s estimator in a truncated poisson likelihood it is shown that a truncated poisson likelihood with log link with all counts truncated besides ones and twos is equivalent to a binomial likelihood with logit link this enables the development of a generalized chao estimator as the estimated expected value of the frequency of zero counts under a truncated all counts truncated except ones and twos poisson regression model if the regression model accounts for the heterogeneity entirely the generalized chao estimator is asymptotically unbiased a simulation study illustrates the potential in gain of bias reduction comparisons of the generalized chao estimator with the homogeneous zero truncated poisson regression approach are supplied as well the method is applied to a surveillance study on the completeness of farm submissions in great britain