BACKGROUND
the advent of technologies to rapidly sequence entire genomes provides a resource of sequenced genomes spanning the entire tree of life  <cit> .
indeed, as the cost and time to sequence genomes have decreased, it has become possible to sequence multiple individuals from within a species.
re-sequencing efforts have led to the following discovery: the representation of gene families in isolates from the same bacterial species is highly variable  <cit> .
this variability poses conceptual as well as applied problems.
conceptually, the variability suggests the need to further re-visit species definitions that rely upon comparisons of highly conserved components of the genome, such as 16s rrna sequences  <cit> .
in addition, horizontal gene transfer and other genome rearrangements such as gene deletions and duplications can radically change the phenotype of a bacterium, even within individuals of the same species  <cit> .
for example, the introduction of toxin genes can render a bacterium pathogenic.
hence, from an applied perspective, there is an increasing need to quantify the gene diversity of a species or genus with pathogenic potential  <cit> .
the core and pan genome concepts have been proposed as a way to characterize the distribution of gene families within a group of organisms, e.g., within a species or genus  <cit> .
the core genome is the set of genes found in every organism within a group .
the pan genome is the set of all genes found within organisms of a group , including core genes and genes which appear in a fraction of genomes.
intuitively, the core genome preserves the notion that genomes of closely related organisms have something in common, while the pan genome is in accord with the finding that gene composition differs even among genomes of closely related organisms.
in that sense, the core and pan genome concepts begin to address both conceptual problems  and applied problems .
multiple attempts have been made to estimate the size of pan and core genomes in hopes of quantifying how open or closed a particular set of genomes is to gene exchange  <cit> .
however, estimating the actual list of genes in the pan and core genomes remains intractable.
thus far, attempts to quantify the size of the core and pan genomes have been based on extrapolations from a limited number of sequenced strains  to the entire group .
results of such extrapolations have been widely divergent.
in the most well-studied case, the pathogen streptococcus agalactiae, estimates of the pan genome size vary from tens of thousands  <cit>  to infinite  <cit> .
extreme variation in estimates of core and pan genome sizes makes it difficult to utilize these measures to quantify or compare the degree of acquisition and loss of gene families within a particular group or to make meaningful biological interpretations of the core and pan genome concepts.
one might suspect that robust quantification of core and pan genomes sizes could be achieved with improved statistical estimation methods, combined with increased sequencing coverage.
this is not the case.
the problem of estimating pan and core genome sizes will not be resolved by gradual improvements in sequencing.
in this paper we demonstrate that current methods to estimate pan and core genome sizes are statistically ill-posed.
we do so by demonstrating that sample gene distributions drawn from artificially generated groups of genomes with radically different pan and core genomes sizes are statistically indistinguishable.
in contrast, we present an alternative diversity metric, genomic fluidity, whose expected value is equivalent whether estimated from the sample or from the true gene distribution.
we then apply a bioinformatics pipeline so as to estimate genomic fluidity within  <dig> multiply-sequenced bacterial species containing  <dig> sequenced genomes.
we test the robustness of genomic fluidity to changes in the number of sequenced genomes as well as to changes in alignment parameters.
in so doing we demonstrate when it is possible to reliably rank order species in terms of genomic fluidity and discuss the implications of our work for inferring information about gene distributions based on subsamples.
RESULTS
pan and core genome sizes cannot be reliably estimated
we claim that current methods to estimate pan and core genome sizes are statistically ill-posed  <cit> .
to demonstrate this in a case where the pan and core genome sizes are known, we artificially generated gene distributions for three "species" such that their pan genome sizes were  <dig> ,  <dig> , and  <dig>  and their core genome sizes were  <dig> ,  <dig> , and  <dig>  .
note that species a and c had distinct gene frequency distributions despite having the same pan and core genome sizes.
next, we computationally generated ensembles of genomes for each species, each of which had  <dig> genes.
each gene in a genome was chosen at random from a frequency distribution specific to a given species, i.e., some genes occurred in all, or nearly all, genomes and some genes occurred very rarely.
importantly, a gene that only appears in  <dig> % of genomes  contributes as much to the pan genome as does a core gene , however, the rare gene will almost certainly not be detected in a sample set of tens or hundreds of sequenced genomes .
furthermore, none of the genes that are detected in the sample set of genomes provide any indication that this rare gene exists while performing standard rarefaction analysis .
in essence, the problem of estimating the pan genome is equivalent to estimating the level of rare genes, which, because they are rare, are recalcitrant to quantification.
similar difficulties are faced when trying to quantify the size of the core genome.
for example, a gene that appears in  <dig> % of genomes is technically not a core gene .
yet the rare genome without this core gene will not be detected in a sample set of genomes , nor will the sample provide any indication that an apparent core gene is absent from some small number of organisms in the group .
intuitively, both pan and core genome size estimates depend on accurate estimation of the frequency of rare events that any small sub-sample of sequenced genomes will not enable.
in principle, there may be cases where pan and core genome sizes can be accurately estimated from a subsample due to particularly low population gene diversity and/or the existence of particular parametric gene frequency distributions.
however, such cases will be difficult to identify, because of the difficulty in estimating how many rare genes and rare genomes exist in the population.
to further address this point, we consider alternative degrees of rarity, while continuously varying the rarest genes in the population from 10- <dig> to less than 10- <dig>  in doing so, we show that sample diversities plateau so long as the number of samples is sufficiently less than the inverse of the rarest gene .
for example, this means that one cannot estimate pan genome sizes using dozens or even hundreds of genomes if one expects that rare genes are found in one in a thousand  genomes .
hence, estimates of pan and core genome sizes may have no correspondence to true values.
despite all of these issues, the pan and core genome concepts have merit, even if their estimation is problematic.
instead, some alternative metric is needed that  is robust to small sample size ;  quantifies the relative degree of gene acquisition and loss within a group of genomes; and  validates prior expectations that gene diversity increases within groups of increasingly unrelated organisms.
genomic fluidity is a robust and reliable estimator of gene diversity
we propose the use of genomic fluidity, φ, as a robust diversity metric which can be applied to small numbers of sequenced genomes whether at the species level or amongst groups of increasingly unrelated organisms.
genomic fluidity is defined as the ratio of unique gene families to the sum of gene families in pairs of genomes averaged over randomly chosen genome pairs from within a group of n genomes:
ϕ=2n∑k,l= <dig> ..nk <luk+ulmk+ml.
where uk and ul are the number of gene families found only in genomes k and l respectively and mk and ml are the total number of gene families found in k and l respectively.
importantly, the same formula for fluidity applies whether n represents the total number of genomes in the population or n represents the total number of genomes in the sample.
in other words, genomic fluidity is an estimate of gene-ic dissimilarity, akin to similarity measures used in the study of ecological communities  <cit>  ).
more specifically, genomic fluidity estimates how dissimilar genomes are when evaluated at a gene level.
for example, a genomic fluidity of  <dig>  represents that a pair of genomes have on average 10% unique genes and share 90% of their genes.
as fluidity increases, so too does the probability that gene content differs between genomes in a sample.
genomic fluidity also provides information on novelty in sequencing projects.
to see how, note that the best estimate for the probability that a random gene from a newly sequenced genome is not found in a randomly selected prior sequenced genome is simply φ. importantly, genomic fluidity is robust to small sample size: it can be reliably estimated from a few sampled genomes.
for example, in figure  <dig> we show how the genomic fluidities for synthetically generated gene distributions are equivalent whether estimated from the true distribution or from a few dozen sampled genomes.
in addition, subtle differences in the genomic fluidity between two species can be detected from a small number of sampled genomes.
the estimated variance of fluidity was calculated using the jackknife estimate  <cit> , which is based on leave-one-out statistics .
in contrast, rarefaction curves used to estimate pan and core genome sizes are statistically indistinguishable for synthetically generated gene distributions, even when the underlying pan and core genome sizes are radically different .
fluidity and its variance can be estimated from a group of sequenced genomes
we developed a bioinformatics pipeline to estimate genomic fluidity at the species level among sequenced genomes , but later , we apply it to more diverse groups.
using this pipeline we calculated genomic fluidity for  <dig> species including  <dig> sequenced genomes from: bacillus anthracis, escherichia coli, neisseria meningitidis, staphylococcus aureus, streptococcus agalactiae, streptococcus pneumoniae, and streptococcus pyogenes .
we find that estimates of fluidity converge rapidly even when evaluated on a small number of sequenced genomes, as has been the case for all published studies of gene diversity within a species or genus.
these results are consistent with the rapid convergence of fluidity when estimated from synthetically generated genomes .
when applied to genomes from multiply resequenced bacterial species we find the mean value of fluidity is consistent when evaluated on a small subsample or on the entire sample .
we find convergence of fluidity estimates to approximately 10% relative standard deviation after a dozen or so genomes .
the variation in fluidity estimates found in small subsamples of sequenced genomes suggests caution should be applied in attempting to establish when we can reliably say that the fluidity of a particular species is greater than that of another.
importantly, the use of the jackknife estimate of variance permits us to evaluate how both the mean and the variance of fluidity converge as more genomes are added and provides a metric to indicate when sufficient sequencing has been accomplished for use in comparing relative values of fluidity between species or between groups.
rank-ordering of genomic fluidity is robust to variation in alignment parameters
the estimate of genomic fluidity varied with alignment parameters as expected.
when either minimum alignment identity or coverage is increased, more gene families are formed and fluidity increases .
nonetheless, the relative values of fluidity between species remained nearly invariant even as the magnitude of fluidity changed.
we applied the fluidity pipeline detailed in figure  <dig> and restricted our analysis to gene family assembly values of alignment identity  and coverage  from  <dig>  to  <dig>  in increments of  <dig>  .
in  <dig> trials, we found  <dig> distinct orderings of genomic fluidity, three of which accounted for 224/ <dig> orderings .
the robust rank-ordering suggests that it is possible to make comparative statements classifying one group as being more or less "open" to net gene acquisition.
specifically, we used the mean and variances estimates of fluidity to determine whether the φ of one species is significantly greater or less than another .
we find there is a statistically significant and unambiguous rank order of genomic fluidity for  <dig> =  <dig> comparisons of relative rank order among the  <dig> species examined in all  <dig> alignment parameter conditions corresponding to the dominant rank orderings .
in all conditions tested, b. anthracis had the lowest value of φ and either n. meningitidis or e. coli had the highest value of φ. further, strep.
agalactiae always had an intermediate value of fluidity.
however, strep.
agalactiae had the lowest number of available genomes and a particularly high variance; therefore we were unable to rank-order it relative to any other genome with the exception of b. anthracis.
these results are generally consistent with previous suggestions that b. anthracis has a closed genome, that n. meningitidis may have an open genome due to its natural competence, and that strep.
agalactiae has an open genome  <cit> .
however, now we can describe a group of organisms as being relatively open or closed, instead of being strictly open or strictly closed.
in addition, we can utilize variance estimates to suggest when greater sequencing is needed.
the comparison of the rank order of φ between species is consistent with recent calls  <cit>  to utilize the rank, not the absolute magnitude, when comparing the relative diversity of complex ecological communities.
this issue is particularly important in the case of gene diversity studies when identification of gene families is strongly depend on thresholds utilized in bioinformatics pipelines.
note that we do not suggest ranking of pan and core genome size estimates, since common genes and genomes do not, in general, inform estimates of rare genes and genomes, respectively.
genomic fluidity is a natural metric spanning phylogenetic scales from species to kingdom
thus far we have estimated genomic fluidity within a bacterial species, though the metric can be applied, in principle, to any group of genomes.
therefore, we estimated values of φ at the species level and at higher taxonomic groupings and found that φ varies from close to  <dig>  to nearly  <dig>  .
a phylogenetic tree of  <dig> bacterial species was assembled using amphora  <cit> .
species in this calculation were chosen to include those whose strain-level variation we had analyzed, as well as a hand-curated selection of genomes from different parts of the tree.
each leaf with a corresponding strain group therefore represents a collapsed subtree that clusters closely around the representative strain with respect to the overall tree.
the phylogenetic tree selected here is not meant to represent the entire diversity of life, but rather to illustrate how fluidity changes when closely and distantly related organisms are grouped together.
note the transition from relatively "solid" genomes at the level of isolates from within a bacterial species to a nearly totally "fluid" bacterial kingdom.
further, estimates of genomic fluidity are consistent with expectations that φ should increase as we move up the phylogenetic tree from species to genus to family, etc.
hence, we find that genomic fluidity is a natural metric for describing gene level similarity between groups of closely and distantly related organisms.
these results suggest the suitability of genomic fluidity at coarse-grained scales, e.g.
bacterial kingdom  <cit>  and microbial community levels  <cit> .
in contrast, estimates of pan-genome sizes at such scales will be problematic for the same reasons as outlined here when applied to closely related organisms.
as a general rule, similarity based approaches to quantifying other forms of genome diversity are likely to be robust whereas estimates of the total diversity will be less so.
discussion
the proposal that there exists a core and pan genome for bacterial species represents a significant advance in the conceptualization of gene variability within microorganisms  <cit> .
the basic premise of these two concepts have been borne out by the finding that the gene content of bacteria can vary significantly when comparing the sequence of two isolates from a species or genus  <cit> .
for example, it is now well established that some genes are found in most, if not all, sequenced genomes of isolates from within a sample.
in addition, it is also well established that some genes are found in very few, if only one, sequenced isolate within a sample.
however, as we have demonstrated here, efforts to infer the size of the pan and core genomes of an entire species or genus from the frequency distribution of genes within a small sample of sequenced genomes will almost certainly fail.
similarly, efforts to compare the core or pan genomes sizes of bacterial species or genera will be uninformative.
the reason is that pan and core genome sizes depend sensitively on the frequency of rare events  whose frequency cannot be accurately estimated from a small sample of sequenced genomes.
instead, we have proposed the use of an alternative diversity metric - genomic fluidity - which is a reliable and robust estimator of the gene dissimilarity amongst a group of sequenced genomes.
this study has a number of key implications for future sequencing efforts.
first, it suggests that efforts to understand a single species by sequencing as many isolates as possible may be limited in their ability to comprehensively define the diversity within that species  <cit> .
clearly, such studies will remain important in their ability to describe expected genomic differences .
next, our findings also suggest that the expected gene dissimilarity within a given species can be well characterized by sequencing a relatively small number of well-chosen representatives.
sequencing a few dozen genomes is a fairly straightforward task given recent advances in sequencing technology.
finally, perhaps the most far-reaching implication of the work presented here is that we have shown it is possible to compare the relative genomic fluidity of different groups of bacteria .
we have shown that genomic fluidity can reliably distinguish between subtle differences in true gene distributions  as well as determine when it is possible to rank-order a set of  <dig> species based on the analysis of  <dig> whole genomes .
genomic fluidity necessarily varies with the phylogenetic diversity of the group of genomes under consideration.
in many cases, this level of diversity is defined through a species or other group definition via observed phenotypic aspects, and not through any account of genome-level divergence.
as a result, within-species gene diversity of bacterial species varies greatly.
to facilitate fluidity-based comparisons between species, one possibility is to normalize genomic fluidity by the average or median phylogenetic distance between members of the considered group, such as the phylogenetic distance computed using a multiple alignment of housekeeping genes  <cit> .
however, other normalizations are possible and we consider this to be an important target for future research.
despite its merits, genomic fluidity is not meant to describe all forms of genome variation.
genomic fluidity can provide a reliable estimate for how many new genes additional sequencing is likely to reveal, with respect to a previously sequenced genome.
it cannot, however, provide an estimate of the amount of sequencing necessary to cover the gene novelty in the entire group .
in addition, genomic fluidity restricts itself to one component of genomic difference.
there are a variety of forms of genomic differences beyond gene compositional differences or the more classic finding of single-nucleotide polymorphisms.
genomes may differ in terms of gene synteny  <cit> , copy number variation  <cit> , plasmid and/or prophage presence  <cit> , codon biases  <cit> , and methylation state  <cit> .
it would be prudent to consider other diversity metrics, in addition to the metric of genomic fluidity studied here, that account for forms of variation in genome state amongst closely related organisms.
CONCLUSIONS
genomic fluidity is an integrated measure of gene diversity within a group of organisms.
genomic fluidity is both estimable given a small number of sequenced genomes and robust to variation in alignment parameters.
as such, we recommend that genomic fluidity be used in place of pan and core genome size estimates when assessing gene diversity within a species or a group of closely related organisms.
however, the precise relationship between variation in gene composition and genomic fluidity with underlying mechanisms of gene family diversification are yet to be resolved  <cit> .
recent calls for comparing and contrasting the average overlap of gene content with respect to average nucleotide divergence provide one possible route to disentangling the effects of ecological and genomic structure  <cit> , but much work remains at the interface of bioinformatics and ecological analysis.
for example, the detailed comparison of complete bacterial genomes from closely related biofilm-forming bacteria revealed how and why different organisms have adapted to and shaped their environment  <cit> .
similarly, genomic analysis of cyanoviruses sampled in the oceans helped uncover photosynthetic pathways which enable the exploitation of a niche distinct from previously cultured e. coli based phages despite sharing many common genes and genome architecture  <cit> .
genomic fluidity complements the detailed functional comparison of genomes by robustly estimating dissimilarity of genes within groups of genomes and providing insight into their potential evolvability.
in so doing, our results highlight the need for continued focus on developing new toolsets for assessing what can be inferred about the genome composition and diversity of prokaryotic species and communities based on analysis of a sub-sample of genomes.

 the advent of technologies to rapidly sequence entire genomes provides a resource of sequenced genomes spanning the entire tree of life. indeed, as the cost and time to sequence genomes have decreased, it has become possible to sequence multiple individuals from within a species. re-sequencing efforts have led to the following discovery: the representation of gene families in isolates from the same bacterial species is highly variable. this variability poses conceptual as well as applied problems. conceptually, the variability suggests the need to further re-visit species definitions that rely upon comparisons of highly conserved components of the genome, such as 16s rrna sequences. in addition, horizontal gene transfer and other genome rearrangements such as gene deletions and duplications can radically change the phenotype of a bacterium, even within individuals of the same species. for example, the introduction of toxin genes can render a bacterium pathogenic. hence, from an applied perspective, there is an increasing need to quantify the gene diversity of a species or genus with pathogenic potential. the core and pan genome concepts have been proposed as a way to characterize the distribution of gene families within a group of organisms, e.g., within a species or genus. the core genome is the set of genes found in every organism within a group. the pan genome is the set of all genes found within organisms of a group, including core genes and genes which appear in a fraction of genomes. intuitively, the core genome preserves the notion that genomes of closely related organisms have something in common, while the pan genome is in accord with the finding that gene composition differs even among genomes of closely related organisms. in that sense, the core and pan genome concepts begin to address both conceptual problems and applied problems. multiple attempts have been made to estimate the size of pan and core genomes in hopes of quantifying how open or closed a particular set of genomes is to gene exchange. however, estimating the actual list of genes in the pan and core genomes remains intractable. thus far, attempts to quantify the size of the core and pan genomes have been based on extrapolations from a limited number of sequenced strains to the entire group. results of such extrapolations have been widely divergent. in the most well-studied case, the pathogen streptococcus agalactiae, estimates of the pan genome size vary from tens of thousands to infinite. extreme variation in estimates of core and pan genome sizes makes it difficult to utilize these measures to quantify or compare the degree of acquisition and loss of gene families within a particular group or to make meaningful biological interpretations of the core and pan genome concepts. one might suspect that robust quantification of core and pan genomes sizes could be achieved with improved statistical estimation methods, combined with increased sequencing coverage. this is not the case. the problem of estimating pan and core genome sizes will not be resolved by gradual improvements in sequencing. in this paper we demonstrate that current methods to estimate pan and core genome sizes are statistically ill-posed. we do so by demonstrating that sample gene distributions drawn from artificially generated groups of genomes with radically different pan and core genomes sizes are statistically indistinguishable. in contrast, we present an alternative diversity metric, genomic fluidity, whose expected value is equivalent whether estimated from the sample or from the true gene distribution. we then apply a bioinformatics pipeline so as to estimate genomic fluidity within multiply-sequenced bacterial species containing sequenced genomes. we test the robustness of genomic fluidity to changes in the number of sequenced genomes as well as to changes in alignment parameters. in so doing we demonstrate when it is possible to reliably rank order species in terms of genomic fluidity and discuss the implications of our work for inferring information about gene distributions based on subsamples.  pan and core genome sizes cannot be reliably estimated we claim that current methods to estimate pan and core genome sizes are statistically ill-posed. to demonstrate this in a case where the pan and core genome sizes are known, we artificially generated gene distributions for three "species" such that their pan genome sizes were,, and and their core genome sizes were,, and. note that species a and c had distinct gene frequency distributions despite having the same pan and core genome sizes. next, we computationally generated ensembles of genomes for each species, each of which had genes. each gene in a genome was chosen at random from a frequency distribution specific to a given species, i.e., some genes occurred in all, or nearly all, genomes and some genes occurred very rarely. importantly, a gene that only appears in % of genomes contributes as much to the pan genome as does a core gene, however, the rare gene will almost certainly not be detected in a sample set of tens or hundreds of sequenced genomes. furthermore, none of the genes that are detected in the sample set of genomes provide any indication that this rare gene exists while performing standard rarefaction analysis. in essence, the problem of estimating the pan genome is equivalent to estimating the level of rare genes, which, because they are rare, are recalcitrant to quantification. similar difficulties are faced when trying to quantify the size of the core genome. for example, a gene that appears in % of genomes is technically not a core gene. yet the rare genome without this core gene will not be detected in a sample set of genomes, nor will the sample provide any indication that an apparent core gene is absent from some small number of organisms in the group. intuitively, both pan and core genome size estimates depend on accurate estimation of the frequency of rare events that any small sub-sample of sequenced genomes will not enable. in principle, there may be cases where pan and core genome sizes can be accurately estimated from a subsample due to particularly low population gene diversity and/or the existence of particular parametric gene frequency distributions. however, such cases will be difficult to identify, because of the difficulty in estimating how many rare genes and rare genomes exist in the population. to further address this point, we consider alternative degrees of rarity, while continuously varying the rarest genes in the population from 10- to less than 10- in doing so, we show that sample diversities plateau so long as the number of samples is sufficiently less than the inverse of the rarest gene. for example, this means that one cannot estimate pan genome sizes using dozens or even hundreds of genomes if one expects that rare genes are found in one in a thousand genomes. hence, estimates of pan and core genome sizes may have no correspondence to true values. despite all of these issues, the pan and core genome concepts have merit, even if their estimation is problematic. instead, some alternative metric is needed that is robust to small sample size; quantifies the relative degree of gene acquisition and loss within a group of genomes; and validates prior expectations that gene diversity increases within groups of increasingly unrelated organisms. genomic fluidity is a robust and reliable estimator of gene diversity we propose the use of genomic fluidity, φ, as a robust diversity metric which can be applied to small numbers of sequenced genomes whether at the species level or amongst groups of increasingly unrelated organisms. genomic fluidity is defined as the ratio of unique gene families to the sum of gene families in pairs of genomes averaged over randomly chosen genome pairs from within a group of n genomes: ϕ=2n∑k,l=..nk <luk+ulmk+ml. where uk and ul are the number of gene families found only in genomes k and l respectively and mk and ml are the total number of gene families found in k and l respectively. importantly, the same formula for fluidity applies whether n represents the total number of genomes in the population or n represents the total number of genomes in the sample. in other words, genomic fluidity is an estimate of gene-ic dissimilarity, akin to similarity measures used in the study of ecological communities ). more specifically, genomic fluidity estimates how dissimilar genomes are when evaluated at a gene level. for example, a genomic fluidity of represents that a pair of genomes have on average 10% unique genes and share 90% of their genes. as fluidity increases, so too does the probability that gene content differs between genomes in a sample. genomic fluidity also provides information on novelty in sequencing projects. to see how, note that the best estimate for the probability that a random gene from a newly sequenced genome is not found in a randomly selected prior sequenced genome is simply φ. importantly, genomic fluidity is robust to small sample size: it can be reliably estimated from a few sampled genomes. for example, in figure we show how the genomic fluidities for synthetically generated gene distributions are equivalent whether estimated from the true distribution or from a few dozen sampled genomes. in addition, subtle differences in the genomic fluidity between two species can be detected from a small number of sampled genomes. the estimated variance of fluidity was calculated using the jackknife estimate, which is based on leave-one-out statistics. in contrast, rarefaction curves used to estimate pan and core genome sizes are statistically indistinguishable for synthetically generated gene distributions, even when the underlying pan and core genome sizes are radically different. fluidity and its variance can be estimated from a group of sequenced genomes we developed a bioinformatics pipeline to estimate genomic fluidity at the species level among sequenced genomes, but later, we apply it to more diverse groups. using this pipeline we calculated genomic fluidity for species including sequenced genomes from: bacillus anthracis, escherichia coli, neisseria meningitidis, staphylococcus aureus, streptococcus agalactiae, streptococcus pneumoniae, and streptococcus pyogenes. we find that estimates of fluidity converge rapidly even when evaluated on a small number of sequenced genomes, as has been the case for all published studies of gene diversity within a species or genus. these results are consistent with the rapid convergence of fluidity when estimated from synthetically generated genomes. when applied to genomes from multiply resequenced bacterial species we find the mean value of fluidity is consistent when evaluated on a small subsample or on the entire sample. we find convergence of fluidity estimates to approximately 10% relative standard deviation after a dozen or so genomes. the variation in fluidity estimates found in small subsamples of sequenced genomes suggests caution should be applied in attempting to establish when we can reliably say that the fluidity of a particular species is greater than that of another. importantly, the use of the jackknife estimate of variance permits us to evaluate how both the mean and the variance of fluidity converge as more genomes are added and provides a metric to indicate when sufficient sequencing has been accomplished for use in comparing relative values of fluidity between species or between groups. rank-ordering of genomic fluidity is robust to variation in alignment parameters the estimate of genomic fluidity varied with alignment parameters as expected. when either minimum alignment identity or coverage is increased, more gene families are formed and fluidity increases. nonetheless, the relative values of fluidity between species remained nearly invariant even as the magnitude of fluidity changed. we applied the fluidity pipeline detailed in figure and restricted our analysis to gene family assembly values of alignment identity and coverage from to in increments of. in trials, we found distinct orderings of genomic fluidity, three of which accounted for 224/ orderings. the robust rank-ordering suggests that it is possible to make comparative statements classifying one group as being more or less "open" to net gene acquisition. specifically, we used the mean and variances estimates of fluidity to determine whether the φ of one species is significantly greater or less than another. we find there is a statistically significant and unambiguous rank order of genomic fluidity for = comparisons of relative rank order among the species examined in all alignment parameter conditions corresponding to the dominant rank orderings. in all conditions tested, b. anthracis had the lowest value of φ and either n. meningitidis or e. coli had the highest value of φ. further, strep. agalactiae always had an intermediate value of fluidity. however, strep. agalactiae had the lowest number of available genomes and a particularly high variance; therefore we were unable to rank-order it relative to any other genome with the exception of b. anthracis. these results are generally consistent with previous suggestions that b. anthracis has a closed genome, that n. meningitidis may have an open genome due to its natural competence, and that strep. agalactiae has an open genome. however, now we can describe a group of organisms as being relatively open or closed, instead of being strictly open or strictly closed. in addition, we can utilize variance estimates to suggest when greater sequencing is needed. the comparison of the rank order of φ between species is consistent with recent calls to utilize the rank, not the absolute magnitude, when comparing the relative diversity of complex ecological communities. this issue is particularly important in the case of gene diversity studies when identification of gene families is strongly depend on thresholds utilized in bioinformatics pipelines. note that we do not suggest ranking of pan and core genome size estimates, since common genes and genomes do not, in general, inform estimates of rare genes and genomes, respectively. genomic fluidity is a natural metric spanning phylogenetic scales from species to kingdom thus far we have estimated genomic fluidity within a bacterial species, though the metric can be applied, in principle, to any group of genomes. therefore, we estimated values of φ at the species level and at higher taxonomic groupings and found that φ varies from close to to nearly. a phylogenetic tree of bacterial species was assembled using amphora. species in this calculation were chosen to include those whose strain-level variation we had analyzed, as well as a hand-curated selection of genomes from different parts of the tree. each leaf with a corresponding strain group therefore represents a collapsed subtree that clusters closely around the representative strain with respect to the overall tree. the phylogenetic tree selected here is not meant to represent the entire diversity of life, but rather to illustrate how fluidity changes when closely and distantly related organisms are grouped together. note the transition from relatively "solid" genomes at the level of isolates from within a bacterial species to a nearly totally "fluid" bacterial kingdom. further, estimates of genomic fluidity are consistent with expectations that φ should increase as we move up the phylogenetic tree from species to genus to family, etc. hence, we find that genomic fluidity is a natural metric for describing gene level similarity between groups of closely and distantly related organisms. these results suggest the suitability of genomic fluidity at coarse-grained scales, e.g. bacterial kingdom and microbial community levels. in contrast, estimates of pan-genome sizes at such scales will be problematic for the same reasons as outlined here when applied to closely related organisms. as a general rule, similarity based approaches to quantifying other forms of genome diversity are likely to be robust whereas estimates of the total diversity will be less so. discussion the proposal that there exists a core and pan genome for bacterial species represents a significant advance in the conceptualization of gene variability within microorganisms. the basic premise of these two concepts have been borne out by the finding that the gene content of bacteria can vary significantly when comparing the sequence of two isolates from a species or genus. for example, it is now well established that some genes are found in most, if not all, sequenced genomes of isolates from within a sample. in addition, it is also well established that some genes are found in very few, if only one, sequenced isolate within a sample. however, as we have demonstrated here, efforts to infer the size of the pan and core genomes of an entire species or genus from the frequency distribution of genes within a small sample of sequenced genomes will almost certainly fail. similarly, efforts to compare the core or pan genomes sizes of bacterial species or genera will be uninformative. the reason is that pan and core genome sizes depend sensitively on the frequency of rare events whose frequency cannot be accurately estimated from a small sample of sequenced genomes. instead, we have proposed the use of an alternative diversity metric - genomic fluidity - which is a reliable and robust estimator of the gene dissimilarity amongst a group of sequenced genomes. this study has a number of key implications for future sequencing efforts. first, it suggests that efforts to understand a single species by sequencing as many isolates as possible may be limited in their ability to comprehensively define the diversity within that species. clearly, such studies will remain important in their ability to describe expected genomic differences. next, our findings also suggest that the expected gene dissimilarity within a given species can be well characterized by sequencing a relatively small number of well-chosen representatives. sequencing a few dozen genomes is a fairly straightforward task given recent advances in sequencing technology. finally, perhaps the most far-reaching implication of the work presented here is that we have shown it is possible to compare the relative genomic fluidity of different groups of bacteria. we have shown that genomic fluidity can reliably distinguish between subtle differences in true gene distributions as well as determine when it is possible to rank-order a set of species based on the analysis of whole genomes. genomic fluidity necessarily varies with the phylogenetic diversity of the group of genomes under consideration. in many cases, this level of diversity is defined through a species or other group definition via observed phenotypic aspects, and not through any account of genome-level divergence. as a result, within-species gene diversity of bacterial species varies greatly. to facilitate fluidity-based comparisons between species, one possibility is to normalize genomic fluidity by the average or median phylogenetic distance between members of the considered group, such as the phylogenetic distance computed using a multiple alignment of housekeeping genes. however, other normalizations are possible and we consider this to be an important target for future research. despite its merits, genomic fluidity is not meant to describe all forms of genome variation. genomic fluidity can provide a reliable estimate for how many new genes additional sequencing is likely to reveal, with respect to a previously sequenced genome. it cannot, however, provide an estimate of the amount of sequencing necessary to cover the gene novelty in the entire group. in addition, genomic fluidity restricts itself to one component of genomic difference. there are a variety of forms of genomic differences beyond gene compositional differences or the more classic finding of single-nucleotide polymorphisms. genomes may differ in terms of gene synteny, copy number variation, plasmid and/or prophage presence, codon biases, and methylation state. it would be prudent to consider other diversity metrics, in addition to the metric of genomic fluidity studied here, that account for forms of variation in genome state amongst closely related organisms.  genomic fluidity is an integrated measure of gene diversity within a group of organisms. genomic fluidity is both estimable given a small number of sequenced genomes and robust to variation in alignment parameters. as such, we recommend that genomic fluidity be used in place of pan and core genome size estimates when assessing gene diversity within a species or a group of closely related organisms. however, the precise relationship between variation in gene composition and genomic fluidity with underlying mechanisms of gene family diversification are yet to be resolved. recent calls for comparing and contrasting the average overlap of gene content with respect to average nucleotide divergence provide one possible route to disentangling the effects of ecological and genomic structure, but much work remains at the interface of bioinformatics and ecological analysis. for example, the detailed comparison of complete bacterial genomes from closely related biofilm-forming bacteria revealed how and why different organisms have adapted to and shaped their environment. similarly, genomic analysis of cyanoviruses sampled in the oceans helped uncover photosynthetic pathways which enable the exploitation of a niche distinct from previously cultured e. coli based phages despite sharing many common genes and genome architecture. genomic fluidity complements the detailed functional comparison of genomes by robustly estimating dissimilarity of genes within groups of genomes and providing insight into their potential evolvability. in so doing, our results highlight the need for continued focus on developing new toolsets for assessing what can be inferred about the genome composition and diversity of prokaryotic species and communities based on analysis of a sub-sample of genomes. 
the dual concepts of pan and core genomes have been widely adopted as means to assess the distribution of gene families within microbial species and genera. we use a combination of mathematical, statistical and computational methods to show that current predictions of pan and core genome sizes may have no correspondence to true values. a variety of methods have provided drastically different estimates of the sizes of pan and core genomes from sequenced representatives of the same groups of bacteria. pan and core genome size estimates are problematic because they depend on the estimation of the occurrence of rare genes and genomes, respectively, which are difficult to estimate precisely because they are rare. the core genome is the set of genes shared by a group of organisms; the pan genome is the set of all genes seen in any of these organisms. as such, we recommend that genomic fluidity be used in place of pan and core genome size estimates when assessing gene diversity within genomes of a species or a group of closely related organisms. instead, we introduce and evaluate a robust metric - genomic fluidity - to categorize the gene-level similarity among groups of sequenced isolates. genomic fluidity is a measure of the dissimilarity of genomes evaluated at the gene level. the genomic fluidity of a population can be estimated accurately given a small number of sequenced genomes. further, the genomic fluidity of groups of organisms can be compared robustly despite variation in algorithms used to identify genes and their homologs.
 the dual concepts of pan and core genomes have been widely adopted as means to assess the distribution of gene families within microbial species and genera. the core genome is the set of genes shared by a group of organisms; the pan genome is the set of all genes seen in any of these organisms. a variety of methods have provided drastically different estimates of the sizes of pan and core genomes from sequenced representatives of the same groups of bacteria.  we use a combination of mathematical, statistical and computational methods to show that current predictions of pan and core genome sizes may have no correspondence to true values. pan and core genome size estimates are problematic because they depend on the estimation of the occurrence of rare genes and genomes, respectively, which are difficult to estimate precisely because they are rare. instead, we introduce and evaluate a robust metric - genomic fluidity - to categorize the gene-level similarity among groups of sequenced isolates. genomic fluidity is a measure of the dissimilarity of genomes evaluated at the gene level.  the genomic fluidity of a population can be estimated accurately given a small number of sequenced genomes. further, the genomic fluidity of groups of organisms can be compared robustly despite variation in algorithms used to identify genes and their homologs. as such, we recommend that genomic fluidity be used in place of pan and core genome size estimates when assessing gene diversity within genomes of a species or a group of closely related organisms. 