BACKGROUND
repeat sequences are a common feature of prokaryote and eukaryote genomes  <cit>  and in both types of organisms the selective neutrality or otherwise of extra copies of sequences has been debated for decades  <cit> .
since the start of the genomics era in the mid-1990s the hitherto unexpectedly large amount of repetitive sequences found in bacteria, which may account for more than 10% of the total genome, prompted a flurry of investigations of the functional and evolutionary significance of these elements  <cit> .
more recently, aras et al.
surveyed  <dig> bacterial genomes to quantify the effect repeat sequences might have on genome plasticity due to intragenomic recombination  <cit> .
the authors conclude that in bacteria repeats might be selected for their positive effect on the adaptability of their host  <cit> .
in another in silico survey of  <dig> completely sequenced bacteria, achaz et al.
noted that inverted repeats are underrepresented in bacterial genomes due to their destabilizing effect on genome structure  <cit> .
in eukaryotes the discrepancy between dna content and apparent organismic complexity had been noted even before the discovery of the double helix leading to the conclusion that "the relationship between dna and the size or number of genes is obscure" .
in the 1960s dna reannealing studies uncovered that eukaryotic genomes contain a highly variable fraction of repetitive dna.
since the sequencing of complex genomes these observations have been made precise: approximately 50% of the human genome is made up of repetitive sequences  <cit> .
however, the term "repetitive sequences" encompasses a rather heterogeneous set of elements: 45% of the human genome is covered by transposons, 3% are repeats of less than a hundred base pairs , and 5% consist of recent duplications of large segments of dna.
broadly similar observations have been made in other mammalian genomes  <cit> .
the human genome contains low, but appreciable, genetic variation caused by transposable elements, indicating that transposable elements have been active over the short time span since humans diverged from their last common ancestor  <cit> .
however, the decline of transposon activity in the hominoid lineage contrasts with more recent insertions in mouse, where new spontaneous mutations are  <dig> times more likely to be caused by transposition than in human  <cit> .
the hypothesis that transposable elements are molecular parasites was originally designed to explain the apparently excessive dna baggage of eukaryotes  <cit> .
a number of contemporary observations support this view.
transposon-derived sequences are rare close to transcription start sites and inside coding regions, suggesting that insertions are usually deleterious  <cit> .
moreover, the four human hox clusters and other highly regulated genomic regions contain very few transposable elements  <cit> .
direct deletion of megabase-sized regions devoid of known genes also seems to have no effect on mice, even though these regions contain elements that have been conserved since the emergence of mammals  <cit> .
there is no contradiction between these observations and the fact that occasionally transposable elements can give rise to beneficial structures including novel gene regulatory regions  <cit>  and the vj recombination mechanism that generates the antibody diversity expressed by vertebrate b cells  <cit> .
since the publication of whole genome data, the quantification and classification of repeat elements has become a major area of research in computational biology  <cit> .
perhaps the best-known program for the detection of repeat elements is repeatmasker  <cit> , which looks for two things:  tandem repeats of a few nucleotides, and  homology to known repetitive elements.
this approach has the advantage of dealing with elements of known origin.
its disadvantage is that the presence of hitherto unknown repetitive elements might be missed.
the program repeatfinder implements a highly efficient and more generic approach based on suffix trees that makes no assumptions about the type of repeat present  <cit> .
such methods can be used to compute, for example, the percentage of a given dna sequence covered by repeats and most methods provide a means of checking the statistical significance of the repeats returned.
suffix trees allow the efficient detection of all exact repeats in a sequence.
in contrast, the widely used relative simplicity factor   <cit>  is based on the local density of repeat motifs up to four bases long compared to their density in a shuffled version of the input sequence  <cit> .
application of the rsf to diverse genomes revealed that eukaryotes are characterized by an elevated "micro-repetitiveness" compared to prokaryotes  <cit> .
what is lacking, though, is an all-inclusive measure of repetitiveness.
under the rsf repetitiveness is defined as a quantity that is minimized by shuffling the investigated sequence.
as suggested by the term simplicity factor, studies of repetitiveness are related to investigations of complexity  <cit>  – if repetitiveness is high, complexity is low, though the converse is not always true.
for example, the "linguistic complexity" of a string s is defined as the number of substrings of lengths  <dig>   <dig>  ..., |s| observed in s compared to the maximum number of substrings of these lengths  <cit> .
a random dna sequence with g/c content  <dig>  has maximal complexity and minimal repetitiveness.
however, a random dna sequence with a g/c content of, say,  <dig>  does not have maximal complexity, while its repetitiveness should still be minimal.
in this paper we propose a novel measure of repetitiveness which considers repeats of any length, takes into account g/c content, and does not necessitate shuffling for its computation.
as explained in detail in the methods section, our index of repetitiveness, ir, is expected to be zero in random dna sequences of any g/c content and length, and can be computed in time proportional to sequence length.
we apply the ir to  <dig> sequenced bacterial genomes,  <dig> archaebacteria, and six model eukaryotes: baker's yeast , nematode worm , thale cress , fruit fly , mouse , and human .
RESULTS
our first goal was to establish the null distribution of ir.
this can be obtained by shuffling a genomic sequence.
as an example we repeatedly randomized the genome of bacteriophage λ, which consists of  <dig>  bp of dna, and calculated the ir from these "repeatless" sequences.
figure  <dig> shows the resulting histogram, which is symmetrically distributed around a mean close to the expected zero .
further analysis of this distribution using the shapiro-wilk test  <cit>  revealed that deviation from normality increased as more replicates were added .
the reason for this was an increase in kurtosis , while the skewness  decreased with higher replication.
notice also that the ir of the unshuffled λ genome is significantly greater than its randomized version.
this is not surprising, as biological sequences are no more random sequences of residues than prose is a random sequence of letters.
survey of ir values
we calculated ir values for  <dig> completely sequenced prokaryote genomes, as well as for representative eukaryotic model organisms: baker's yeast , nematode worm , thale cress , and fruit fly .
figure 2a displays the ir values of eubacteria as a function of the log genome size .
in this domain of life ir was not correlated with log genome size .
the average ir of eubacteria was  <dig> .
<dig> % of bacteria had an ir ≤  <dig>  on the other hand, there were  <dig> bacteria where ir >  <dig>  with the highest value found in methylobacillus flagellatus kt .
the other members of this group were streptococcus agalactiae nem <dig> , dehalococcoides ethenogenes  <dig> , francisella tularensis subsp.
tularensis schu s <dig> , neisseria meningitidis mc <dig> , francisella tularensis subsp.
holarctica , and escherichia coli o157:h <dig> edl <dig> .
at the other extreme of the distribution, buchnera aphidicola str.
bp had the smallest ir value , which was even smaller than that observed in phage λ .
with one exception the ten eubacteria with the lowest ir values comprised only intracellular organisms sampled form the genera buchnera, chlamydophila, candidatus, neorickettsia, and rickettsia.
the exception was the highly abundant photosynthetic bacterium prochlorococcus marinus subsp.
marinus str.
ccmp <dig> .
in order to further investigate some of the extreme ir values observed in eubacteria , we subjected them to sliding window analyses.
figure 3a shows such an analysis for m. flagellatus kt and reveals that its global ir value  was caused by two large peaks of local ir indicating the presence of a very long exact repeat .
this turned out to be a tandem repeat comprising an astonishing  <dig>  bp.
removal of one copy of this duplication lead to a much deflated ir of  <dig> .
however, not all large ir values among eubacteria were caused by single exact repeats.
figure 3b displays a sliding window analysis of the genome of onion yellows phytoplasma oy-m, which had a global ir value of  <dig>  .
a scan of its local ir values indicated the presence of numerous regions of significant repetitiveness .
the bacterium with the second highest global ir-value, strepotococcus agalactiae nem <dig>  was an outlier among the other  <dig> streptococci investigated, which have an average ir of  <dig>  .
window analysis of s. agalactiae nem <dig> revealed three exact repeats of  <dig> kb  and their removal resulted in an ir of  <dig> .
similarly, escherichia coli oh157:h <dig> edl <dig> had an exceptionally high ir of  <dig>   compared to the other five strains of e. coli sampled .
in this case window analysis of e. coli oh157:h <dig> edl <dig>  highlighted a repeat region of approximately  <dig> kb located at positions  <dig> ,000– <dig> , <dig> and  <dig> ,000– <dig> , <dig>  which contained several long exact repeats with the longest spanning over  <dig> kb.
removal of one copy of the  <dig> kb repeat region reduced the ir to  <dig> .
mouse and human chromosomes
the average ir for human chromosomes was  <dig>  and values for individual chromosomes ranged from  <dig>  in chromosome  <dig> to  <dig>  in the y chromosome .
the y chromosome was the only human chromosome with ir >  <dig>  which agrees with the view that it has the highest dna turnover in the genome  <cit> .
the average ir for mouse chromosomes was  <dig>  , which is significantly larger than that of humans .
this agrees with the observation that the rodent lineage has experienced a higher rate of retro-transposition than hominoids  <cit> .
individual mouse chromosomes had ir values ranging from  <dig>  in chromosome  <dig> to  <dig>  in the y chromosome.
as in the human genome, the y chromosome from mouse was characterized by the largest ir.
in addition, chromosomes  <dig> and x had ir values >  <dig> .
hox genes in human and d. melanogaster
the hox genes encode transcription factors that function as fundamental developmental switches in all animals.
in human the four clusters of hox genes contain very few insertion sequences  <cit> .
to assess the effect of this on the landscape of human ir values, we carried out a sliding window analysis of  <dig> mb around the hoxa cluster on chromosome  <dig>  figure 5a displays the conspicuous footprint of low ir values that coincides with the location of the hoxa cluster.
in order to make this eye-ball analysis more quantitative, we searched the fragment of chromosome  <dig> displayed in figure 5a for runs of ir ≤  <dig> that extended for at least  <dig> kb.
this uncovered  <dig> intervals ranging in size from  <dig>  to  <dig>  kb .
ten of these intervals were located within the hoxa cluster.
the remaining three arrows are marked by stars in figure  <dig>  two of the corresponding regions with low ir values intersected with scap <dig>  a src family associated phosphoprotein involved in signal transduction leading to t cell activation  <cit> .
the last region of low ir outside of the hoxa region intersected with evx <dig>  this is a homologue of the even-skipped homeobox gene originally discovered in d. melanogaster.
in vertebrates it is involved in eye development  <cit> .
human evx <dig> is located just  <dig>  kb upstream from the most 5' of the hoxa genes, hoxa <dig> .
a sliding window analysis of the antennapedia complex in d. melanogaster, which is homologous to part of the human hoxa cluster, revealed a very different topology of repetitiveness .
on a background of ir ≈  <dig>  large peaks marked the presence of long exact repeats and the antennapedia cluster was not characterized by a conspicuous change in ir values.
discussion
"at this point we do not know what most of the dna in eukaryotes is doing" .
today, thirty-five years later, the function of apparently excess dna in both eukaryotes and prokaryotes remains a topic of intense research activity  <cit> .
our method to quantify this excess dna, the index of repetitiveness, is close in spirit to the investigation of linguistic complexity based on suffix trees  <cit> .
linguistic complexity is maximized in random sequences with equiprobable residues.
deviations from equiprobability lead to a reduction in complexity even if the sequence remains completely random.
in contrast, in this paper we were interested in quantifying repetitiveness with respect to genome composition and to make this measure comparable across genomes.
our starting point was an investigation of the complement of repeats, the unique sequences.
these are trivially easy to find, for example a sequence is always unique with respect to itself, and for this reason we have concentrated on shortest unique substrings.
a shortest unique substring occurs only once in its parent string and cannot be reduced in length without losing its uniqueness.
a genome with many long repeats contains many excessively long shortest unique substrings, while its shuffled version contains only the shortest unique substrings expected to be there by chance alone .
since we have derived the latter quantity analytically  <cit> , the ir is constructed as the logarithm of the ratio between the observed and expected aggregate number of nucleotides found in shortest unique substrings.
at the cost of ignoring homology relationships, this measure has the advantage that it can be computed for any double-stranded dna sequence and its expectation is always zero.
it is also possible to estimate an ir value for sequences over alphabets other than the four nucleotides.
in this case the quantity ae defined in equation  can be estimated by shuffling the input sequence.
for example, the ir of this paper is approximately  <dig> .
since the construction of the underlying suffix tree takes only time proportional to the length of the sequence analyzed, the ir can be computed in time proportional to the length of the input sequence.
in contrast, traditional repeat analysis such as implemented in the program repeatmasker  <cit>  runs in time proportional to the product of the length of query and subject sequence.
like most suffix tree implementations, the suffix tree on which our analysis is based, is kept entirely in the main memory  of the computer  <cit> .
this has the advantage of being relatively easy to implement.
the disadvantage of this approach is that the amount of sequence data that can be analyzed in a single run of the program is limited by the available ram rather than by the much cheaper hard disk space.
we are currently studying advances in disk-based suffix tree construction  <cit>  in order to break through the ram barrier.
it may come as a surprise that the ir values for human and mouse chromosomes were within the range of ir values observed for less complex eubacterial genomes .
however, this does not contradict the well-known fact that mammalian genomes are full of interspersed repeats, while bacteria usually contain fewer of these elements.
the apparent paradox is due to the fact that the effect of interspersed repeats on the excess amount of exact repeats in a given genome – which is what the ir measures – depends not only on the fraction of sequence covered by repetitive elements; equally important is the number of mutations accumulated since the divergence of an interspersed repeat from its most recent ancestor.
as a result of the mutation process, ancient repetitive elements may not contain longer motifs repeated elsewhere than the rest of the genome.
the presence of such elements would leave the ir unchanged compared to the identical genome without them.
a similar argument applies to the interpretation of the high ir values found in the y chromosomes of human and mouse.
the two factors determining the accumulation of sequence polymorphisms, time to the most recent common ancestor and mutation rate, cannot be separated.
in addition, the effective mutation rate differs between autosomes and the y chromosome.
under neutrality the number of snps expected for a pair of homologous sequences is θ = 4neμ, where ne is the effective population size and μ the rate of mutation.
since the effective population size of mammalian y chromosomes is only one quarter that of autosomes, repeat pairs on the y chromosome are broken up more slowly by mutations than elsewhere in the genome contributing to higher ir values.
it should be noted at this point that neither the mouse nor the human genome are completely sequenced to date.
if new sequence data comes predominantly from regions that are difficult to sequence due to their repetitiveness, future editions of the human and mouse genomes are expected to have higher ir.
the ir values found in our whole genome analyses  correlate well with the relative simplicity factors  reported previously  <cit>  .
this correlation is not perfect due to the fact that the rsf measures the local excess of short repeats, while the ir measures the excess of all repeats throughout the sequence.
moreover, no significant correlation between archaebacterial genome size and rsf was observed by hancock  <cit> , in contrast to our finding.
this effect, however, is simply due to differences in sampling; if we reduce our sample of  <dig> archaebacterial genomes to the nine investigated by hancock, the correlation between ir and log genome size also vanishes.
in contrast, a tenfold increase in the number of bacterial genomes investigated between hancock's and our study only confirmed the earlier diagnosis of no correlation between rsf and genome size.
the average ir for eubacteria was  <dig> .
however, it is clear that there are a few extreme ir values that inflate this average .
the largest ir for bacteria  was found in methylobacillus flagellatus kt .
this value was the most extreme of a set of seven organisms with ir >  <dig> that also included the human pathogens neisseria meningitidis mc <dig> and escherichia coli o157:h <dig> edl <dig> .
in a previous survey of  <dig> bacteria, neisseria meningitidis was already singled out as having a highly repetitive genome  <cit> .
the low ir values found by us among obligately host-associated bacteria also agree with a known lack of repeats in these genomes  <cit> .
while other bacteria appear to harbor repeats to increase genome plasticity  <cit> , we speculate that intracellular symbionts and pathogens are less dependent on genome shuffling for their survival as they live in more stable environments.
our sliding window analyses revealed that the computation of ir values for entire genomes averages out sharp regional fluctuations in ir .
in bacteria a high ir value may be caused by a few extreme duplications, as was the case for m. flagellatus kt  and s. agalactiae nem <dig>  in the human genome the  <dig> genes making up the hoxa cluster were characterized by a  <dig> kb footprint of low ir values .
the fact that additional runs of low ir outside the hoxa cluster also coincided with known genes leads us to currently search the entire human genome for further regions of low ir.
CONCLUSIONS
investigations of repetitiveness are traditionally carried out using some form of alignment algorithm.
such algorithms tend to run in time proportional to the product of the length of the query and subject sequence.
in this paper we present an approach that runs in time linear in the length of the input sequence.
it is based on a comparison between the observed and expected sums of the lengths of shortest unique substrings.
we apply the resulting index of repetitiveness, ir, to prokaryote and eukaryote genomes.
our global repetitiveness measures agree qualitatively with current knowledge about genome structure.
however, a more detailed picture emerges by subjecting the genomes to window analyses.
in the human genome the highly regulated hoxa cluster is known to lack insertion sequences.
accordingly, it is characterized by a footprint of low ir.
this suggests that in mammalian genomes regions of low ir may be due to strong selection against mutagenesis by insertion sequences.
if this is the case, scanning mammalian genomes for further intervals of low ir may reveal tracts under strong purifying selection.

 repeat sequences are a common feature of prokaryote and eukaryote genomes and in both types of organisms the selective neutrality or otherwise of extra copies of sequences has been debated for decades. since the start of the genomics era in the mid-1990s the hitherto unexpectedly large amount of repetitive sequences found in bacteria, which may account for more than 10% of the total genome, prompted a flurry of investigations of the functional and evolutionary significance of these elements. more recently, aras et al. surveyed bacterial genomes to quantify the effect repeat sequences might have on genome plasticity due to intragenomic recombination. the authors conclude that in bacteria repeats might be selected for their positive effect on the adaptability of their host. in another in silico survey of completely sequenced bacteria, achaz et al. noted that inverted repeats are underrepresented in bacterial genomes due to their destabilizing effect on genome structure. in eukaryotes the discrepancy between dna content and apparent organismic complexity had been noted even before the discovery of the double helix leading to the conclusion that "the relationship between dna and the size or number of genes is obscure". in the 1960s dna reannealing studies uncovered that eukaryotic genomes contain a highly variable fraction of repetitive dna. since the sequencing of complex genomes these observations have been made precise: approximately 50% of the human genome is made up of repetitive sequences. however, the term "repetitive sequences" encompasses a rather heterogeneous set of elements: 45% of the human genome is covered by transposons, 3% are repeats of less than a hundred base pairs, and 5% consist of recent duplications of large segments of dna. broadly similar observations have been made in other mammalian genomes. the human genome contains low, but appreciable, genetic variation caused by transposable elements, indicating that transposable elements have been active over the short time span since humans diverged from their last common ancestor. however, the decline of transposon activity in the hominoid lineage contrasts with more recent insertions in mouse, where new spontaneous mutations are times more likely to be caused by transposition than in human. the hypothesis that transposable elements are molecular parasites was originally designed to explain the apparently excessive dna baggage of eukaryotes. a number of contemporary observations support this view. transposon-derived sequences are rare close to transcription start sites and inside coding regions, suggesting that insertions are usually deleterious. moreover, the four human hox clusters and other highly regulated genomic regions contain very few transposable elements. direct deletion of megabase-sized regions devoid of known genes also seems to have no effect on mice, even though these regions contain elements that have been conserved since the emergence of mammals. there is no contradiction between these observations and the fact that occasionally transposable elements can give rise to beneficial structures including novel gene regulatory regions and the vj recombination mechanism that generates the antibody diversity expressed by vertebrate b cells. since the publication of whole genome data, the quantification and classification of repeat elements has become a major area of research in computational biology. perhaps the best-known program for the detection of repeat elements is repeatmasker, which looks for two things: tandem repeats of a few nucleotides, and homology to known repetitive elements. this approach has the advantage of dealing with elements of known origin. its disadvantage is that the presence of hitherto unknown repetitive elements might be missed. the program repeatfinder implements a highly efficient and more generic approach based on suffix trees that makes no assumptions about the type of repeat present. such methods can be used to compute, for example, the percentage of a given dna sequence covered by repeats and most methods provide a means of checking the statistical significance of the repeats returned. suffix trees allow the efficient detection of all exact repeats in a sequence. in contrast, the widely used relative simplicity factor is based on the local density of repeat motifs up to four bases long compared to their density in a shuffled version of the input sequence. application of the rsf to diverse genomes revealed that eukaryotes are characterized by an elevated "micro-repetitiveness" compared to prokaryotes. what is lacking, though, is an all-inclusive measure of repetitiveness. under the rsf repetitiveness is defined as a quantity that is minimized by shuffling the investigated sequence. as suggested by the term simplicity factor, studies of repetitiveness are related to investigations of complexity – if repetitiveness is high, complexity is low, though the converse is not always true. for example, the "linguistic complexity" of a string s is defined as the number of substrings of lengths..., |s| observed in s compared to the maximum number of substrings of these lengths. a random dna sequence with g/c content has maximal complexity and minimal repetitiveness. however, a random dna sequence with a g/c content of, say, does not have maximal complexity, while its repetitiveness should still be minimal. in this paper we propose a novel measure of repetitiveness which considers repeats of any length, takes into account g/c content, and does not necessitate shuffling for its computation. as explained in detail in the methods section, our index of repetitiveness, ir, is expected to be zero in random dna sequences of any g/c content and length, and can be computed in time proportional to sequence length. we apply the ir to sequenced bacterial genomes, archaebacteria, and six model eukaryotes: baker's yeast, nematode worm, thale cress, fruit fly, mouse, and human.  our first goal was to establish the null distribution of ir. this can be obtained by shuffling a genomic sequence. as an example we repeatedly randomized the genome of bacteriophage λ, which consists of bp of dna, and calculated the ir from these "repeatless" sequences. figure shows the resulting histogram, which is symmetrically distributed around a mean close to the expected zero. further analysis of this distribution using the shapiro-wilk test revealed that deviation from normality increased as more replicates were added. the reason for this was an increase in kurtosis, while the skewness decreased with higher replication. notice also that the ir of the unshuffled λ genome is significantly greater than its randomized version. this is not surprising, as biological sequences are no more random sequences of residues than prose is a random sequence of letters. survey of ir values we calculated ir values for completely sequenced prokaryote genomes, as well as for representative eukaryotic model organisms: baker's yeast, nematode worm, thale cress, and fruit fly. figure 2a displays the ir values of eubacteria as a function of the log genome size. in this domain of life ir was not correlated with log genome size. the average ir of eubacteria was. % of bacteria had an ir ≤ on the other hand, there were bacteria where ir > with the highest value found in methylobacillus flagellatus kt. the other members of this group were streptococcus agalactiae nem, dehalococcoides ethenogenes, francisella tularensis subsp. tularensis schu s, neisseria meningitidis mc, francisella tularensis subsp. holarctica, and escherichia coli o157:h edl. at the other extreme of the distribution, buchnera aphidicola str. bp had the smallest ir value, which was even smaller than that observed in phage λ. with one exception the ten eubacteria with the lowest ir values comprised only intracellular organisms sampled form the genera buchnera, chlamydophila, candidatus, neorickettsia, and rickettsia. the exception was the highly abundant photosynthetic bacterium prochlorococcus marinus subsp. marinus str. ccmp. in order to further investigate some of the extreme ir values observed in eubacteria, we subjected them to sliding window analyses. figure 3a shows such an analysis for m. flagellatus kt and reveals that its global ir value was caused by two large peaks of local ir indicating the presence of a very long exact repeat. this turned out to be a tandem repeat comprising an astonishing bp. removal of one copy of this duplication lead to a much deflated ir of. however, not all large ir values among eubacteria were caused by single exact repeats. figure 3b displays a sliding window analysis of the genome of onion yellows phytoplasma oy-m, which had a global ir value of. a scan of its local ir values indicated the presence of numerous regions of significant repetitiveness. the bacterium with the second highest global ir-value, strepotococcus agalactiae nem was an outlier among the other streptococci investigated, which have an average ir of. window analysis of s. agalactiae nem revealed three exact repeats of kb and their removal resulted in an ir of. similarly, escherichia coli oh157:h edl had an exceptionally high ir of compared to the other five strains of e. coli sampled. in this case window analysis of e. coli oh157:h edl highlighted a repeat region of approximately kb located at positions,000–, and,000–, which contained several long exact repeats with the longest spanning over kb. removal of one copy of the kb repeat region reduced the ir to. mouse and human chromosomes the average ir for human chromosomes was and values for individual chromosomes ranged from in chromosome to in the y chromosome. the y chromosome was the only human chromosome with ir > which agrees with the view that it has the highest dna turnover in the genome. the average ir for mouse chromosomes was, which is significantly larger than that of humans. this agrees with the observation that the rodent lineage has experienced a higher rate of retro-transposition than hominoids. individual mouse chromosomes had ir values ranging from in chromosome to in the y chromosome. as in the human genome, the y chromosome from mouse was characterized by the largest ir. in addition, chromosomes and x had ir values >. hox genes in human and d. melanogaster the hox genes encode transcription factors that function as fundamental developmental switches in all animals. in human the four clusters of hox genes contain very few insertion sequences. to assess the effect of this on the landscape of human ir values, we carried out a sliding window analysis of mb around the hoxa cluster on chromosome figure 5a displays the conspicuous footprint of low ir values that coincides with the location of the hoxa cluster. in order to make this eye-ball analysis more quantitative, we searched the fragment of chromosome displayed in figure 5a for runs of ir ≤ that extended for at least kb. this uncovered intervals ranging in size from to kb. ten of these intervals were located within the hoxa cluster. the remaining three arrows are marked by stars in figure two of the corresponding regions with low ir values intersected with scap a src family associated phosphoprotein involved in signal transduction leading to t cell activation. the last region of low ir outside of the hoxa region intersected with evx this is a homologue of the even-skipped homeobox gene originally discovered in d. melanogaster. in vertebrates it is involved in eye development. human evx is located just kb upstream from the most 5' of the hoxa genes, hoxa. a sliding window analysis of the antennapedia complex in d. melanogaster, which is homologous to part of the human hoxa cluster, revealed a very different topology of repetitiveness. on a background of ir ≈ large peaks marked the presence of long exact repeats and the antennapedia cluster was not characterized by a conspicuous change in ir values. discussion "at this point we do not know what most of the dna in eukaryotes is doing". today, thirty-five years later, the function of apparently excess dna in both eukaryotes and prokaryotes remains a topic of intense research activity. our method to quantify this excess dna, the index of repetitiveness, is close in spirit to the investigation of linguistic complexity based on suffix trees. linguistic complexity is maximized in random sequences with equiprobable residues. deviations from equiprobability lead to a reduction in complexity even if the sequence remains completely random. in contrast, in this paper we were interested in quantifying repetitiveness with respect to genome composition and to make this measure comparable across genomes. our starting point was an investigation of the complement of repeats, the unique sequences. these are trivially easy to find, for example a sequence is always unique with respect to itself, and for this reason we have concentrated on shortest unique substrings. a shortest unique substring occurs only once in its parent string and cannot be reduced in length without losing its uniqueness. a genome with many long repeats contains many excessively long shortest unique substrings, while its shuffled version contains only the shortest unique substrings expected to be there by chance alone. since we have derived the latter quantity analytically, the ir is constructed as the logarithm of the ratio between the observed and expected aggregate number of nucleotides found in shortest unique substrings. at the cost of ignoring homology relationships, this measure has the advantage that it can be computed for any double-stranded dna sequence and its expectation is always zero. it is also possible to estimate an ir value for sequences over alphabets other than the four nucleotides. in this case the quantity ae defined in equation can be estimated by shuffling the input sequence. for example, the ir of this paper is approximately. since the construction of the underlying suffix tree takes only time proportional to the length of the sequence analyzed, the ir can be computed in time proportional to the length of the input sequence. in contrast, traditional repeat analysis such as implemented in the program repeatmasker runs in time proportional to the product of the length of query and subject sequence. like most suffix tree implementations, the suffix tree on which our analysis is based, is kept entirely in the main memory of the computer. this has the advantage of being relatively easy to implement. the disadvantage of this approach is that the amount of sequence data that can be analyzed in a single run of the program is limited by the available ram rather than by the much cheaper hard disk space. we are currently studying advances in disk-based suffix tree construction in order to break through the ram barrier. it may come as a surprise that the ir values for human and mouse chromosomes were within the range of ir values observed for less complex eubacterial genomes. however, this does not contradict the well-known fact that mammalian genomes are full of interspersed repeats, while bacteria usually contain fewer of these elements. the apparent paradox is due to the fact that the effect of interspersed repeats on the excess amount of exact repeats in a given genome – which is what the ir measures – depends not only on the fraction of sequence covered by repetitive elements; equally important is the number of mutations accumulated since the divergence of an interspersed repeat from its most recent ancestor. as a result of the mutation process, ancient repetitive elements may not contain longer motifs repeated elsewhere than the rest of the genome. the presence of such elements would leave the ir unchanged compared to the identical genome without them. a similar argument applies to the interpretation of the high ir values found in the y chromosomes of human and mouse. the two factors determining the accumulation of sequence polymorphisms, time to the most recent common ancestor and mutation rate, cannot be separated. in addition, the effective mutation rate differs between autosomes and the y chromosome. under neutrality the number of snps expected for a pair of homologous sequences is θ = 4neμ, where ne is the effective population size and μ the rate of mutation. since the effective population size of mammalian y chromosomes is only one quarter that of autosomes, repeat pairs on the y chromosome are broken up more slowly by mutations than elsewhere in the genome contributing to higher ir values. it should be noted at this point that neither the mouse nor the human genome are completely sequenced to date. if new sequence data comes predominantly from regions that are difficult to sequence due to their repetitiveness, future editions of the human and mouse genomes are expected to have higher ir. the ir values found in our whole genome analyses correlate well with the relative simplicity factors reported previously. this correlation is not perfect due to the fact that the rsf measures the local excess of short repeats, while the ir measures the excess of all repeats throughout the sequence. moreover, no significant correlation between archaebacterial genome size and rsf was observed by hancock, in contrast to our finding. this effect, however, is simply due to differences in sampling; if we reduce our sample of archaebacterial genomes to the nine investigated by hancock, the correlation between ir and log genome size also vanishes. in contrast, a tenfold increase in the number of bacterial genomes investigated between hancock's and our study only confirmed the earlier diagnosis of no correlation between rsf and genome size. the average ir for eubacteria was. however, it is clear that there are a few extreme ir values that inflate this average. the largest ir for bacteria was found in methylobacillus flagellatus kt. this value was the most extreme of a set of seven organisms with ir > that also included the human pathogens neisseria meningitidis mc and escherichia coli o157:h edl. in a previous survey of bacteria, neisseria meningitidis was already singled out as having a highly repetitive genome. the low ir values found by us among obligately host-associated bacteria also agree with a known lack of repeats in these genomes. while other bacteria appear to harbor repeats to increase genome plasticity, we speculate that intracellular symbionts and pathogens are less dependent on genome shuffling for their survival as they live in more stable environments. our sliding window analyses revealed that the computation of ir values for entire genomes averages out sharp regional fluctuations in ir. in bacteria a high ir value may be caused by a few extreme duplications, as was the case for m. flagellatus kt and s. agalactiae nem in the human genome the genes making up the hoxa cluster were characterized by a kb footprint of low ir values. the fact that additional runs of low ir outside the hoxa cluster also coincided with known genes leads us to currently search the entire human genome for further regions of low ir.  investigations of repetitiveness are traditionally carried out using some form of alignment algorithm. such algorithms tend to run in time proportional to the product of the length of the query and subject sequence. in this paper we present an approach that runs in time linear in the length of the input sequence. it is based on a comparison between the observed and expected sums of the lengths of shortest unique substrings. we apply the resulting index of repetitiveness, ir, to prokaryote and eukaryote genomes. our global repetitiveness measures agree qualitatively with current knowledge about genome structure. however, a more detailed picture emerges by subjecting the genomes to window analyses. in the human genome the highly regulated hoxa cluster is known to lack insertion sequences. accordingly, it is characterized by a footprint of low ir. this suggests that in mammalian genomes regions of low ir may be due to strong selection against mutagenesis by insertion sequences. if this is the case, scanning mammalian genomes for further intervals of low ir may reveal tracts under strong purifying selection. 
here we propose a novel measure of genome repetitiveness, the index of repetitiveness, ir, which can be computed in time proportional to the length of the sequences analyzed. mouse chromosomes have a significantly higher ir than human chromosomes and within each genome the y chromosome is most repetitive. a sliding window analysis reveals that the human hoxa cluster and two surrounding genes are characterized by local minima in ir. a sliding window analysis helps to analyze the intragenomic distribution of repeats. this reveals a broad spectrum of repetitiveness among diverse genomes which agrees qualitatively with previous studies of repeat content. the expected value of ir is zero for random sequences of any g/c content and greater than zero for sequences with excess repeats.
 genome sequences vary strongly in their repetitiveness and the causes for this are still debated. here we propose a novel measure of genome repetitiveness, the index of repetitiveness, ir, which can be computed in time proportional to the length of the sequences analyzed. we apply it to genomes from all three domains of life.  the expected value of ir is zero for random sequences of any g/c content and greater than zero for sequences with excess repeats. we find that the ir of archaea is significantly smaller than that of eubacteria, which in turn is smaller than that of eukaryotes. mouse chromosomes have a significantly higher ir than human chromosomes and within each genome the y chromosome is most repetitive. a sliding window analysis reveals that the human hoxa cluster and two surrounding genes are characterized by local minima in ir. a program for calculating the ir is freely available at.  the general measure of dna repetitiveness proposed in this paper can be efficiently computed on a genomic scale. this reveals a broad spectrum of repetitiveness among diverse genomes which agrees qualitatively with previous studies of repeat content. a sliding window analysis helps to analyze the intragenomic distribution of repeats. 