Reliability and use of published sequence data (2003)

Holst-Jensen et al. (2004) and Hawksworth (2004) have identified particularly important issues in relation to the deposit, validation and use of published DNA sequence data, in response to our recent article on the reliability of such information (Bridge et al., 2003; Vilgalys, 2003). We are grateful for these, and welcome the opportunity to provide some additional comment. With regard to the letter from Holst-Jensen et al. we would agree that repeated keyword searching is an inefficient means of obtaining information, particularly because of the apparent lack of standardization of descriptive terms for sequences in the databases. We are not surprised that different workers obtain different numbers of sequences, and Eriksson lists 129 accession numbers for SSU sequences from the order Helotiales in Myconet (http://www.umu.se/myconet/SSU2003.html#leot). Our searches were largely consecutive and usually included Boolean combinations of taxonomic and sequence related terms, such as ‘Helotiales’ and ‘18S’. However we found it necessary to perform several different searches in order to allow for the terms ‘18S’, ‘small subunit’ and ‘SSU’ in order to obtain all of the sequences. Similarly, some of the searches included terms for higher taxonomic ranks such as ‘Leotiomycetes’ in order to identify sequences from taxa where the taxonomy was uncertain when the sequence was deposited. Some fungal SSU sequences are described in the databases as ‘17S’ or ‘16S-like’ (also referred to by Eriksson), and searches including these terms may also be necessary for some fungal groups. We agree that the methodology that we used was very basic. This was a deliberate decision as we wanted to try and simulate a situation whereby sequences were screened against the data in an ‘as is’ state, as may occur during large-scale molecular diversity or ecology exercises. One of our primary aims was to determine whether simple screening procedures could be employed and a major consideration was given to determining ‘to what extent these sequences can be used as reliable reference sources without prior checking of the sequence, and in some cases the associated literature.’ As stated in our paper, we believe that the sequence databases can be used reliably but that ‘a degree of comparison, and in some cases recourse to other literature’ will be required. The use of the databases as stand-alone resources within wider studies may therefore lead to errors. While we would agree that examination of all original literature and material may be necessary before an exact identification can be made, this is often not practical for nonspecialists. In retrospect it might have been more helpful if we had cited all of the literature that was consulted in arriving at our conclusions. However, on most occasions we consulted the original literature to verify data, and in our opinion this did not require citation in the text. As the sequence data was obtained directly from the databases, we limited the literature cited to that which appeared immediately necessary for the discussion. We were however, careful to check all database fields and references before including sequences, and as a result we did not include a number of sequences where there were likely duplications of sequences from the same material, or where there were obvious corrections or ambiguities in the entries or taxonomy. This included one sequence that was described as being from the Helotiales, but was deposited with a Graphium name, with an additional note that it should be excluded from Graphium. These checks reduced our data sets from the total numbers of sequences available. As the primary object of our exercise was to test the data as deposited, we were initially uncertain as to the most suitable method for dealing with introns in the sequences. We were aware of the reported intron in sequences AJ266069 and 266070. Repeated BLAST and FASTA searches with these and associated sequences gave alignable matches to various fungal SSU sequences, some of which were reported to contain between one and three introns, and some of which were not reported to contain introns. Again we found considerable variation in the reporting of the presence and type of introns in deposited sequences. In some cases their type, presence and position were clearly indicated, in others their presence was indicated, but their position or type was not, and the alignments we obtained suggested that small introns may also be present in some sequences where they have not been reported. In view of this we made the decision to use the data as deposited, rather than to edit and risk introducing errors into what would then have been derived data. We debated the validity of including the eight sequences in question, and made a decision that because of their alignments with other sequences and their inclusion in the Myconet SSU list, there was a possibility of them being used in other studies. We were however, conscious that we could be biasing our data set and we took the decision to include two of the eight sequences. In retrospect we should have clarified our intention and initial data criteria in the text. We agree that the identification of three of the Phoma sequences to Paraphaeosphaeria species may well not constitute a misidentification at the generic level. These three sequences were all named as species for which other sequences were available, and in the matches we undertook they did not match closely to these. Our suggestion is that these are misidentified at species level, and possibly not at genus level. The only other sequences from Phoma that we considered to be misidentified are the three referred to in the text, and we specifically stated that the identification of Phoma epicoccina with Epicoccum nigrum was in agreement with the current taxonomy. We highlighted the sequence for P. herbarum in Fig. 2, as when we undertook the study there were only two sequences available for this species – this one, and a short 230 bp ITS1 sequence. There was only around 70% homology between these sequences in the ITS1 region, and our opinion was that the two sequences did not represent the same species. Since we prepared our data set four more sequences labelled as P. herbarum have been deposited. Comparison of all six sequences gives between 70 and 95% homologies in the ITS1 regions. The teleomorphs given for the deposited P. herbarum sequences include Didymella and Westerdykella, and our opinion remains that sequences deposited as P. herbarum probably represent at least two different taxa, and that without detailed study of the original materials it is not immediately clear which sequence(s) may be correct for that name. It was not our intention in this work to ‘impugn mycologists’. As we stated in our introduction, a significant number of possibly chimaeric sequences have been reported in the bacterial database, and our own experience with sequence databases for other organisms suggests that the observations we have reported will probably be relevant for all organisms. Contamination and other issues have been discussed in relation to human genome databases (Taylor, 2001) and there have also been estimations made of chimaeric sequences and other contaminations in expressed sequence tag libraries (Sorek & Safer, 2003). The comments from David Hawksworth regarding previous reporting of erroneous sequence data are correct, and we would not suggest that our paper is the first example of this. As far as we are aware such reports for microbial sequence data have generally not included figures for the numbers of sequences that were excluded from studies, with the exception of some detailed discussions of individual cases. There are however, unquantified statements in the literature including the ‘many’ referred to by Hawksworth, and while a search of the literature can demonstrate that there may be a problem with some sequence information, in our opinion there is not sufficient detail to quantify this without examining the sequence data directly. We would agree that the figure of 20% that we determined may be very inaccurate. Our sample size was relatively small, and was limited in terms of both the groups studied and the sequences considered. In our opinion these proportions may vary considerably depending on the DNA region and the group studied. We agree that some current apparent misidentifications may merely reflect the knowledge of the systematics of the organism at the time, and as we detailed in the article, this can make interpretation of sequence data very difficult, particularly with a largely incomplete data set. Both Holst-Jensen et al. and Hawksworth have suggested flaws in our methodology, and while we accept that our study is far from perfect, it is difficult to design realistic quantitative tests. Discrepancies in sequence descriptions, together with changes in taxonomic opinion make key word searching at best difficult, and other approaches such as by ‘author’ or ‘publication’ rely on having information a priori. Subsequent publication or amendment of both sequences and names also complicates the situation if database entries are not updated. Holst-Jensen et al. make some important points regarding validation and subsequent updating of sequence information, and we are certainly aware of cases where sequences listed as unpublished have subsequently been included in publications. In our experience this has resulted in some database entries being updated, but not others. As far as we are aware the current system places the responsibility for this on the depositor, but there may be instances where this is not practical. Most herbaria have systems that allow re-determinations and notes to be included from scientists other than the depositor, and it may be that a facility for this could be useful for sequence data, particularly in cases where the original depositor has retired, or is otherwise no longer involved in science. There has been a significant growth of secondary and tertiary data sources for sequence data, and there are also implications for these data sources if sequences are updated or subsequently modified in any way. A secondary source that contains links to the primary database will not necessarily be affected, but any secondary source that maintains copies of the original data will also require modification. In our opinion the discussions from our article appear to suggest the common message that the sequence databases can be used as an important resource, but that their utilisation may often require a significant background knowledge of the organisms concerned and the methodologies used, and that this may also need to be combined with information from original and subsequent publications.