Integration of Data Management and Analysis for Genome Research

Technological advances in genome research have produced unprecedented volumes of genetic and molecular data that now provide the context for any biological research. However, data access, curation, and analysis have remained challenging areas for continued research and development and often prove to be the bottleneck for scientific progress. Many a paper in bioinformatics or even in general molecular biology these days start out just like the abstract above, with an acknowledgement of the explosive growth of molecular sequence, structure, and expression data. What fit nicely within the printed pages of a thin booklet only 25 years ago now comprises large and increasingly complex databases that are Web-accessible to the public. Figure 1 shows the growth on one major molecular sequence repository GenBank, maintained at the U.S. National Center of Biotechnology Information (NCBI). The slope of the curve is indeed impressive. However, the actual size of the data sets would seem to be easily dwarfed by database sizes in other commericial, governmental, or even other research fields. What then is the real problem, if any, facing the biology community? I think there are many aspects for consideration. One important facet is that the molecular databases themselves have evolved over the years, and surely many details of database design should have been done differently in hindsight. However, the rapid pace of new data acquisition has so far prevented any major re-design and re-construction of the databases the community is accustomed to. Another critical point is that the data derive from a large variety of sources and are intrinsically heterogenous. There are no uniform standards for data quality and annotation. In these notes I shall not further discuss the challenges faced by the large database providers, but rather I shall review the problem first from the point of a user and then suggest some approaches we have pursued to provide intermediate solutions. Figure 1: Molecular Database Growth (from www.ncbi.nlm.nih.gov/Genbank/genbankstats.html). 1 Some Comments on the Current Status of Molecular Databases 1.1 Existing Data Retrieval Systems Two of the centralized, prominent Web access points to existing sequence and molecular biology data are the Entrez system at NCBI (www.ncbi.nlm.nih.gov/Entrez/) and the SRS system developed in Europe (www.embl-heidelberg.de:80/srs5/). Both systems are limited in three ways: 1) data are provided in flat file formats only, often requiring much user manipulation after receipt; 2) retrievals are inflexible, and do not support items such as ranges and numerical operators (e.g., , , etc.), and 3) the Web and email server user interfaces are non-intuitive and force users to learn specific query syntaxes in order to use them effectively. For example, with current query protocols it is not possible to retrieve ”all GenBank genomic DNA entries with a single CDS feature annotation”; or the subset of the previous query restricting the retrieved entries to ”CDS with at least two annotated coding exons”; or ”the DNA sequence segments in FASTA format for 500 nucleotides immediately upstream of the annotated ATG start codon”; or protein records including their cognate cDNA sequence (see Section 1.3 for further examples). 1.2 Annotation Errors in Public Databases Publicly available biological databases should be distinguished as either data repositories or curated databases. These two categories serve very different needs. Data repositories, including most prominently GenBank, provide centralized access to original data. Curated databases, on the other hand, provide annotated data according to the curators’ discretion. In practice, of course, these distinctions tend to be blurred, and most databases serve a mixture of original and annotated data, or data with rather limited annotation. The data repositories are essential when a researcher needs to locate particular data from its original source. For example, a search of GenBank might quickly turn up a particular protein sequence translated from its corresponding genomic and cDNA sequence determined by author A in year X. Subsequently, errors in the interpretation of the data may be discovered, or author B may publish a corrected sequence independently in year Y. In this case, the data repository would still retain author A’s entry, but in a curated database a single entry should represent the combined interpreted data (one correct sequence, or annotation of allelic variation). Brenner [Bre99] estimated the error rate of functional assignment in gene annotation to be at least 8%. To illustrate these problems, we followed up on a number of erroneous GenBank entries pointed out by Korning et al. [KHRB96]. These authors built a training set for a neural network algorithm to predict splice sites in Arabidopsis genes and encountered ”an alarmingly high error rate” in the requisite GenBank annotation. While some errors were due to typographical misprints that could be corrected by comparison with the original papers, many errors were found to be systematic shift errors created by wrong assignments of splice sites (see below). Table 1 of the Korning et al. paper listed 24 specific GenBank entries with erroneous splice site annotations. Our present re-checking of these files revealed that only five of these entries have been corrected in the past six years. For others, the mostly obvious annotation errors remain. Figure 2 gives one example. For other files, splice sites are erroneously assigned when the and sequences are identically AGGT (the correct assignment is agGT ... AGgt, intron residues in capitals; based on cognate cDNA matching alone, there are four alternative assignments matching up the aggt of the cDNA). It is evident that erroneous data place a heavy burden on individual researchers who have to devote a large effort to clean up the errors before the data can be successfully used in their research. Even more disturbing is the persistence and propagation of errors in the LOCUS ATKIN2 880 bp DNA PLN 23-JUL-1992 CDS join(104..160,320..390,504..579) EST Accession 3450035: Exon 1 78 160 ( 83 n); cDNA 1 80 ( 80 n); score: 0.867 Intron 1 161 321 ( 161 n); Pd: 0.976 (s: 0.90), Pa: 0.972 (s: 1.00) Exon 2 322 390 ( 69 n); cDNA 81 149 ( 69 n); score: 0.971 Intron 2 391 504 ( 114 n); Pd: 0.999 (s: 0.96), Pa: 0.964 (s: 0.98) Exon 3 505 785 ( 281 n); cDNA 150 429 ( 280 n); score: 0.996 Alignment (genomic DNA sequence = upper lines): TTTTACAAGA AAAAAATATC TGAAAAATGT CAGAGACCAA CAAGAATGCC TTCCAAGCCG 137 |||| |||| ||| ||| || |||||||| | | | |||| | | |||||||| ||||||||| TTTTCCAAGG AAA-AATTTC TGAAAAAT-T CNGGGACC-A CNAGAATGCC TTCCAAGCCC 57 GTCAGGCCGC TGGCAAAGCT GAGGTACTCT TTCTCTCTTA GAACAGAGTA CTGATAGATT 197 |||||||||| |||| ||||| ||| GTCAGGCCGC TGGCCAAGCT GAG....... 80