From: lamoran@gpu.utcs.utoronto.ca (L.A. Moran)
Subject: Author errors and omissions.
Organization: UTCS Public Access
Distribution: bionet
Date: Tue, 14 Apr 1992 04:27:53 GMT


Keith Robison suggests several problems with sequence entries,

       "1)  Information not being entered correctly the first time.

        This includes all errors initially entered, including typos,
        errors in delineation of features, errors in assigning sequences
        to GenBank divisions.

        Most errors of this type CAN be caught automatically. Spell-
        checking all entries (using a standard dictionary augmented with
        biological terms) will eliminate typos and make keyword searching
        reliable (currently only the ORGANISM field has any sort of 
        restricted vocabulary). Most feature span delineations can be 
        checked -- CDS's can be checked to see if they translate, functional 
        sites can checked against their consensus, etc. Note that I am 
        NOT suggesting that sites be _assigned_ automatically, just that 
        they be _verified_ automatically. I know from experience that the 
        majority of non-consensus mRNA splice sites represent discrepancies
        between GenBank and the literature, but some are really non-consensus 
        sites."

It is extremely difficult to detect errors such as sequencing errors and 
cloning artifacts but it is also surprisingly difficult to detect more 
careless errors such as typos (in the sequence itself). I have been looking 
carefully at the family of HSP70 genes with a view to detecting such errors. 
Since these are the most highly conserved genes in biology (possibly) it 
should be feasible to estimate the frequency of such errors if the sequence 
does not correspond to what would be predicted from an evolutionary history.

Out of 120 GenBank entries I have identified about a dozen where there are
mistakes that are due to careless recording of the data. In some cases parts
of the sequence are deleted or transposed in GenBank relative to the
published entry. (In one case the published DNA sequence is also wrong but 
the predicted amino acid sequence is correct!) In most of the cases there
is a difference of only a few nucleotides between the published sequence and 
the GenBank entry.

Most of these genes do not have introns but in several cases the intron/exon
junctions are clearly wrong. I agree with Keith that this is a high error
rate. The proper recording of intron/exon boundaries is tedious and the 
authors probably do not check their data very closely. As for other "features"
they usually don't exist in these HSP70 sequences because the authors can't 
be bothered to enter them when they submit their data. (I can understand why 
one would not want to enter useless information such as ribosome binding 
sites! (-: ) In some cases the "features" are imaginary or incorrect which 
suggests that authors are not always reliable even when annotating their own
sequence.

All but two open reading frames that I have checked are actually open reading
frames - this is now routinely verified on submission. However, there are 
about ten examples where the predicted amino acid sequence cannot be correct
in some small regions of the sequence. By shifting the reading frame by a 
single nucleotide the "correct" amino acid sequence is restored over this
stretch and then shifted back to the original reading frame. This suggests
strongly that the sequence contains a pair of insertion/deletions. I assume
that this is due to sequencing error but it could be a typo.

Keith Robison also lists a third point (I deleted number two);

       "3) Information not being upgraded

        A serious flaw with the current databases is that the information
        already there is rarely checked. There's no consistent level of 
        annotation quality (completeness, accuracy, parsability) -- entries 
        for 1984 are at an entirely different level than entries from 1992.
        The only way to tackle this is to have programs constantly analyzing
        the databanks, looking for under-specified features (such as the
        misc_feature -- misc_binding above), incorrectly specified features, 
        unmarked features, etc. These programs should also be keeping close
        tabs on those /note qualifiers Tom hates -- if a particular word 
        appears frequently in them for a particular class of qualifier, 
        than it may well be time to create a new qualifier. And, such
        programs could constantly be on the lookout for possible merges 
        and duplications, and report them to the human curators for further
        action."

Unfortunately it is not so easy for a computer to update old entries so the 
task falls to someone who is knowledgeable and willing to spend enough time 
to do the job correctly. I hope to go back and annotate the old HSP70 entries 
and make a lot of corrections to more recent ones although it is still not 
clear whether a third party (such as myself) has the right to alter other 
people's entries. There are several examples of old HSP70 sequences that are 
known to be incorrect because the authors have published corrections that have
not been entered in the database. In a couple of cases an entirely new, and 
correct, entry has been created for an old incorrect sequence but the old 
sequence remains in the database. Should it be deleted?

Merges are a tricky problem. If two groups sequence the same gene should
there be only one entry? What if the two sequences are not identical? If
two sequences overlap should they be merged? Conversely, is anyone else
bothered by the policy of having separate entries for each exon just because
the entire intron was not sequenced? What should a curator do about these
problems? We need to have a firm policy.

Can anyone think of a way to encourage authors to look after their data?

Keith continues,

       "If such measures are not implemented soon, the resultant mess 
        will be staggering. Not only is sequence acquisition growing 
        phenomenally, but the expectations of the database-user community
        are increasing as well.

        When people were just FASTAing their new sequence to see what it 
        looked like, the reliance on annotation quality was low. But now, 
        people want to be able to ask questions such as "what is the 
        information content of binding sites for X", or "how many exon 
        types are there" or "what is the rRNA tree for all eukaryotes" or
        "how do I build a bacterium", and it's appalling that we can't 
        just pull the information from GenBank and TRUST it.  

        Too much time is spent by we end users doing the quality checks 
        (and corrections) which should be performed by the keepers of the
        databases."

It is certainly true that you can't just retrieve the information from
GenBank and trust it. You should retain a healthy degree of skepticism 
concerning this data just as you would about most data which is published.
This is particularly true when constructing evolutionary trees because
the errors in some sequences will affect the topology.

I agree that SOME quality checks and corrections are the responsibility
of the keepers of the databases and for the most part these checks are
being performed currently. However, the most serious difficulties require
the intervention of a curator, and this includes annotation that has not been
added by the original authors. There aren't many curators who can devote this 
kind of effort to a task for which there is little reward - is anyone willing
to pay for this (or count it as several publications)?

I have no idea how to solve this problem. It seems unreasonable to expect
every author to fully annotate their sequences since in some cases they
don't know very much about the gene they have cloned. For example, recently
there were several thousand sequences added from a single laboratory. Most
of these entries consisted of a few hundred base pairs of a cDNA clone.
Two of the sequences were HSP70 genes but there were no annotations and it 
is unlikely that the authors had time to learn enough about these genes to 
properly annotate them. (Incidently, at least one of these sequences has 
errors - does that invalidate the patent?)

Laurence A. Moran (Larry)
Dept. of Biochemistry               lamoran@gpu.utcs.utoronto.ca
Medical Sciences Blgd.                    (416) 978-2704
University of Toronto               FAX   (416) 978-8548
Toronto M5S 1A8, CANADA