From: lamoran@gpu.utcs.utoronto.ca (L.A. Moran)
Newsgroups: bionet.molbio.genbank
Subject: Author errors and omissions.
Date: 15 Apr 92 15:29:11 GMT
Distribution: bionet
Organization: UTCS Public Access


In an earlier posting I (Larry Moran) said

      "Unfortunately it is not so easy for a computer to update old 
       entries so the task falls to someone who is knowledgeable and 
       willing to spend enough time to do the job correctly."
 
Keith Robison replied,

      "I disagree entirely. It is all too EASY to update them 
       automatically -- you just have to unconvince yourself that it 
       is difficult."

Keith, I just spent several hours meditating and trying to unconvince myself.
Do you know what I discovered when I returned to my computer? It still 
requires someone knowledgeable to do the job! Maybe I didn't I didn't work
hard enough at "unconvincing" myself! (-:

Keith goes on to say,

      "I do not advocate a program which would make changes unguided,
       only one which would find possible changes and make it as easy 
       as possible for a human curator to fix them (preferably <one 
       keystroke/change). The program could prepare a list of proposed 
       corrections, the curator would confirm them, and then tell the 
       program to implement them."

It seems to me that this still requires someone "...knowledgeable who is 
willing to spend the time to do the job correctly." Keith, I'm sure that
you are thinking about relatively trivial changes such as correcting 
spelling mistakes and incorrect reading frames. I agree that these kinds of
errors can be handled with minimal (but significant) intervention on the part
of a human. However, I'm thinking about more important updates such as adding
annotations and features. These require more than a single keystroke, in fact
they require a lot of work by a curator.

In defense of computers Keith says,

      "If you can state a rule, then you can write a program to test 
       whether any entries break that rule. Most of the rule-breakers 
       will be errors, and those which aren't are likely to hold some 
       interesting biology."

What an interesting proposition. Here are a couple of "rules" that we can look
at. I'm certain that we will all agree that these rules define information
that we would like to see added to the database. I maintain that computers
can't do this.

       A GAL4 binding site is defined as a piece of DNA to which
       GAL4 has been shown to bind by direct experiment.

       A transcription start site is the nucleotide(s) that represent 
       the 5' end of transcribed RNA (precursor).

       Signal peptides are amino acid sequences at the N-terminus of a
       polypeptide that are cleaved when the protein reaches the ER.

       The map position of a gene corresponds to the chromosome which
       contains the gene and its genetic distance from other genes.

       The tissue or cell type or time of development where a gene is 
       expressed is determined by measuring whether mRNA is present.

       The function of a gene is related to the enzymatic activity of its
       protein product.

       The intracellular location of an RNA or protein is an important clue
       to the function of the gene product and the regulation of the gene.

       A correction is a change in the nucleotide sequence as a result of
       further analysis which reveals that the original data was wrong.

       A "reference" refers to a published paper that contains information
       relevent to the sequence in the entry.

       A LOCUS name or the name of a gene is a term that is used to uniquely
       identify the sequence entry.

It seems to me that it would be difficult to write a computer program that
could examine DNA sequences to see if these "rules" were followed because the
information cannot be deduced from the sequence alone! Even in the case of 
interesting DNA sequences such as similarity to GAL4 consensus binding sites 
it is difficult to formulate the "rule" in a meaningful manner that would 
eliminate considerable human intervention. In this case the search would 
presumably be limited to yeast sequences and furthermore to regions upstream 
of a known promoter. Would that suffice, or are there still too many "hits" 
that are quite irrelevent? And what do we do when the consensus changes? 
(This has happened with heat shock elements and others.)

Keith, to reiterate, I understand that there are several trivial corrections 
that can be done by computer but I was thinking of more substantial 
corrections and updates. I also understand that there are several searches,
based on our current knowledge of DNA sequences, that can be carried out by
computer. On the other hand, there is much hard information that needs to be 
added to the database in order to generate new correlations and comparisons.
I think that it is more important to add real experimental data, such as
transcription start sites, translation start sites, known binding regions, 
information on gene regulation, properites of the gene product, corrections,
etc. All of these additions involve appropriate references.

Also, we should be aware of the fact that there are many predicted amino acid 
sequences (ORFs) for which there is no experimental support. If we know that 
a protein, or RNA, is actually encoded then this should be part of the entry 
otherwise a prediction is just a prediction. (A large number of predicted 
amino acid sequences are likely to be incorrect, especially if the gene is 
unique. There is not much point in analyzing such predictions for secondary
structure motifs and amino acid consensus sequences and there is a great
danger in constructing a new dataset, or consensus on the assumption that
all of the entries are correct.)
  
When I raised a question about who should be allowed to alter sequence entries
in the database, Keith responded,

      "So long as the change is documented, ANYONE should be able to make 
       corrections. Add a REFERENCE L.A. Moran (Unpublished) field and
       tack on /reference qualifiers to everything you add."

I would like to believe that this is a possible solution. Unfortunately we
can't have everyone altering the database simultaneously. Nor can we risk
having extensive debates within a sequence entry if some workers disagree
with the annotations. The best solution, IMHO, is that the curators moderate
the changes. This means that we will have to have some confidence in the
curators. 

When I pointed out that the database was not as accurate as most people
believe, Keith said,

      "Of course you should retain a healthy degree of skepticism.
       But preparing a dataset from GenBank should be something you do 
       over coffee, not weeks. Wouldn't you be upset if 10% of the
       published phylogenetic trees were drawn incorrectly?"

But AT LEAST 10% of phylogenetic trees ARE incorrect ... haven't you been
reading about the "Eve/Out of Africa" controversy? Do you imagine that
all phylogenetic trees are completely accurate?

I have trouble understanding how one can prepare a meaningful dataset from 
GenBank over coffee. If all you mean is to retrieve a bunch of similar
sequences then I suppose that this is possible (depending on how long your
coffee break is!). However, once you have these sequences the real work 
begins when you start to analyze the dataset. For example, aligning 
sequences for phylogenetic comparison requires an extraordinary amount of
work that cannot be done by computer (yet). Keith, can you give me some 
examples of datasets that you would like to prepare over coffee but cannot
because of limitations in the sequence entries? How would you fix this?

Keith says,

      "Even though they have been suggested frequently, automatic checks 
       for keyword spelling, vector sequence contamination, repetitive 
       elements, consensus site agreement, etc. are not performed routinely.
       As a result, the community spends extra hours."

As Paul Gilna has already pointed out the managers of GenBank (and EMBL?)
already check for spelling mistakes and vector sequence contamination. I'm
not sure that it is their responsiblity to look for repetitive sequence
elements - that should be up to the author. When the sequence is published
the reviewers should be concerned about issues such as this and they should
insist upon checking if there is a potential problem. Besides, new repetitive
sequences are discovered every year and it is not possible to keep up with the 
literature if this is not your field (ie. authors can't be faulted for not
going back and annotating new repetitive elements).

How would you implement scans for consensus site agreement? Do you want to
search every eukaryotic sequence for CCAAT and annotate every hit? What
consensus sequences do you think that the database managers should look for?
Can you give specific examples that can be automated? Should the database
be re-analyzed every time a new "consensus" sequence is proposed? Why can't
you just look for these consensus sequences yourself if that is what interests
you?

Laurence A. Moran (Larry)                        lamoran@utcs.utoronto.ca
Dept. of Biochemistry                            (416) 978-2704 (voice)
Medical Sciences Building                        (416) 978-8548 (FAX)  
University of Toronto, Toronto M5S 1A8, CANADA   (416) 978-6673 (computer)   
Newsgroups: bionet.molbio.genbank