From: mathog@seqvax.caltech.edu (David Mathog)
Newsgroups: bionet.molbio.genbank
Subject: Re: Apologies for Disappearing Division
Date: 18 Dec 1992 12:08 PST
Organization: Division of Biolgy, CALTECH
Distribution: world
NNTP-Posting-Host: seqvax.bio.caltech.edu
News-Software: VAX/VMS VNEWS 1.41    

In article <BzGILx.41C@usenet.ucs.indiana.edu>, gilbertd@sunflower.bio.indiana.edu (Don Gilbert) writes...
> 
>It is asking too much to produce a file for each possible category of interest.
>What we need is software to search the data properly.  It shouldn't
>matter whether the data sits in one or hundreds of files.  The current
>NCBI e-mail "retrieve" service now *does* allow you to search for keywords
>in a specific field.  So, network band-width permitting, you can re-build
>your own organelle file if you want by searching ORGANISM for Mitochondrion, 
>Chloroplast and other organelles if they exist.  But what we should have
>(I don't know if we do) is other software like BLAST, etc. to do analyses
>and to limit those analyses to just some subset of genbank based on
>field selections.
> 

Exactly!

Not only is the current design of the flat file distribution arbitrary (not
illogical, just arbitrary), but there seems to be no good reason to 
maintain it into a future replete with multiple fully sequenced organisms. 

Many of you may have noticed that as the entries get bigger and bigger the
existing tools become less and less useful.  For instance, what does one do
with a BLAST hit on the Yeast Chromosome III entry? There are lots of ways
to get the entry from one of the Genbank servers (NCBI, IUBIO...) to your
directory, but all of the ones that I'm familiar with insist on sending the
whole thing, even though you generally only want a small piece of it.
Obviously, we're going to have to find a better way long before we start
getting BLAST hits on the "human X chromosome, complete sequence" entry! 

As a start, I'd really like to see NCBI RETRIEVE implement the following
(never mind the syntax). 

  SEND <entry number> [ FROM <START> TO <END> ]

For the sequence this is a trivial operation.  Chopping out the appropriate
parts of the reference section *should* be possible too, if maybe a bit 
more challenging.

Probably in the next few years we will have to formally set up "central"
servers for a lot of the searching/retrieving that is currently done
locally.   While the growth of sequence information is exponential, our
disk drive budgets are not :-(.  

It would be really useful to have (at least) a SQL like access to these
database servers. That way we could specify exactly the set to be searched
or retrieved. I'll plead guilty to having not read the Delila paper, but
from what I've seen in these news groups it looks like Delila could be
interpreted into SQL, and that the existing Genbank schema would only need
to be minimally modified with the addition of a couple of views or extra
tables. 

In addition, lets see if we can agree on a SINGLE syntax for accessing 
these databases.  Maybe that way we can get by with a single client (per
platform) to get at Genbank, NBRF, EMBL etc. 

Now Santa knows what I'd like under my tree.

David Mathog
mathog@seqvax.caltech.edu
Manager, sequence analysis facility, biology division, Caltech