From: jasper@vax.path.ox.ac.uk
Newsgroups: bit.listserv.info-gcg
Subject: Re: database access
Date: 15 Sep 92 16:32:06 GMT
Organization: Oxford University Molecular Biology Data Centre
Nntp-Posting-User: jasper

In article <009609B3B5E3CE80.21805C86@immunex.wa.com>, farrah@IMMUNEX.WA.COM writes:
> 
>     This a question for GCG:
> 
>     Are you developing any more sophisicated tools for sequence database
>     access than StringSearch and FindPatterns?  There is so much parseable
>     information in the headers of GenBank, EMBL/Swiss-Prot, and PIR
>     sequence entries that it is a real shame to only be able to access it
>     using simple string-matching.  This is my single biggest frustration
>     with the GCG package.

Seconded.

> 
>     And now a question for GCG users:
> 
>     What tools are you using for this purpose, other than StringSearch?
>     Are there any publicly available programs that run on the VAX?  Are any
>     of them GCG-compatible - i.e., they make use of GCG-formatted sequence
>     files?  I saw a posting recently for Etzold's SRS (Sequence Retrieval
>     System) - any others?

Well there are lots of options, but to cover some of those we are looking at, 
and would encourage others to explore too:

We have just started with NBRF/PIR's ATLAS (aka XQS) which is in the style of
PSQ/NAQ, but has everything indexed. It is fast! If you use it straight from
the CD then no problem: otherwise the biggest potential problem is going to be
over maintaining the Embl and Genbank updates for this, specifically because
the TERM file is a single file encompassing the entire database collection,
resulting in the requirement to rebuild completely every time a database is
updated. If the TERM files were set up at one per section then updating would
be fine and the various term files would have to be linked by software. Or if
the term files on the CD could be linked to other databases on hard disk this
would be even better.  I'm not speaking for anyone at NBRF on this, but I
understand that this change is planned for the future. There are clients for
ALTAS are VMS and DOS at present, unix adn one hopes Macin the future.  To use
GCG and Atlas together (from databases on the hard disk rather than CD) we
seem to need to have the databases reformatted by the PIR utilities rather
than the GCG ones, but this is really just a question of the format of the ref
files, GCG removes the line starting XX from EMBL and PIR leaves them, this
difference seems to be the cause of the problems. Otherwise the indexing files
do not conflict. 

Another option is the NCBI Entrez CD, for which there are clients for unix
(all flavors, Sun and SGI clients are on the NCBI server), VMS (Motif 1.0A is 
required), Mac, and Windows (not just DOS). The software on the server 
compiled cleanly on VMS, Sun's probably require Motif to compile but the 
binaries are available, SGI binaries are there too, though this should be fine 
directly I guess: which makes it the first 'Open' software I can think 
of - for which I think NCBI deserve real praise!

I guess I can get to a full report later today (I have a CD here, the software
is installed, the ISO-9660 drivers are installed, I'm supposed to be writing a
grant :( but it looks really neat, there are links betweeen the databases and
the appropriate Medline entries and good things like this, so check it out!
There is no sensible way (or reason to) move these databases from the CD's. 

And of course the EMBL CD has software on as well, though I haven't tested it 
yet, again it has a good reputation.

Off the CD's there is SRS, which will integrate into a GCG environment, 
though it is VMS specific at present. Again it looks good, though its using 
character cell rather than X windows. (It is nice to have X, but we all will 
need both options for the next few years until everyone has ethernets and X 
windows....)

With all of this collection available, I think the question for GCG has to be:
Unless there is something already in the works (for version 8 say) that adds
significantly to existing software, would it not be better to integrate the 
package with these, rather than adding to the options? 

I guess the other area that should be considered carefully is how to deal with
database in the future; with CDs for nearly everything now (with Brookhaven
being due in October this really only leaves GDB out of the game now) it does 
not make sense for the databases to be held on disk provided that the access 
to them is indexed.   The solution we are exploring at present is the 
Infoserver from DEC; with client software for Ultrix, VMS, DOS and Mac from 
DEC, and for SunOS and other unixes from third party software houses promised 
this is an attractive way of getting all these databases out to all the users 
in a OS independent fashion. Somehow I like the idea of serving the CDs to 
all the machines over the network, and updating by changing the CD ! Hopefully 
by the end of the year this will have happened here and we will have a better 
idea of what is still needed on the VAX. ( lots of free diskspace for users :)

The only thing that we need on disk is the raw sequence data for searching, 
and mostly for searching with software that cannot be done by mail servers 
(ProfileSearch comes to mind) or that one wishes to run locally for whatever 
other reason. Configured appropriately it should be possible to hold the data 
for these in 2 bit format for DNA and 8 bit for protein, and just have the 
.SEQ files, which would solve everyones disk space problems nicely.

Enough, if I wrote grants this fast my problems would be over....

regards,    jasper

Jasper Rees
Oxford University Molecular Biology Data Centre

jasper@vax.path.ox.ac.uk
Comments: Gated by NETNEWS@AUVM.AMERICAN.EDU