From: johnd@physiol.su.OZ.AU (John Dodson)
Subject: Re: ABN Medline gateway available to public ???
Organization: Department of Physiology, University of Sydney, NSW, Australia
Date: Thu, 23 Jul 1992 01:26:05 GMT

In article <1992Jul21.124350.24775@wehi.edu.au> tony@wehi.edu.au wrote...

>	We have been following this ABN/Medline debate with interest. All those
>wishing for an AARNet service are, I am sure, well intentioned, but what
>SERVICE do you all have in mind. It is all very well to have the Medline data,
>but you need some software to access it.

He then went on to highlight the following points (which I have precis'd and
marked with ">      *") that I will comment on...

>	* The current Medline runs on IBM/MVS with tape updates from the US.

I most certainly would not advocate a system like that ;-)
But as I understand it that is also what it runs on in the US (I hope I'm wrong)

>	* The (private) CD vendors DO NOT like the idea of their CDs, data and
>	  indexes being networked. Their business plan was... 
>
>		x number of scientists in the world
>		sell nearly x scientists each a subscription
>		hopefully sell as close to x scientists a CD drive
>		and hopefully a few PCs to boot!
>		= a tidy profit!

I agree they see networking as cutting profits. Which confirms the fact that
they have been narrow minded, short sighted & even in some cases stupid ;-)

However as you point out later they are (eventually) prepared to give you a
multiuser licence when you push them hard enough.

The best company I have found in this regard is Silver Platter, (Aldis is the
Australian Distributor among others) they are "network aware" & partially
"network prepared". Others Like Compact Cambridge who are depending on the
copyright on their index will hopefully not last long in the "networking"
marketplace. (this is me being narrow minded, short sighted & stupid ;-)

>(You should see a "technical" paper that they are circulating supposedly
>explaining why it is technically impossible to mount a networked service!)

I would like to see it. (Could you possibly fax it to me ? on 02-692-2058)

>	* We have Medline CDs on a Mac fileserver (6 CD drives in a 14 drive
>	  tower) serving 50 Macs.
>	  The company that we are dealing with seemed uncomfortable
>	  with this arrangement, but eventually we got a 8 concurrent user 
>	  licence for their software and we strictly enforce it. But the
>	  software is not network aware. It only allows searching of one CD at
>	  a time. You cannot mount multile years simultaneously to do a search.
>	  We would like to put more CD drives in the tower for more years
>	  of Medline, Current Contents, EMBL, GenBank, Entrez DNA data etc
>	  (Yes I know that we could copy to hard disc, and that's what we might
>	  have to do).
>
>	* PCs (IBM compatibles) cannot access the Mac fileserver Medline CDs
>	  (the only option being to purchase a Novell server & more CDs)

Lets look first at costs of CD verses hard drives...
	- For all Medline CDs you need 14(at present) CD drives @ $800-$1000
	  each. (I may be a bit expensive here ;-)

	- You can buy large (>2Gb) SCSI Hard discs for about $2 per MB.
	  (and the price is falling)

	- Each CD does not usually contain the Maximum 600Mb of data files.
	  So you should not need 14 x 600Mb of hard disc.

	- Therefor the cost difference between hard discs & CDs would not be
	  great. (and you don't have controller problems needing to attach 14
	  or more CD drives - have say 1 CD drive for loading & 3 x 2Gb drives
	  for the data)

	- When you have 8 users accessing the same CD performance is very bad.
	  (changing CDs is a pain)

	- Hard discs with 8 users would have an average server 90% idle
	  performance would be better than 1 user with his own CDs.

	- So performance would be far greater with Hard discs than CDs.

Other CDs on the server (whatever is chosen) would be a simple operation (but
you will need to read on to find out why)

Searching multiple years of Medline is a matter of implementation of indexes
(read on).

Now the problem of mixing Macs & PCs...
(Oops almost forgot all those Unix/Vax-VMS and a host of other users out there)

In order to satisfy the need for a server that is "All things to all people"
or more precisely hardware independent we need to consider a totally new
concept (over buying commercial software) because as yet a commercial
solution to this problem does not exist, and the vendors are reluctant to
work on it since it could mean a drop in revenue.

The system I would advocate is one based on a WAIS (Wide Area Information
Service) system, where the users & their terminals/Macs/PCs act as clients
to the server. WAIS clients are available for many hardware platforms (including
Mac, MS-DOS, X and dumb terminals)

>	* And all of you who are thinking of writing your own access software
>	  for the CDs, be aware that there is a claim of copyright on the
>	  encoded indexed files etc. You would be up for a fair bit of reverse
>	  engineering as well.

The easiest way to overcome this is to throw away the vendor index & create
your own. (No reverse engineering needed, besides even using their index is
from the copyright point of view "using their software")
Before you get worried about the amount of work this entails there is already
code to create indexes as part of the "Public Domain - Freely distributable
WAIS package"

>	* So a national service might seem a nice idea, but what would it
>	  entail? 

Load the files from the CDs onto hard discs on a server, build the index...

Initially the Software may not be exactly the same as the current
Medline(elhill) software, but inclusion of facilities that people want would
not be too difficult and the result would be a non-commercial implementation.
(so you could even build your own private local server & get your own CDs
and use the WAIS software)

Searching multiple years is easy if the indexes are created for each Medline
"file" and the search software can cope with multiple indexes (we do it here
now but not with Medline, yet ;-)

>	* We would like Current Contents, EMBL, GenBank, Entrez DNA data etc
>	  as well as Medline.

Current Contents would be a simple addition, it requires an average of 5Mb per
issue, including index (9Mb with Abstracts), One year online is about 250Mb
(less than 500Mb with Abstracts)
(the above figures are from my experience with the "Life Sciences" version,
others would I expect be similar)

Genbank is already supplied at Sydney Uni by Angis (Australian National Genbank
Information Service) on machine morgan.angis.su.oz.au (a SUN Sparc 2). It
shadows the US Genbank database.
I would suggest you discuss EMBL and other DNA type databases with Angis.
However there are network newsgroups in which Genbank and I think EMBL
distribute new data (see newgroups bionet.molbio.genbank.updates and
bionet.molbio.embldatabank) so perhaps you don't need the CDs for these.

I have already discussed the possibility of merging Angis with a National
Medline/Current Contents server, the Angis people are keen. The most cost
effective hardware platform for the server would probably be either a
SUN Sparc 2 or DEC Station 5000/240 with a lot of hard disc ;-)

Accessing a different database is a matter of selection on the WAIS client,
the client then queries the required port on the server (one port per database)
or a different server if that is where the database is.

Authentication of a user is a matter to be decided and is dependant upon
whatever licencing arrangement is made. (It may require inclusion of an
authentication widget in the WAIS server & client, but that is not an
insurmountable problem)

Licencing of the data is another grey area, however I have been given a
networking price of US$250 pa per simultaneous user (This amount is paid to NLM
via the CD vendor) or NLM will accept database usage charges, based on actual
use of the database. (the cost structure of this option I am having trouble
with, as NLM will only negotiate this structure with one national facility,
at present that facility is The Dept of Health in Canberra)

Obviously a national server would have to pay necessary licence fees, but
there is a difference between what some vendors want and what is seen by all
to be fair (I only wish I had an answer to this problem).

>	* Financing direct telnet logins to the NLM seems the only possibility,

Yes, in the short term that is the best solution, until you find out how many
people use it.
Then the overseas network traffic may justify a local server (as would having
other databases available).
The AARNet people (Geoff or Peter) should be able to give us the benefit of
their experience here, there was obviously a point where it was more economical
to purchase the AARNet "archie" machine & run a local ftp archive service than
it was to upgrade the international link to cope with the traffic.

I really think that many people don't realise how popular this kind of service
would be. (but that could be me being open minded, long sighted & stupid ;-)

>	* unless these private companies come to their senses, or NLM forces a
>	  change of contract or someone wants to take them to court - surely
>	  these artifical restraints on networking must be a restraint of trade
>	  or something.

The computer software industry has for many years been full of such restraints,
why do you think IBM mainframe(Dinosuars) have lasted so long ;-)

>			Regards Tony Kyne, WEHI

Sorry to have taken so long to reply (I also accidentally posted an incomplete
version of this - essay - yesterday which I cancelled, so hopefully it was
never seen ;-)

John

John Dodson,					Dept of Physiology,
johnd@physiol.su.oz.au				University of Sydney,
Phone	+61-2-692-3277				NSW 2006
Fax	+61-2-692-2058				Australia.