From: peterd@cc.mcgill.ca (Peter Deutsch)
Subject: Re: Writing an"archie-like" progam, anyone want to pitch in?
Organization: Bunyip Information Systems (the archie people)
Date: Thu, 19 Nov 1992 08:26:12 GMT

In article <46078@ogicse.ogi.edu> nabil@ogicse.ogi.edu (Aaron Nabil) writes:
>I'd like to write a freely availiable archie server, preferably one
>that can read existing archie databases.
>
>If you'd be interested in help out in planning or implementation, 
>drop me a line.

Well, as one of the guys who has devoted most of the past
couple of years of his life to building up and keeping the
existing archie service alive I would have to point out
that the code _is_ freely available. Of course, it is no
longer _free_ to an archie service operator, which is a
different issue, but those who buy it also get a lot of
support, including coordination between the multiple
archie sites, bug fixes, answers to a _lot_ of user's
questions, documentation, etc. I say this only in case
anyone who might be wondering about why we must now charge
for access to archie wonder exactly what we are charging
for....

Although I certainly don't want to discourage
experimentation, I think anyone thinking of turning an
archie-like tool loose on the net should consider the
impact that hundreds or thousands of similar programs
gathering the same data would have on volunteer information
providers, the problems of bandwidth drain at your site (a
popular service can have a _significant_ effect on your
resources) and even the problems providing user support
for freebie services that require significant admin
support. Note that these are all operational issues - we
found that the coding was the easy bit. Supporting the
_service_ (not the project) was the hard bit.

Having said all of this, if you would like to know how
_we_ did it, you might want to pick up a copy of the paper
we did on archie for the January, 1992 Usenix conference.
I believe that it is available at most of the archie sites
through anonymous FTP. It has been circulating for a
while, so should be easy to find (I'd like to be able to
point to a specific machine at McGill, but sadly the
McGill volunteer machine for archie died and noone has
been willing to help the net out with a replacement.
Anyone with suggestions on how we could get such a public
service machine back online in a world where no money is
changing hands for use of what seems to be a useful and
popular service are most welcome to post ideas...)

BTW, I'm not sure I understand what you mean by "read
existing archie databases"? The new release of archie is
capable of supporting arbitrary database systems and
ships with a copy of the public domain version of WAIS.
If you are referring to the ability of querying an
existing archie server for its data, then simply pick up
any of the freely available archie clients and look at the
Prospero code that talks to archie. It's available in
source and is very well written (by Cliff Neuman at ISI).

If you are referring to the anonymous FTP collecions that
we actually track, well simply login and do a dir command.
If you are referring to the ability to read an already
built archie database available directly on your machine,
well this would presumably only be of interest to people
operating our version of archie. Sure the database format
we provide for the anonymous FTP archie database has a
programatic interface but we have not made this freely
available since only those operating an archie server can
even access the database in such a format, so there is no
need to release this (or am I mising something here?)

There is one other possibility you might be referring to
that I can see. We have mechanisms for ensuring
consistency between multiple archies and you might
presumably wish to sync your "archie" (and personally, I'd
really wish you picked another name for what will
obviously be a different project. The people working on an
archie-like server for Gopher called theirs "veronica" and
this certainly should be appreciated by users who could
only be confused by unnecessary naming collisions like
this).

Unfortunately, as a struggling commercial start-up we
don't have the resources to subsidize such an effort by
providing information on this syncing mechanism, but if any
of our clients are interested in this they should give us
a call (after all, they're paying for consultation
support... :-) 

If it turns out that a global database syncing mechanism
is something for which there is significant demand, then
work should probably be done to standardize this through
the IETF and if such an effort is started we'd be glad to
participate (in fact, I'm at the IETF as I write this). We
are certainly not averse to working to make Internet
services interoperable and the IETF is the forum for this.



				- peterd