Date: Thu, 19 Nov 92 18:48:09 PST
From: barries@nye.nscee.edu (Fred Barrie)
Original-To: gopher-news@boombox.micro.umn.edu
Subject: Status report on Veronica development.
Newsgroups: comp.infosystems.gopher
Distribution: comp


Status report on Veronica development.     11-19-92

1.	Imminenent enhancements to Veronica:

	A.	Veronica should return one and only one reference to each 
		distinct item on the Net.  We are experimenting with several 
		approaches and will implement a trial of at least one method,
		in the next few days.

		The Veronica database will be rebuilt to give faster accesses
		with the redundancy-removing algorithm.

2.	Preliminary thoughts on proliferation of Veronica servers.

	A.	Each organization possessing several gopher servers should
		be encouraged to run a site-wide Veronica service for those
		gopher servers.

	B.	There should be "several" Veronica servers which index all
		of gopherspace, possibly excepting the most transient data
		( see C below ).   Thusfar, our little NeXT Veronica at 
		Nevada has not crashed under the load  ( typical "load"
		between 3 and 8 ).  But it's response is far from fast enough.
		These large Veronicas need faster hardware.

	C.	Specialized Veronica servers for transient data.  USENET
		offerings in particular need not be indexed and reindexed
		by all the Veronica servers.  We have found it difficult to 
		automatically filter USENET news directories and to 
		eliminate redundant presentation of USENET items.  Total 
		Veronica loads could be reduced by implementing specialized
		Veronica servers for such transient data.

	D.	Some Veronica servers could specialize by topics.  This could 
		be implemented easily if these servers got the relevant 
		data by ftp from the 'large' Veronicas of B. above.
		We think it would be much more effective to acquire the data
		from the large servers, rather than propagating 100s of
		gopherwalks all over the net.

		In general we are wary of this proposal.  The utility of 
		Veronica derives from its ability to index "everything".
		Topic subdivision is only as good as the keyword scheme.
		We think those schemes are never very good.  It is much 
		better to have a fast index of "everything" than to have
		a very fast index that has been subjected to someone else's
		preconceived categories.

		Several early commentators have suggested that topical
		division are important to Veronica's utility.  Most of
		these comments objected to a volume of seemingly irrelevant
		items returned by the searches.  We think this volume will
		be much reduced when we elimate duplicate items.  Further,
		we think that much of the annoyance was caused by WAITING
		for duplicate and seemingly-irrelevant items.  We think
		that faster searches without redundant items can satisfy
		most users while still indexing "everything".

		We recognize that we may repent this opinion if (when) the
		volume of gopherspace explodes.   

		Meanwhile, we suggest that the following additions to 
		the gopher+ spec could make Veronica more efficient. 


3.	Suggested additions to gopher+ protocol.
	Add some information to the new gopher-item fields.  

	A.	An "indexing" field ( boolean ) could be added, perhaps
		to the ADMIN block.  A Veronica could ignore items so marked,
		such as News directories.

	B.	A "subject=" field could be added.  This would rationalize
		the topic-oriented searches discussed in 2-D above.
		"Subject" must take multiple values.  We recognize that this
		is a watered-down ABSTRACT.
		
		Items lacking a "subject" field would inherit "subject"
		from their parent.

	C.	A boolean "local" field.   This would help prune the 
		duplicate references.
		

Fred Barrie, Steve Foster
University of Nevada
Newsgroups: comp.infosystems.gopher