HDTenEightyP/NetNews huggingface dataset:
From: francois@welchgate.welch.jhu.edu (Francois Schiettecatte)
Newsgroups: comp.infosystems.wais
Subject: Re: FAQ?
Date: 12 Aug 92 13:06:59 GMT
Organization: Johns Hopkins Univ. Welch Medical Library

Mike


In article <1992Aug12.051608.5763@midway.uchicago.edu> mss2@midway.uchicago.edu writes:
>Is there an FAQ or other source of basic information for this group?
>I want to ask some very basic questions (on the order of "What is
>WAIS?"-- I've seen the acronym expanded, and I've been reading the
>group for a few weeks, but I'm still pretty vague) and I'm wondering
>if there's a source where such answers already exist.  Thanks
>


You may want to check the following:


----------------------------------------------------------------------

Where can I find more information about wais?

There are a number of documents located on the following machines:

quake.think.com:/wais
think.com:/wais

Also here is a bibligraphy of wais documents and ftp sites for the 
software:

quake.think.com:/wais/doc/bibliography


----------------------------------------------------------------------


Also I am currently putting together a FAQ for wais and would be
happy to share the contents with others and take submissions, etc.


francois


Francois Schiettecatte
Software Engineer
Welch Medical Library
Johns Hopkins University
Internet: francois@library.welch.jhu.edu
Phone	: (410) 955-7581
Newsgroups: comp.infosystems.wais
From: mcclen@advtech.uswest.com ( Chris McClenaghan)
Subject: wias and proxy internet servers
Organization: US WEST Advanced Technologies, Boulder, Colorado
Date: Thu, 20 Aug 1992 16:21:50 GMT

We use a gateway host to internet with access through a ``proxy''
style of ftp and telnet. I know how to make ange-ftp work through
this proxy, but need a pointer into the wais code. Could someone
help me out?

--
Chris McClenaghan    mcclen@advtech.uswest.com

Newsgroups: comp.infosystems.wais
From: davidsen@ariel.crd.GE.COM (william E Davidsen)
Newsgroups: comp.infosystems.wais,comp.lang.postscript,alt.security
Subject: Re: WAISified PostScript would constitute a security hole
Date: 17 Aug 92 15:49:54 GMT
Reply-To: davidsen@crd.ge.com (bill davidsen)
Organization: GE Corporate R&D Center, Schenectady NY

In article <skyhawk.713832721@milton>,
 skyhawk@milton.u.washington.edu (Scott Northrop) writes:

| Is a setuid-root executable that setuid()s to a safe user (like nobody) a
| reasonable way to "start up a subprocess with limited privileges without having
| to have super-user authority"?  I hope so, it's what I've always done.  This is
| what Mr. da Silva was saying, though he wasn't being explicit about the setuid
| bit.  (Of course, it's also good practice to toast the environment variables
| that aren't needed -- no telling what holes you might open up otherwise.)
| 
| If necessary, a chroot() could also be done, but I have my doubts about how
| much this would impede the functionality of the previewer, and how necessary it
| would be in the first place.

  I see no benefit to having the program setuid root so it can setuid
user, when you can make it owned by user and set the setuid bit in the
first place. Maybe you're doing something as superuser first, but in
general I don't see any benefit to going through a two step process.

-- 
bill davidsen, GE Corp. R&D Center; Box 8; Schenectady NY 12345
    I admit that when I was in school I wrote COBOL. But I didn't compile.
From: brewster@quake.think.com (Brewster Kahle)
Newsgroups: comp.infosystems.wais
Subject: Re: How many servers are there ?
Date: 16 Aug 92 16:44:30 GMT
Organization: Thinking Machines Corporation, Cambridge MA, USA
NNTP-Posting-Host: quake.think.com
In-reply-to: jjkim@cosmos.kaist.ac.kr's message of 27 Jul 92 02:04:41 GMT

In article <1992Jul27.020441.1349@kum.kaist.ac.kr> jjkim@cosmos.kaist.ac.kr (Kim Jungju) writes:

   WAIS, GOPHER and WWW servers are increading. Does anyone out there
   know how many servers of them are running now ?
   And if possible could someone make a table like below ?

   The number of information servers running on Internet.

There are 282 WAIS servers right now.  If you look back through the
wais-discussion@think.com archives (available on quake.think.com) or on the
wais-discussion-archive.src WAIS server you will get a some datapoints.

-brewster

--
Brewster Kahle					Thinking Machines Corporation
Brewster@Think.com				245 First Street
Project Leader					Cambridge MA 02142
Wide Area Information Servers			617-234-2874 (or -1000)
From: jxh@netcom.com (Jim Hickstein)
Newsgroups: comp.infosystems.wais
Subject: Trouble getting xwais to connect reliably to sources
Date: 22 Aug 92 01:11:33 GMT
Organization: Netcom - Online Communication Services (408 241-9760 guest)

I have built xwais (from wais-8-b5) on a Sun 3/60 running SunOS 4.1.1.
I use it under X11R5, on a standalone host that has a SL/IP connection
to the Internet.  I do NOT use NIS, nor do I want to, so I went and got
resolv+ (version 1), and things generally work.  But xwais has two
problems.  First, it cannot make a connection to a host by name, unless
I stick it in /etc/hosts.  Obviously I'm doing something wrong with my
libc.so and all that, but everything else works fine.  Do I need a
"-ldl" in the Makefile or something?

Second, during a session, even after a successful connection has been
made, it will sometimes barf, saying that the socket is already
connected.  I quote:

Connect to socket did not work (1): Socket is already connected
Connect to socket did not work (2): Socket is already connected
host='archie.au' port=210

I can reproduce this simply by picking the first source,
aarnet-resource-guide.src, in the standard list and trying to search
for something.  I also get this same message later on, apparently due
to attempts to connect to hosts that had failed in the past.  I added
the last line, so I would know who it was trying to talk to when it
failed.

Is this a known bug?  I tried to use WAIS itself to find a list of bugs
and patches (right?), but struck out.  The best I could do was dig
through a source of old alt.wais stuff, to no avail.  Where do I find out
about the very latest patches and reported bugs?  Why is there no WAIS
source for this? (There must be: I just don't know where it is.)  Help!
Organization: Distributed Workstation Services, Carnegie Mellon, Pittsburgh, PA
Newsgroups: comp.infosystems.wais
Date: Tue, 18 Aug 1992 16:03:29 -0400 
From: Robert Joseph Kuszewski <lizard+@CMU.EDU>
Subject: Re: Comments? (Re: Adapting to post-boolean searching)
	<1992Aug18.142525.12588@cs.joensuu.fi>

Excerpts from netnews.comp.infosystems.wais: 18-Aug-92 Comments? (Re:
Adapting to.. Jarmo Vaisanen@joyds2.jo (1490)

> I need information for explaining to other people the way WAIS search
> engine ranks "and" questions.

This is my interprutation of what is done for AND in WAIS, please
correct me if I'm wrong.  I'll use Joe Jackson's "secret information"
string as my explanitory search string.

For multuple word searches, WAIS effectively breaks the query up into a
query for each word(a search on "secret" and a search on "information").
 It then takes the resultant sets, merges them together, sorts them by
number of hits, and presents this list to the user.  Unfortunately, this
does not achieve the expected goal of a logical AND, but the resultant
set it equivalent to a logical OR.

In my opinion, this is a poor solution at best.  There are two things
that I think are very wrong here:  (1) The expected default operator is
AND; (2) The actaul default operator is OR.

The current standards folk have agreed that ADJ (adjacency) should
really be the natural/logical default operator.  I originaly disagreed
with this, but have worked in a system that uses it (Mercury's LIS II)
for two years now and I have learned to like it.  Hopefully, this will
change when WAIS moves to z39.50 1992 (popularly known as version 2).

The fact that the wrong set is returned to the user bothers me.  Some
have suggested reordering of the resultant set to divide what should be
the results of AND from the rest of the results of OR, but the fact
remains that you are showing the user information that the user did not
ask for.  My personal opinion on the subject is:  always give the user
the set s/he asked for.  If the user specifies too narrow a search, then
the user will try again with a more open search. 

-Bob
From: lizard+@CMU.EDU (Robert Joseph Kuszewski)
Newsgroups: comp.infosystems.wais
Subject: Re: Comments? (Re: Adapting to post-boolean searching)
Date: 19 Aug 92 13:51:16 GMT
Article-I.D.: andrew.4eYg9Iq00VR5MWU1JK
	<1992Aug19.155848.4430@cunews.carleton.ca>
Organization: Distributed Workstation Services, Carnegie Mellon, Pittsburgh, PA

Excerpts from netnews.comp.infosystems.wais: 19-Aug-92 Re: Comments?
(Re: Adapting.. Neal Holtz@cascade.carle (1456)

>  It is easy to tell when the search is too open.

The question is: If you insist that the search always remain too open,
how do you reasonably restrict that search? Perhaps a user-setable
default operator is the best solution, but changeing the meaning of the
intuitive logical AND is a poor choice.

I agree that for some users and some datasets, it is often optimal to
return a larger dataset than the user requested becuase that expansion
will only amount to a small number of records.  Unfortunately, we are
dealing with all kinds of users and all kinds of datasets.

On the other hand, for advanced users and for many datasets, it is best
to return the exact set that the user requested.  I often find that the
difference between a search of "secret AND information" versus "secret
OR information" is a thousand records.  Clearly, I need the ability to
get the overlap of the two sets when I request the overlap of the two
sets.

Does making the default search operator a user-setable variable sound
like a good idea to anyone else?

-Bob
From: Joseph_Jackson@transarc.com
Newsgroups: comp.infosystems.wais
Subject: Adapting to post-boolean searching
Date: 17 Aug 92 13:46:33 GMT
Article-I.D.: transarc.8eY1st70BwxIB2AnYW
Organization: Carnegie Mellon, Pittsburgh, PA

Folks,

I'm not catching on to the new mode of thinking required by these
"post-boolean" search engines!  Perhaps someone can help indoctrinate
me in the new ways of searching.

Suppose I have a set of documents that I want to search to obtain some
secret information.  I know that the document has the words "secret"
and "information" in it, so I use WAIS to search on:

		secret information

The target document is way at the bottom of the resulting hit list.
At the top were a bunch of documents which contained several
occurances of "secret" and a bunch which had many occurances of
"information".  None of the top documents had both words in them,
however, so I'm sure that none are the ones I want.  The "louder"
documents managed to drown out the one I was looking for by sheer
repetitiveness.

That example I contructed from three files just to demonstrate the
point to myself before posting this message.  What I really wanted to
do is search the wais-*-archives sources for messages that discuss
alternative search engines, so I used the phrase:

		boolean search 

Among the top contenders on the hit list is a waisserver statistics
script which contains "search" a whole bunch of times but not the word
"boolean".  From this I learned to shorten my search string to just:

		boolean

That teaches me that adding extra words can be detrimental to a
search, widening it out to left field.  From what I've been reading,
people seem to think that you can type anything at WAIS and the noise
will automatically get stripped out by the weighting scheme and/or
relevance feedback.  I don't see how it can help.

Then there's this opinion:

> Date: Thu, 3 Oct 91 15:09:44 EDT
> From: Franklin Davis <fad@Think.COM>
> Subject: boolean searching & suffixing

> I believe the only reason people are satisfied with boolean
> search on keywords is that it's the only hammer that has been
> available.  Full-text search, and relevance feedback on full
> text, have been too computationally demanding...until recently.
>
> Once we let people out of the boolean/keyword straightjacket,
> they may wonder why they ever did it that way in the first place!

Wonderful concept, and I'd really like to believe it, but so far I've
been amazed at how little I can use WAIS without some way of
specifying simple conjunctions.  Is there another approach that I
should be taking that WAIS will satisfy?

Joe Jackson.
Newsgroups: comp.infosystems.wais
From: nholtz@cascade.carleton.ca (Neal Holtz)
Subject: Re: Comments? (Re: Adapting to post-boolean searching)
Organization: none
Date: Wed, 19 Aug 1992 15:58:48 GMT

In article <QeYJSFW00VR58V2t9j@andrew.cmu.edu> Robert Joseph Kuszewski <lizard+@CMU.EDU> writes:
>In my opinion, this is a poor solution at best.  There are two things
>that I think are very wrong here:  (1) The expected default operator is
>AND; (2) The actaul default operator is OR.
>
>The current standards folk have agreed that ADJ (adjacency) should
>really be the natural/logical default operator.

In almost all of my queries so far, I have wanted the AND operator
(and certainly not adjacency -- there are too many permuations to
try).  For example, I was trying to search the nsf-awards database
for all abstracts containing the words "civil", "engineering" and
"standards".  There may have been one or two relevant documents
but I couldn't tell -- there are approximately 300 documents in
the result set.  Adding more common keywords simply doesn't help.

>  My personal opinion on the subject is:  always give the user
>the set s/he asked for.  If the user specifies too narrow a search, then
>the user will try again with a more open search. 

The problem is, a user will not always know when the search is too narrow.
Does a small result set mean that there are few relevant documents,
or does it mean that you simply missed some?  It is easy to tell when
the search is too open.

--
Prof. Neal Holtz,  Dept. of Civil Eng.,  Carleton University,  Ottawa,  Canada
Internet: nholtz@civeng.carleton.ca  Ph: (613)788-2600x5797 Fax: (613)788-3951
From: sanderso@dcs.glasgow.ac.uk (Mark Sanderson)
Newsgroups: comp.infosystems.wais
Subject: Re: Adapting to post-boolean searching
Date: 18 Aug 92 09:03:08 GMT
Organization: Computing Sci, Glasgow Univ, Scotland

In article <1992Aug17.235813.21461@cs.cornell.edu> allan@cs.cornell.edu (James Allan) writes:
>Joseph_Jackson@transarc.com writes:
>
>So instead of 'boolean search' try some more elaborate such as
>'boolean search of full-text documents in a network environment such
>as the Internet using software such as WAIS'.  Well, if that's what
>you're interested in.... :-)

Thats all very well saying that but if the user doesn't know any other words
then there is a problem. James Allan's boolean system produced better results
than WAIS. It wouldn't surprise me if this doesn't happen quite often, it is
well known that user's typically enter very small queries. To say that larger
queries should be entered isn't really a solution.

The document ranking procedure in WAIS should be changed, to put a document
with multiple single hits above a document containing all the user's query
words seems wrong. Probably the better way of organising the ranking is to 
rank documents based on a binary occurance (is the term in the doc or not ?) of
query words in the document. This way documents containing the largest number
of query words will apear at the top of the ranking.


Mark S.
From: matt@ra.oc.com (Matthew Lyle)
Newsgroups: comp.infosystems.wais
Subject: I must be doing something wrong with waisindex
Date: 21 Aug 92 02:48:53 GMT
Distribution: usa
Organization: OpenConnect Systems, Dallas, TX

No doubt I'm missing something obvious, but here goes...

I'm indexing a large number of text files with a command
in the following format:

  waisindex -d /wais/wais-sources/prs_vms -r -export -e waisindex.err \
     -pos -pairs -nocat -contents -t first_line /usr/notes/wais/vms

(continuation mark added for clarity)

It is set up to run out of inetd:

  z3950	stream	tcp	nowait	root	/wais/bin/waisserver \
    waisserver.d -e/wais/server.log -l 10 -d/wais/wais-sources 

The source file looks like this:

  (:source 
     :version  3 
     :ip-address "137.1.0.99"
     :ip-name "mistxt.oc.com"
     :tcp-port 210
     :database-name "prs_mvs"
     :cost 0.00 
     :cost-unit :free 
     :maintainer "root@mistxt.oc.com"
     :description "Server created with WAIS release 8 b5 on Aug 20 17:52:35 1992 by root@mistxt.oc.com
  The files of type first_line used in the index were:
     /usr/notes/wais/mvs
  "
  )

But I get these errors in server.log:

  44989: 0: Aug 20 18:16:46 1992: 1: Accepted connection from:  [192.82.215.92],
      WAIS release 8 b5
  44989: 1: Aug 20 18:16:46 1992: 100: Init message: swais WAIS release 8 b5, 
     from host: ra.oc.com.
  44989: 2: Aug 20 18:16:48 1992: 3: Search! Database: prs_admin_util, 
     Seed Words: snsadmin
  44989: 3: Aug 20 18:16:48 1992: -1: can't open the word hash file 
     ./prs_admin_util.dct
  44989: 4: Aug 20 18:16:48 1992: 4: Returned 0 results.  Aww.
  44989: 5: Aug 20 18:19:15 1992: 3: Search! Database: prs_admin_util, 
     Seed Words: snsadmin
  44989: 6: Aug 20 18:19:15 1992: -1: can't open the word hash file 
     ./prs_admin_util.dct
  44989: 7: Aug 20 18:19:15 1992: 4: Returned 0 results.  Aww.
  44989: 8: Aug 20 18:20:28 1992: 2: Done handling client

Now I thought that the "-d /wais/wais-sources" would cause to set the
default directory for the indices, but lines #3 and #6 indicate that is
not happening.  I am running this on:

	RS/6000
	AIX 3.2 (I know... I know...)
	WAIS 8-b5
	GCC 2.2.2

Any suggestions as to what I am doing wrong??  I really don't want
to have to go through all of the *.src files and fully qualify
the path names to the database.

Thx...
-- 
Matthew Lyle                                           matt@oc.com
Technical Support Specialist                           matt@utdallas.bitnet
West/SW/SE/Mid Atlantic Region (*Phew*)
OpenConnect System, Dallas, Texas                      (214) 490-4090
Newsgroups: comp.infosystems.wais
From: marc@stc.lockheed.com (Marc Fleischmann)
Subject: Indexing large databases
Reply-To: marc@stc.lockheed.com
Organization: LMSC Software Technology Center
Date: Tue, 18 Aug 92 15:11:04 GMT

I have finally figured out the changes necessary to get b5 to handle our 
master library card catalog and other databases that are larger than 150mb.

To recap:

I built vanilla b5 and started to index our library catalog.  I got the
error

 6901: 999999: Aug  5 14:18:17 1992: -1: Fatal Error:    In a call to
 write_bytes, the value 65543 can not be represented in 2 bytes

I changed  DICTIONARY_ENTRY_HASH_CODE_SIZE  from 2 to 3 and rebuilt
wais without errors and started again.

I indexed the database and it worked but I got lots of:

10785: 290: Aug 11 04:45:14 1992: 100: Can not merge the index block
for recordid, since it would create one that is too big.  Deleting it.

I went back to irfiles.h and modified INDEX_BLOCK_SIZE_SIZE from 2 to 3
and rebuilt wais.  I indexed the file again and got no errors but when
I tried to search the database using xwais or swais I got a segmentation
fault.

The fix to this problem seems to be that in irfiles.h INDEX_BLOCK_HEADER_SIZE
has to be set to be the sum of:

	#define INDEX_BLOCK_SIZE_SIZE 3
	#define NEXT_INDEX_BLOCK_SIZE 4 
	#define INDEX_BLOCK_FLAG_SIZE 1

	#define INDEX_BLOCK_HEADER_SIZE 8

I rebuilt wais again and indexed my files without a problem.  Swais and
xwais work both on the local database and accessing b3 and b4 servers.

While I was at it I did make the modifications described at the end of 
the ir/README to:

	 DOC_TAB_ENTRY_FILENAME_ID_SIZE
	 DOC_TAB_ENTRY_HEADLINE_ID_SIZE
	 DOC_TAB_ELEMENT_SIZE 

/marc

Marc Fleischmann - Information Integration
Lockheed Missiles & Space Company, Inc. - R&D Division
Palo Alto CA 94304-1191 (415) 354-5693
Internet: marc@stc.lockheed.com




---
Marc Fleischmann - Information Integration
Lockheed Missiles & Space Company, Inc. - R&D Division
Palo Alto CA 94304-1191 (415) 354-5693
Internet: marc@stc.lockheed.com
From: francois@welchgate.welch.jhu.edu (Francois Schiettecatte)
Newsgroups: comp.infosystems.wais
Subject: xv and PICT
Date: 13 Aug 92 13:20:48 GMT
Organization: Johns Hopkins Univ. Welch Medical Library


For those of you who want to display PICT files
on a X based system using xv, here is a modified
version of the wais-pict-display script:



	#!/bin/sh
	if [ "$1" != "" ]; then
	  foo="if=$1"
	else
	  foo=""
	fi
	(dd ${foo} bs=128 skip=1 | /big/wais/bin/picttoppm | \
	/big/wais/bin/ppmtogif > $1.tmp; mv $1.tmp $1; \
	/big/wais/bin/xv $1 >/dev/null) 2>/dev/null


francois

Francois Schiettecatte
Software Engineer
Welch Medical Library
Johns Hopkins University
Internet: francois@library.welch.jhu.edu
Phone	: (410) 955-7581
Newsgroups: comp.infosystems.wais,comp.lang.postscript,alt.security
From: nagle@netcom.com (John Nagle)
Subject: Re: WAISified PostScript would constitute a security hole
Date: Wed, 12 Aug 92 16:34:21 GMT
Organization: Netcom - Online Communication Services  (408 241-9760 guest) 

      You don't want to trust some big interpreter that executes 
foreign programs.  That's a big piece of code to trust.  It might have
security holes.  It probably does.  You want to be able to execute such 
programs in an environment where they can't do any harm.

      What's needed is the ability to start up a process with limited
privileges.  UNIX doesn't have this.  Anything a UNIX user runs can do anything
that user can do.  And that's no good.

      Short of a capability-based operating system, though, it's going to
be very difficult to fix this.  On the other hand, if it isn't fixed,
a few years downstream, viruses that use this medium may start to appear.
Especially if something like Netnews starts supporting PostScript.

					John Nagle
From: brewster@quake.think.com (Brewster Kahle)
Newsgroups: comp.infosystems.wais
Subject: Re: Running mac wais off a file server?
Date: 16 Aug 92 18:41:17 GMT
Article-I.D.: quake.BREWSTER.92Aug16114117
	<CKD.92Jul24210754@loiosh.eff.org>
	<1992Jul26.014844.10149@scott.skidmore.edu>
	<1992Jul26.114508.8887@news.acns.nwu.edu>
Followup-To: comp.infosystems.wais
Organization: Thinking Machines Corporation, Cambridge MA, USA
NNTP-Posting-Host: quake.think.com
In-reply-to: Albert-Lunde@nwu.edu's message of 26 Jul 92 11:45:08 GMT

In article <1992Jul26.114508.8887@news.acns.nwu.edu> Albert-Lunde@nwu.edu (Albert Lunde) writes:

   >>Ah!  I think I know what the problem is.  0.6.2, at least, *must* be on
   >>the startup disk.  (A message that just came across wais-talk tickled my
   >>brain on this one.)
yes, this is a bug.  sorry about that.

   I'm also concerned about other related issues.  We would like to run Mac 
   WAIS in our labs multi-launch off of a read-only server folder.  It seems 
   like the usual organization of source and document folders does not allow this.

You can indeed do this, but any updates you are thinking you are making do
not appear in the files (very very confusing).  We do this, however, in our
demo areas so that the application will look the same every time we
restart.

-brewster

--
Brewster Kahle					Thinking Machines Corporation
Brewster@Think.com				245 First Street
Project Leader					Cambridge MA 02142
Wide Area Information Servers			617-234-2874 (or -1000)
From: smyth@ac.dal.ca
Newsgroups: comp.infosystems.wais
Subject: WAIS for the PC
Date: 20 Aug 92 15:40:35 -0300
Organization: Dalhousie University, Halifax, Nova Scotia, Canada

I am looking for a DOS or Windows WAIS that works.  I have downloaded and
tried a couple, but both fail intermittently.  Suggestions would be
appreciated.

- Frank Smyth
  Dalhousie University
  SMYTH@AC.DAL.CA
Newsgroups: comp.infosystems.wais