HDTenEightyP/NetNews huggingface dataset:
From: jik@athena.mit.edu (Jonathan I. Kamens)
Subject: ENHANCEMENT: (8 b5) more granularity for -mem option
Organization: Massachusetts Institute of Technology
Date: Thu, 13 Aug 1992 22:55:11 GMT

This is a small patch to allow "-mem 3" and "-mem 4" to have different
meanings from "-mem 2" and "-mem 5" and from each other, to allow for
more granularity in that range of memory usage.  I found this
necessary because I was running out of memory with -mem 5, but not
with -mem 1, and I know that the amount of memory I could actually use
was somewhere in between the two.

*** 1.47	1992/05/10 14:48:17
--- irbuild.c	1992/08/13 01:21:56
***************
*** 644,649 ****
--- 659,672 ----
      else if(memory_to_use <= 2){
        hashtable_size = 1L<<16;
        flush_after_n_words = 50000;
+     }
+     else if(memory_to_use <= 3) {
+       hashtable_size = 1L<<16;
+       flush_after_n_words = 80000;
+     }
+     else if(memory_to_use <= 4) {
+       hashtable_size = 1L<<16;
+       flush_after_n_words = 110000;
      }
      else if(memory_to_use <= 5){
        hashtable_size = 1L<<16;

Jonathan Kamens                                         jik@MIT.Edu
MIT Information Systems/Athena              Moderator, news.answers
    (Send correspondence related to the news.answers newsgroup
        {and ONLY correspondence related to the newsgroup}
                 to news-answers-request@MIT.Edu.)
Newsgroups: comp.infosystems.wais
From: peter@julian.uwo.ca (Peter Marshall)
Subject: Inconsistent headline/header length in waisindex
Organization: University of Western Ontario, London
Date: Wed, 19 Aug 1992 01:57:09 GMT
Keywords: MAX_HEADLINE_LEN MAX_HEADER_LEN

There seems to be some inconsistency between the various files that
make up waisindex.  In the header file irlex.h, MAX_HEADLINE_LEN is
defined to be 300 (characters).  This is used by irfiles.c and
irsearch.c.  But in ircfiles.c there is a very similar variable
MAX_HEADER_LEN that is set (in the 8-b5 release) to 100 characters.  I
have some long headlines in the journalism.periodicals database and as
far as indexing was concerned the first line of each entry was being
truncated at character 100.  I just made both variables 300 and the
problem was solved, but probably (for 8-b6?) a single variable should
be used or at least one should be defined in terms of the other.
-- 
--
Peter Marshall, Manager (Academic Networking)
CCS, NSC, U. of Western Ontario, London, Canada N6A 5B7 (519)661-2111x6032
Newsgroups: comp.infosystems.wais,comp.lang.postscript,alt.security
From: peter@taronga.com (Peter da Silva)
Subject: Re: WAISified PostScript would constitute a security hole
Organization: Taronga Park BBS
Date: Thu, 13 Aug 1992 12:16:02 GMT

In article <cx4mcgb.nagle@netcom.com> nagle@netcom.com (John Nagle) writes:
>      What's needed is the ability to start up a process with limited
>privileges.  UNIX doesn't have this.

Oh, I don't know about that. "chroot" seems pretty comprehensive. An
advantage of putting everything in the file system. If you're REALLY
paranoid you can setuid to a user-id with no files of its own.
-- 
                                                                `-_-'
                         Have you hugged your wolf today?        'U`

Peter da Silva, Taronga Park BBS, Houston, TX  +1 713 568 0480/1032
Newsgroups: comp.infosystems.wais
From: vaisanen@joyds2.joensuu.fi (Jarmo Vaisanen)
Subject: Comments?  (Re: Adapting to post-boolean searching)
Reply-To: vaisanen@joyds2.joensuu.fi (Jarmo Vaisanen)
Organization: University of Joensuu, Computer Center
Date: Tue, 18 Aug 1992 14:25:25 GMT


Hi!


I need information for explaining to other people the way WAIS
search engine ranks "and" questions.

Why was the current method selected?  What good points does it have? 
Bellow are my personal feelings about current method.


James Allan writes:

> Note that in the model WAIS
> uses, OVER-specifying your request is unlikely to hurt you (by
> discarding documents that are really close to your request, but
> missing one or two words).

My opinion is that it does hurt.  This might be cause a flame war
but I personally feel that simple 'and' questions like "boolean search"
should rank documents with both words higher than documents with
e.g. many boolean words.

I feel current query ranking quite annoying so that finding
relevant information is hard.


Mark Sanderson writes:

> The document ranking procedure in WAIS should be changed, to put a document
> with multiple single hits above a document containing all the user's query
> words seems wrong. Probably the better way of organising the ranking is to 
> rank documents based on a binary occurance (is the term in the doc or not ?)of
> query words in the document. This way documents containing the largest number
> of query words will apear at the top of the ranking.

I agree with this!  


-----------------------------------------------------------------------
Jarmo V{is{nen          vaisanen@joyds2.Joensuu.FI      (Internet)
University of Joensuu   JVAISANEN@FINUJO                (EARN, BITNET)
Computer Center
From: witts@hoss.unl.edu (Rick Wittstruck)
Newsgroups: comp.infosystems.wais
Subject: Waistation on Mac crashes on PICT downloads
Date: 20 Aug 92 16:20:30 GMT
Organization: University of Nebraska - Lincoln


	I am trying to get PICT files to work with the Mac Waistation
software.  I have a scanned black and white PICT file on our Sun
workstation which is around 330k in size.  I load the Waistation software
on the Mac Quadra 700 with no other programs in the Multifinder, and every
time I try to view the large 330k PICT file I get a "memory getting low"
message, followed by a "Mac OS error" dialogue box.  These two error
messages take turns popping up on the screen, and all I can do is reset
the machine.  Is this a problem with the 8-b5 wais software, or does Mac
Waistation choke on large PICT file downloads?


	thanks,
	Rick Wittstruck
	rwittstr@unlinfo.unl.edu
	Computing Resource Center
	Univ. of Nebraska-Lincoln
From: jcurran@nic.near.net (John Curran)
Newsgroups: comp.infosystems.wais
Subject: Re: Adapting to post-boolean searching
Date: 18 Aug 92 20:40:12 GMT
Organization: NEARnet, Cambridge, MA
NNTP-Posting-Host: nic.near.net

In article <1992Aug17.235813.21461@cs.cornell.edu> allan@cs.cornell.edu (James Allan) writes:
>In full-text searching of unrestricted subject matter, though, you
>probably don't know how the vocabulary is used, so you're quite likely
>to get yourself in trouble with booleans--typically by over
>restricting your search with too many keywords.

This can easily occur, but the capability to restrict searches to items
containing particular phrases is still desperately needed in any target-
rich search.  

We have given folks the ability to simultaneously search a few dozen
databases, but don't provide the ability to enforce a particular set of 
words in resulting documents.  This produces results sets that often
resemble a study in chaos theory.

For example, attempt to search the various sources for the FTP specification.
If you specify "FTP", you get dozens of pointers to files available via ftp.
If you specify "File Transfer Protocol", be ready; it will give you back 
40 documents, and some of them might even be related to protocols for 
moving files about.  If you get one that is that actually about 
"File Transfer Protocol", you should consider yourself very lucky.
This type of problem occurs whenever there is a particular phrase with
a meaning significantly different from the root words. "New England", 
"sweet meat", and "white dwarf" are all fine examples.

Note that relevance feedback doesn't help here: one of the items item 
returned for the ftp search could be the protocol spec, which has little 
resemblance to the latest Internet Anonymous File Archive notes, although 
the later explicitely mentions "File Transfer Protocol" in the text.
A more relevant document (in terms of word occurance) is the TELNET spec.

I'm not sure people are looking for full boolean operations as much as the
ability to specify forced word adjacency for well known phrases.

/John
Newsgroups: comp.infosystems.wais
From: mycroft@hal.gnu.ai.mit.edu (Charles Hannum)
Subject: Re: WAISified PostScript would constitute a security hole
Organization: MIT Laboratory for Computer Science
Date: Sat, 15 Aug 1992 21:08:36 GMT


In article <1992Aug12.201304.396@news.iastate.edu>, John Hascall
(john@iastate.edu) writes:
>
> Oh?   What about:
> 
> ---s--s--x  1 nobody   nobody          0 Aug 12 15:04 psnewsrdr

Then it can't access my PostScript file or my X or Kerberos tickets
(if I have appropriate permissions set).

-- 
 \   Charles Hannum, mycroft@ai.mit.edu
 /\  White heterosexual atheist male (WHAM) pride!
Newsgroups: comp.infosystems.wais
From: allan@cs.cornell.edu (James Allan)
Subject: Re: Adapting to post-boolean searching
Organization: Cornell Univ. CS Dept, Ithaca NY 14853
Date: Mon, 17 Aug 1992 23:58:13 GMT

Joseph_Jackson@transarc.com writes:
>I'm not catching on to the new mode of thinking required by these
>"post-boolean" search engines!  Perhaps someone can help indoctrinate
>me in the new ways of searching.
>
[Example of how 2-word search has problems without booleans.]
>
>That teaches me that adding extra words can be detrimental to a
>search, widening it out to left field.  From what I've been reading,
>people seem to think that you can type anything at WAIS and the noise
>will automatically get stripped out by the weighting scheme and/or
>relevance feedback.  I don't see how it can help.
>
[Franklin Davis' boolean straightjacked message omitted]
>
>Wonderful concept, and I'd really like to believe it, but so far I've
>been amazed at how little I can use WAIS without some way of
>specifying simple conjunctions.  Is there another approach that I
>should be taking that WAIS will satisfy?

In your example, a boolean AND would definitely be a plus.  One might
even argue that it's a requirement.  Anytime you know the vocubulary
of the documents you're searching, booleans may very well be helpful.

In full-text searching of unrestricted subject matter, though, you
probably don't know how the vocabulary is used, so you're quite likely
to get yourself in trouble with booleans--typically by over
restricting your search with too many keywords.

The vector space model works best if your query is longer.  The more
you can describe about your information need, the more likely it is
that you'll get back the stuff you want.  Note that in the model WAIS
uses, OVER-specifying your request is unlikely to hurt you (by
discarding documents that are really close to your request, but
missing one or two words).

So instead of 'boolean search' try some more elaborate such as
'boolean search of full-text documents in a network environment such
as the Internet using software such as WAIS'.  Well, if that's what
you're interested in.... :-)
From: gilbertd@sunflower.bio.indiana.edu (Don Gilbert)
Newsgroups: comp.infosystems.wais
Subject: Re: Adapting to post-boolean searching
Date: 17 Aug 92 22:36:32 GMT
Organization: Biology, Indiana University - Bloomington

The following source has implemented a boolean AND, as well as keyword
searching.  I don't know if this based on the standard WAIS software or not.  

(:source
   :version  3
   :ip-name "ridgisd.er.usgs.gov"
   :tcp-port 210
   :database-name "/usr/opt/wais/db/esdd"
   :cost 0.00
   :cost-unit :free
   :maintainer "tgauslin@ridgisd"
   :description "Server created with WAIS release 8 b3.1 on Mar 17 09:43:46 1992    by tgauslin@ridgisd   
	[... documentation cut for brevity...]
"
)

Some simple modifications of mine a while back implemented a 
boolean NOT (as used in a Gopher server), available thru anon. ftp at
ftp.bio.indiana.edu:/util/wais/wais-bio.patch.


-- 
Don Gilbert                                     gilbert@bio.indiana.edu
biocomputing office, biology dept., indiana univ., bloomington, in 47405
Newsgroups: comp.infosystems.wais
From: jik@athena.mit.edu (Jonathan I. Kamens)
Subject: BUGFIX: (8 b5) extra argument in waislog call
Organization: Massachusetts Institute of Technology
Date: Thu, 13 Aug 1992 22:10:41 GMT

This is an extremely minor bug-fix which I stumbled upon when I was
making some other enhancements which I will be posting shortly.  I am
separating it from those other patches because it is independent of
them.

*** 1.1	1992/08/12 14:51:35
--- irinv.c	1992/08/12 23:33:41
***************
*** 528,534 ****
      if(into_version == -2)
        waislog(WLOG_MEDIUM, WLOG_INDEX,
  	      "Merging version %ld into master version and generating dictionary.",
! 	      from_version, into_version);
      else
        waislog(WLOG_MEDIUM, WLOG_INDEX,
  	      "Merging version %ld into %ld and generating dictionary.",
--- 528,534 ----
      if(into_version == -2)
        waislog(WLOG_MEDIUM, WLOG_INDEX,
  	      "Merging version %ld into master version and generating dictionary.",
! 	      from_version);
      else
        waislog(WLOG_MEDIUM, WLOG_INDEX,
  	      "Merging version %ld into %ld and generating dictionary.",
From: francois@welchgate.welch.jhu.edu (Francois Schiettecatte)
Newsgroups: comp.infosystems.wais
Subject: firewalls
Date: 20 Aug 92 16:56:39 GMT
Organization: Johns Hopkins Univ. Welch Medical Library


Below is a method for getting through network 'firewalls',
thanks go to Jonathan Goldman for the information.
( THERE IS A CAVEAT TOO !!! MAKE SURE YOU READ IT )


-------------------------------------------------------------------------
>From jonathan@philo.quake.think.com Wed Aug 19 16:22:14 1992
>Date: Wed, 19 Aug 92 13:17:10 PDT
>Sender: jonathan@philo.quake.think.com
>To: francois@welchgate.welch.jhu.edu
>Cc: tkevans@eplrx7.es.dupont.com
>Subject: firewalls
>Content-Length: 3117
>X-Lines: 96
>
>There's a little known (read: undocumented) feature in the b5 clients that
>lets you do this automatically.
>
>In ui/source.c define the symbol FORWARDER_SERVER to you gateway machine
>(which must be running the server, of course).  If the server is not
>running on port 210, you must define the FORWARDER_SERVICE to the correct
>port number.
>
>The client will then attempt to use the source description as is first, and
>if it fails to connect, it will modify the structure to use the forwarder.
>
>As I mentioned above, you must be running a WAIS server on the firewall
>machine.  You should probably insure that the server running on your
>firewall can only be connect to from inside your site (most likely using
>packet filtering).
>
>- Jonny G
>


and this caveat also from Jonathan:


>
>There's a very important caveat these people must understand - running a
>server on your firewall is a potential security risk.  If outside
>connections can be made to the server, a (very knowledgeable) hacker could
>get at internal databases using the forwarding the other way (it doesn't
>care which direction the forwarding goes).  The simplest solution is to
>disable outside connections to the server on the firewall machine.
>
>I just point this out because the firewall system adminstrator must know
>about this.
>
>- Jonny G
>


francois


Francois Schiettecatte
Software Engineer
Welch Medical Library
Johns Hopkins University
Internet: francois@library.welch.jhu.edu
Phone	: (410) 955-7581
From: skyhawk@milton.u.washington.edu (Scott Northrop)
Subject: Re: WAISified PostScript would constitute a security hole
Reply-To: skyhawk@u.washington.edu
Organization: very little
Date: Fri, 14 Aug 1992 22:52:01 GMT

>>In article <cx4mcgb.nagle@netcom.com> nagle@netcom.com (John Nagle) writes:
>>>      What's needed is the ability to start up a process with limited
>>>privileges.  UNIX doesn't have this.

>peter@taronga.com (Peter da Silva) writes:
>>Oh, I don't know about that. "chroot" seems pretty comprehensive. An
>>advantage of putting everything in the file system. If you're REALLY
>>paranoid you can setuid to a user-id with no files of its own.

In <616m1d_.nagle@netcom.com> nagle@netcom.com (John Nagle) writes:
>      "chroot" is a step in the right direction, but it itself has enough
>security implications that it can only be used by the super-user, and even
>then, some systems (with auditing turned on) disallow it.
>[...] Setting the user id to "nobody" is promising, but depends on how
>the system does that feature.  And that's still a privileged operation.
>      You want the ability to start up a subprocess with limited privileges
>without having to have super-user authority to do it.

Is a setuid-root executable that setuid()s to a safe user (like nobody) a
reasonable way to "start up a subprocess with limited privileges without having
to have super-user authority"?  I hope so, it's what I've always done.  This is
what Mr. da Silva was saying, though he wasn't being explicit about the setuid
bit.  (Of course, it's also good practice to toast the environment variables
that aren't needed -- no telling what holes you might open up otherwise.)

If necessary, a chroot() could also be done, but I have my doubts about how
much this would impede the functionality of the previewer, and how necessary it
would be in the first place.

With a chroot(), you wouldn't be able to re-read files, or specify new files to
preview without re-invoking the program.  You'd also have to make hard links or
copies into the new directory for the previewer to use.  A PostScript previewer
would, I imagine, use a great many files.  You could make hard links to the
real files if you put the directory on the same disk as all the needed files,
if they're all on the same disk.  Otherwise, you'll need a copy of everything
the previewer would need to run, including copies of all your locally-installed
fonts, etc.  And hope you've found all the necessary files!

I really think a setuid-root setuid(nobody) wrapper is sufficient.
-- 
Scott Northrop  "Don't forget to vot." <skyhawk@u.washington.edu> (206)784-2083
ObVirus:  As a juror in a Trial by Jury, you have the right, power and duty
          to acquit the defendant if you judge the law itself to be unjust.
Newsgroups: comp.infosystems.wais,alt.wais
From: tkevans@eplrx7.es.duPont.com (Tim Evans)
Subject: WAIS "passthrough" in Firewalled Network?
Reply-To: tkevans@eplrx7.es.dupont.com
Organization: DuPont Engineering Physics Laboratory
Date: Wed, 19 Aug 1992 16:57:08 GMT

I'm looking for a means of enabling users on our "firewalled"
network to access WAIS servers in the outside world.  (By
"firewalled" network, I mean that our link to the outside 
world is via a firewall system which filters out all incoming
network traffic other than SMTP; users must telnet to the
firewall system before they can access other Internet
services, such as ftp, telnet, etc.)

Nevertheless, it would be terrific if there were some sort
of "pass-through" mechanism that would allow users inside
our network to access WAIS servers outside.

Please reply via e-mail and I'll summarize.  Thanks.
-- 
Tim Evans                     |    E.I. du Pont de Nemours & Co.
tkevans@eplrx7.es.dupont.com  |    Experimental Station
(302) 695-9353/7395           |    P.O. Box 80357
EVANSTK AT A1 AT ESVAX        |    Wilmington, Delaware 19880-0357
From: Peter Smode <psmode@jeslacs.wimsey.bc.ca>
Newsgroups: comp.infosystems.wais
Subject: Wais via mail?
Date: Fri, 21 Aug 92 13:08:47 PDT
Organization: JES Library Automation, Coquitlam BC, CANADA

Dumb question this may be, but I remember a while back somebody asked about
the potential of connecting a WAIS client to a WAIS server via mail. I really
expect this to be impossible, but lately I have had some nagging recollections
of a surprising answer to the original query.

Is this possible? Replies are welcomed via e-mail.

**************************************************************************
*   Peter Smode                 E-mail:    psmode@jeslacs.wimsey.bc.ca   *
*   JES Library Automation      Voice:     (604)939-6775                 *
*   Coquitlam, BC, CANADA       Fax:       (604)939-9970                 *
**************************************************************************
From: allan@cs.cornell.edu (James Allan)
Newsgroups: comp.infosystems.wais
Subject: Re: Adapting to post-boolean searching
Date: 20 Aug 92 19:44:16 GMT
Organization: Cornell Univ. CS Dept, Ithaca NY 14853

tnnrtg@tnb.phys.tue.nl (Tonnie Geraets) writes:
>If you are able to assign a very high weight (say 1000 or 10000) to the
>first occurance of a term, then you have a simulation of a boolean AND.
>The result of a query would be a list with on top all documents containing
>all terms, followed by documents containing all terms but one, etc.
>I think this is even better then pure boolean AND.

In fact, what you have is a vector space model where terms are given
unnormalized binary weights and similarity is computed with the
standard inner product.  That is, the weight of a matching document is
the number of terms it has in common with the query.

Useful in some cases, I suspect.
From: tnnrtg@tnb.phys.tue.nl (Tonnie Geraets)
Newsgroups: comp.infosystems.wais
Subject: Re: Adapting to post-boolean searching
Date: 20 Aug 92 08:31:44 GMT
Reply-To: tnnrtg@ctrl.phys.tue.nl
Organization: Eindhoven University of Technology - NL

atherton@unixg.ubc.ca (Bruce Atherton) writes:

>  1) Allow each query term to be weighted.  If a single term seems likely
>     to swamp a search, lower its weight and increase the weight of other
>     terms more important to your search.  You could even allow for a
>     negative weight, which simulates a boolean NOT without causing such
>     drastic consequences.
[...]
>  3) Adjust the document term weight calculation so that the term frequency
>     has less of an impact.  I'm not familiar with how WAIS calculates
>     the weight, but it could be that it needs tweaking for this application.

If you are able to assign a very high weight (say 1000 or 10000) to the
first occurance of a term, then you have a simulation of a boolean AND.
The result of a query would be a list with on top all documents containing
all terms, followed by documents containing all terms but one, etc.
I think this is even better then pure boolean AND.

-Tonnie.

-- 
Tonnie Geraets                                E-mail: tonnie@ctrl.phys.tue.nl
Department of Physics     Eindhoven University of Technology, The Netherlands
----------------------------------------------------------------------------
 "Mam, wat vroeger God deed doet nu de computer, he?" - Robbie Bergmans (7)
 ("Mom, what God used to do, is now done by a computer, isn't it?")
From: wh0i+@andrew.cmu.edu (William Earl Hefley)
Newsgroups: comp.infosystems
Subject: Textbooks? (Was: Behind the Terminal)
Date: 14 Aug 92 12:58:37 GMT
Article-I.D.: andrew.keX1txK00Uh7A4yEs4
Organization: Software Engineering Institute, Carnegie Mellon, Pittsburgh, PA


This discussion about inforporating the human and organizational
perspectives into systems analysis has gotten me thinking about the
texts that we use.

What do people consider to be the best systems analysis texts?

-- for undergraduates?

-- for graduates?

-- for non-majors (undergrad or grad)?

Send me mail (wh0i@andrew.cmu.edu) and I'll summarize to the net.
From: aites@hplvec.LVLD.HP.COM (Jim Aites)
Date: Fri, 14 Aug 1992 16:38:41 GMT
Subject: Re: Starting a BBS
Organization: Hewlett-Packard Co., Loveland, CO
Newsgroups: comp.bbs.misc

| I wonder how much hard disk space do you need to start a BBS with? I plan in
| the future to join the fray of proud SysOps running a BBS. 

10 meg is (I believe) the 'suggested' bare minumum.  I'm assuming that you 
want to do a couple of megs each of files and messages.  Most installed and 
'fleshed-out' boards require around 2-5 meg just for overhead.  On the other 
hand, you *can* run a mini-bbs (limited to messages only) off of two floppies.  
I did this for a while.  However, I am current using a solid 15 meg...and 
am looking for an extra 40 meg HD. <grin>

 Truth is where you   |    Starship Artifact BBS    |     Thank you,
 find it, but it      |                             |     Jim 8')s
 helps if you keep    |        303-663-5466         | aites@hpsmpa.lvld.hp.com
 your eyes open       |                             | Surface mount: not just
 while looking...     |    c/o Foxmore de' Foghat   |     a nasty thought.