From: caadams@triton.unm.edu (Clifford A Adams)
Subject: STRN (Scoring/ScreenScan Threaded ReadNews) ALPHA testing.
Date: Mon, 16 Nov 92 12:01:09 GMT
Organization: University of New Mexico, Albuquerque
Summary: Newsreader modifications need major testing.
Keywords: STRN, USENET, sources, testing, filtering, prioritizing

	The STRN (Scoring/ScreenScan Threaded ReadNews) program will be the
first software release of the USENET Moderation Project (see below).
It consists of two major (and some minor) additions to the TRN program
(version 2.2) by Wayne Davison (which is in turn based on RN version
4.4.2 by Stan Barber (based on RN 4.3 by Larry Wall)).  These
additions are in the form of approx. 4500 lines of C in new files,
plus about 500 lines of changes/additions to the original TRN files.
In short, these changes add a new interactive article selection mode
and a fast method of "scoring" articles with features similar to
KILLfiles. 

	I am looking for ALPHA testers for STRN.  While most of the features
I plan to implement are present, several more need to be added before
public release.  STRN has been my newsreader over the past few months,
and has been tested by a few local users for several weeks.  Some
features are crudely implemented such as the startup options (which
currently use environment variables, but should later use .rc files).
Documentation is crude (mostly short explanations of commands) and
needs expansion.  Lots of things will change before STRN is officially
released (current target is March '93).  I need more testers to give
feedback and suggestions (more code would be appreciated too :-).

	Testers will have to get a copy of the TRN source code with
patches up to version 2.2 (the current released version).
Instructions for doing so by FTP are included.

Here is my current plan for testing:
* Now-early December:  Early Alpha testing.
	Correct the worst features and compatibility problems.
Probably at least 2 patches/week.  Most of the corrections will be
bugs and compatibility corrections.  I hope to keep this group fairly
small (probably < 10 people) for easier co-ordination.
* December-late January: Late Alpha testing.
	Add new features requested by alpha testers.  Correct more
bugs and compatibility problems.  Make configuration and usage easier.
Larger group of testers.
* Feb-release: Beta testing.
	Like "alpha" but called "beta" :-).  Mostly polishing.  Even
wider release.

	If anyone would like to help test this code, please send email
to me (caadams@triton.unm.edu).  If you are interested, but not
certain about testing, mail me and I can send more information.  I
only have one request for the testers: please do not release the
alpha/beta versions beyond your own control.  If you cannot apply new
patches to the testing versions, please delete it--I do NOT want to
get a lot of bug-reports about obsolete testing versions.

	If you have any questions or comments just send me email.
Here is more information about ScreenScan, Scoring, and the
USENET Moderation Project:

ScreenScan
----------
	ScreenScan (SS) mode is an interactive article selection mode.
It has many features similar to TRN's thread selector, but does not
require TRN's thread database.  SS can use (almost) any size screen,
and will adjust to window resizings.  Articles can be displayed either
individually, or in "threads" (articles with a common subject).
Articles may be read individually or several may marked to read
sequentially.  SS may be used to select individual articles from TRN
thread trees.  SS may be entered from almost any TRN mode by typing
';' (which has led some people to call it the "semicolon" mode).

Sample ScreenScan display
[author display turned off to protect the guilty :-]:
---------- cut ----------
sci.physics | 790 unread Fold
+x...  [  18] {  9} > Some physics questions
+x...  [  16] {  8} > NEWS: Galileo Cleared of Heresy Charges by Vatican
+x...  [  16] {  3} > Scientists Plan to Blow Up the World!
+x...> [  14] { 10} > The size of electrons, and Fanciful misc SAGA
+x...  [  11] {  4} > Massive Photons Tomorrow (was Scientists Plan...)
+x...  [   8] { 47} > NASA Coverup
                    Keys: Moon Neutral Point calculation
+x...  [   7] {  8} > Gravitation & massless particles (was Re: Some physics qu
+x...  [   3] {  5} > Question on Hawking radiation
                    Summary: Why don't extremal black holes radiate?
+x...  [   3] { 10} > Background Radiation and Olber's Paradox
+x...  [   3] {  2} > Feynman 24 Path Integrals of Action = Probability Amplitu
+x...  [   3] {  1} > What is Omega_o?
+o...  [   1] {  1} Covariant vs. Lie Derivative in Gen. Rel.?
+o...  [   1] {  1} A Phase Space "Slide Rule"
(score (new articles first)  order, 100% scored)
---------- cut ----------
[The top and bottom lines are displayed in STANDOUT mode if the
 terminal has that capacity.]
...ScreenScan has many more features, and more are planned for the
future.

Scoring + Scorefiles
--------------------
	The new Scoring system in STRN allows articles to be given a
score, then sorted in score-decreasing order when presented in
ScreenScan.  Articles may be junked ("killed", marked-as-read) if they
fall below a specified threshold.  On machines which support the
(input-)PENDING option, scoring can take place while the user reads
articles (No more long waits "Looking for articles to kill...")
	Scorefiles are intended as a *fast* replacement for the older
KILLfiles (which are still supported in STRN).  Scorefiles consist of
simple commands and score patterns.  A short scorefile might look like this:
---------- cut ----------
##### sci.physics score file
### '#' is a comment character
# I don't want to read anything which scores <= -10
killthreshold -10
# Include the scores from the "science" file
include ~/News/scores/science
#Someone not worth reading:
-15 From: luser@nowhere.org (J. Random. Luser)
# A great physicist:
40 from: wizard@nobel.winning.com
# I'm tired of this subject:
-30 Subject: Is the Earth Really Round?
# ...But I like this one:
10 Subject: Fascinating New Discoveries in Quantum Chromatography
#Kill xposts to alt.sci.physics.weird
# (unless they are posted by high-scoring authors)
-20 Newsgroups: alt.sci.physics.weird
---------- cut ----------

	A scoring pattern consists of a score, a header name, and text
to look for in the header.  (TRN-style patterns are allowed.)  Each
article is compared to every pattern, and if they match, the score is
added to the individual article's score.  For example, given the
sample file above, an article posted by "wizard@nobel.winning.com"
with the subject of "Is the Earth Really Round" would have a score
total of (40 + -30 =) 10.
	Global, local, and hierarchy ("sci.*") scorefiles are
supported.  "Interest" scorefiles may also be created, and included
from within other scorefiles.  (For instance, a "science" scorefile.)
Scorefiles may be (primitively) edited while reading,

	Scorefiles are fast.  Here is a timing test done with an NNTP
connection to a local newsserver:

[Both server and client are lightly loaded.]
248 unread articles in talk.religion.misc
KILLfile (single KILL line: "/fooxxxxxxxxxxx/h:j")
(88,84,83 seconds: average of 85 seconds)
KILLfile (two lines: "/fooxxxxxxxxxxx/h:j" "/baryyyyyyyyyyy/h:j")
(180,176,175 seconds: average of 177 seconds)
Scorefile (8 files, 400+ different score entries)
(75,67,94 seconds: average of 78 seconds)
[Note: one can read articles while scoring occurs, but scoring will
 take slightly longer.]

NOTE: Scoring and ScreenScan are independent modifications and can
each work without the other (although Scoring without ScreenScan is
only useful as a simple KILLfile replacement).

USENET Moderation Project:
-------------------------
	The purpose of the USENET Moderation Project (UMP) is to
identify and implement methods of dealing with "human bandwidth"
problems of USENET.  Its goal is to allow users to be presented with
the articles they wish to read in the order they wish to read them,
without restricting the posters of articles.  There are currently three
stages planned:

    1. Filtering/Prioritizing
	Given the articles which currently exist, delete the articles
which the user is unlikely to read, and present the remaining articles
in the order of their priority (score).  This builds on and enhances
the currently existing mechanisms such as separate hierarchical
newsgroups and KILLfiles.

    2. Suggestion/Addition
	Adds more information to make the filtering/prioritizing
process more accurate (accuracy is measured by how well the presented
order matches the users wishes).  Rating services may replace
moderated newsgroups, as individuals decide which rating services to
follow and what level of quality is acceptable.  Summarizing and
keywording services could also be made available to allow better
selection and prioritizing.  This stage will introduce the problems of
distributing the additional information and security (especially since
the reputation of moderators may become much more important).

[STRN is intended to test certain ideas in the first and second stages.]

    3. Alteration/Change
	The basic idea of this stage is that what the contents of the
original posting are only a suggestion of what the reader will
eventually view.  While the original text would be propagated
unaltered, various people would propagate their suggested editorial
changes (such as hypertext links, spelling or factual corrections,
moderator's notes, or summaries).  What a reader finally views would
be the combination of the original text with the changes of trusted
editors.  Another possibility at this stage would be the "virtual
newsgroup" which is simply a collection of articles chosen by
moderator(s).

--------------------

	Again, I'm looking for ALPHA testers, but I appreciate
comments from anyone.  The first patch kit should be mailed out
sometime this week to those who request it.  I can mail the current
documentation to those who are interested in the progress of strn, or
I can put you on a list of interested persons who want updates.

				--Cliff
-- 
Clifford A. Adams --- "Understanding is inasmuch as becoming."
caadams@triton.unm.edu | 457 Ash St. NE/Albuquerque, NM 87106 | (505) 242-4519
Programmer in the USENET Moderation Project: tools for advanced filtering
(many-moderator groups, keyword/summary addition, prioritizing newsreaders...)