Return-Path: <ERIC@SEARN.SUNET.SE>
Newsgroups: bit.listserv.lstsrv-l
Date:         Thu, 7 Jan 1993 17:43:09 +0100
From:         Eric Thomas <ERIC@SEARN.BITNET>
Subject:      Clarifications about the digest support
Comments: To: LSTOWN-L@INDYCMS.BITNET

As  I  expected there  has  been  a flood  of  requests  for changes  and
improvements to the digest support totalling  pretty close to 3 months of
work. This  was in fact one  of the main  reasons I had stayed  away from
this (conceptually  simple) new function  for years. I just  can't answer
everyone privately, especially  as many suggestions are  common. I'd like
to clarify what the  intent of the digest support is and  what it is not,
so we can all save time on this.

The reason  for implementing automatic digest  support is not to  make it
technically possible for BITNET lists  to have digested versions - people
have been  doing this for  years. The only purpose  is to allow  users on
high-volume lists to ask for a digest version of the list without placing
a burden on the list owner. The list owner makes a one-time change to the
list header (if  the default option is not satisfactory),  and that's it.
The users get all  the traffic for a particular period  in a single file,
and read  it at their leisure.  The function must remain  simple, so that
the list owner  does not get confused  and does not have to  be afraid to
enable it.

Most of the suggestions are about ways  to allow the list owner to review
and edit the contents of the digest, or to describe to LISTSERV the tasks
he wants to  do - automatically justify messages, format  everything on a
certain amount of columns, remove signature files, and so on. I have been
mailed  descriptions of  all sorts  of complicated  commands which  would
allow the  list owner to interact  with LISTSERV and make  changes to the
digest. I am sorry,  but this is never going to fly. If  you want to edit
your digest, it is  much easier for both LISTSERV and  you to just create
the digest manually from the messages  you are getting from the list than
to use complicated commands to extract the posting Joe made on the 5th of
january with  subject containing  the text  "foo", edit  it, and  send it
back. All you have to do is ask your mail program to write the message to
a certain  folder/notebook, and then edit  that file as you  see fit with
the comfort of your preferred editor.  There are tools available for most
operating systems to  assist you in making a manual  digest, and LISTSERV
is simply not going to become one of them.

Similarly,  a digest  is not  a moderation  tool -  at least  not if  the
non-digested version is available, which  is the case with LISTSERV-built
digests. If  you want to prevent  offensive messages from showing  in the
digest, moderate your list. People who are not getting the digest version
will  see  the offensive  message  anyway,  and  users may  refrain  from
subscribing to the digest if you  are censoring postings. I agree that it
would be useful to  remove messages such as "please add  me to the list",
but they  are pretty rare and,  in my experience, most  list owners don't
bother to remove  them from the list  archives. I don't want  to have 500
lines of code to generate the digests and 2000 lines of code to implement
fancy commands which  are going to be  used maybe once a  month. And this
also  means I  haven't  got the  time to  implement  a scheduling  system
allowing the owner to  request that the digest be cut  on mondays at 7am,
except it should be done on tuesday  if the monday is a holiday, and then
on thursdays at 11am, except that if the thursday falls in a new month it
should be done  on the 1st instead,  or the following working  day if the
1st is  a holiday. We're  not running a bank  but a computer  network, if
monday  is a  holiday  people will  find  the file  in  their mailbox  on
tuesday, or if the machine is shut down on monday the digest will be sent
on tuesday just as you wanted, and  if you have this sort of requirements
you can make your digest yourself, my goal isn't to solve the 1% of weird
requirements but the other 99%.

A lot of suggestions were made regarding the frequency. I understand that
for a small number  of very active lists, a daily digest  is likely to be
very large, but I  don't think the solution is to ask  for a digest every
12 hours,  or even  8 hours  (most postings will  be made  during working
hours, so  you'd get 2  empty digests and a  big one). I  also understand
that  a very  small number  of lists  generate so  little traffic  that a
yearly digest  is better,  in which  case the  list owner  can go  to the
excruciatingly painful trouble  of extracting the yearly  notebook once a
year and  posting it  to the  list. Besides, as  a LISTSERV  maintainer I
refuse to  create lists  with yearly notebooks,  as the  situation always
gets out of hand  after 3 months in spite of all  assurances by the owner
that it would be  very low volume. I will not do  anything that may cause
LISTSERV to mail gigantic digests at the end of the year.

I agree however that it might be useful  to be able to say "don't let the
digest grow  beyond nnnnn lines".  This would  solve the problem  of high
volume daily digests and  that of lists which tend to  work in bursts, so
I'll implement it. On  the other hand I don't think this  is a useful way
to specify the digest frequency (the  proposal was to say "max size 1000"
as a replacement  for "weekly"). If a  "burst" dies, there will  be a few
"last messages"  in the  digest which  everyone will  not have  seen. Why
should people have to  wait until the beginning of the  next burst to see
these messages? Chances are, in fact, that it will restart the discussion
at that point! And you would have users posting garbage just to cause the
digest to be cut.  I see a lot of problems and very  little to be gained,
so this will be something in addition to the frequency.

What I refuse to do however is  to implement an option that says "cut the
digests as usual, but break them into messages of no more than 1390 lines
or 25k,  whichever is smaller, because  that is all my  macintosh gateway
will let through".  As I've said many  times, this is a  problem with the
gateway policy, not with LISTSERV. It affects you whenever you attempt to
order any  document exceeding 1390  lines or  25k, and that  includes not
only digests but  documentation, CREN/EARN documents (such  as minutes of
meetings  or proposals)  which can  be rather  long, the  output of  many
LISTSERV commands such  as LIST GLOBAL, and  so on. What would  you do if
your internal mail service threw  away all envelopes containing more than
3 pages? Would  you complain until they stopped doing  that, or would you
instead  ask  your correspondents  to  please  mail  this contract  in  5
envelopes of 3  pages each? Why is  it you accept things  from people who
run computers that you'd never  take from people running other real-world
services? It is totally unreasonable to expect the world to start cutting
everything in pieces  of 1390 lines or 25k, whichever  is smaller, simply
because of a handful of sites  which can't handle more. By helping people
get past this limitation, I would  only decrease the amount of complaints
sent to  the people who  are responsible for this  ridiculous limitation,
and it  would prevent  the people  complaining from  saying "but  I can't
access the list of lists and get my work done!", because you'd have a way
to get it in  25 tiny pieces. And of course, as a  user of a working mail
system I don't want to have all the messages I get cut into pieces of 25k
which I then  have to reassemble manually just because  there are 3 users
on the list who don't complain loud enough about their mail system. Last,
but not least, I am not responsible for this limitation and don't see why
I should  waste my time  getting around it.  This just is  someone else's
problem.

  Eric
Comments: Gated by NETNEWS@AUVM.AMERICAN.EDU