Newsgroups: bit.listserv.lstsrv-l
Date:         Sun, 11 Oct 1992 01:16:11 +0100
From:         Eric Thomas <ERIC@SEARN.BITNET>
Subject:      Peered list support with 1.7e

1.7e will introduce changes to the behaviour of mailing lists, and peered
lists in particular, in order to  support messages with lines longer than
80 bytes (with the understanding that  this code cannot be activated with
the present  version of  the VM  mailer and  that it  will not  have been
tested when  1.7e is released).  It is assumed  that the new  mailer will
fold (and  not truncate) lines  longer than  80 bytes when  delivering to
"old" mailers.

When a 1.7e site with a "new" mailer  mails a copy of a message with long
lines to a  1.7e site with an  "old" mailer, the lines will  be folded by
the  mailer at  the  sending  site. Even  though  the  message will  look
approximately  the same,  its  CRC will  be different  since  it has,  in
effect,  been altered.  Furthermore the  CRC algorithm  used in  1.7d and
older versions  is not suitable  for messages  with lines longer  than 80
bytes. To avoid duplicating the work that was done with DISTRIBUTE,

- CRC's will  no longer be checked  among peers. 1.7e will  supply no CRC
  value, nor  will it attempt to  verify any CRC value  supplied by older
  servers.

- DISTRIBUTE will be used to deliver the message to the peer, so that its
  integrity can be validated by the dual-CRC system of LSVDIST2.

For similar reasons, support for "Mail-via= Direct" has been removed: all
delivery  will now  take place  through  DISTRIBUTE, which  has code  for
dealing with long lines and their integrity. LSVXMAIL no longer writes to
the punch at all.

Unfortunately, older versions of DISTRIBUTE truncate lines longer than 80
bytes.  This  is because  EXECIO  PUNCH  (or  CARD PUNCH,  or  equivalent
techniques) were  used. In fact, lines  longer than 80 bytes  were simply
not  valid as  input, since  DISTRIBUTE dealt,  by definition,  with card
images. Note that  header lines of 255 characters or  less (or any length
with 1.7d)  *are* properly  folded by  all versions,  including LISTEARN;
this is because they were sometimes generated internally and could exceed
the 80 bytes limit. The problem only  applies to the body of the message,
which means that  delivery will not fail but characters  beyond column 80
will be lost.

This is already a concern in itself, since it will affect delivery to end
users. As far as LISTSERV is  concerned, the early release of 1.7e should
make it  possible to build  a backbone of  servers which can  handle long
lines *before*  the new  mailer is actually  released. There  remains the
problem of LISTEARN,  which would have to be modified  both to fold lines
longer than  80 bytes when distributing  mail and to accept  lines longer
than 255 in job files. I will let Turgut comment on that.

This truncation is  an even more serious concern when  it comes to peered
lists. A new DISTRIBUTE option is used to order DISTRIBUTE to deliver the
CRC'ed jobs  directly to the LISTSERV  at the target node,  without going
through any  intermediate node; this  ensures that truncation  will never
occur if  the target  node is  running 1.7e,  no matter  what may  lie in
between. This  however does not address  the problem of older  servers. A
compatibility list-header option will be  provided to fold the message at
the  source for  all  recipients  (as done  in  previous versions),  thus
ensuring that messages  with long lines are never  generated and removing
the truncation  problem. It  will be  necessary to  use this  option when
peering lists  with servers not  running 1.7e, or  some of the  peers may
distribute truncated copies. That option can  also be used in cases where
lines longer than 80 are simply not desired - many workstation users tend
to forget  that there are people  with terminals that show  less than 260
characters per  line, and some mail  user interfaces do not  display long
lines in a very user-friendly way. In  fact, the support for long line is
mostly to  make it possible  to mail source  code excerpts and  the like,
rather than for actual textual messages.

This new  keyword is called  "Long-Lines= No"  and is a  no-operation for
messages without long lines. In other words,  there is no need to code it
for the transition period during which 1.7e will slowly replace 1.7d - at
least not  before the new mailer  is released. This option  is mostly for
compatibility with  LISTEARN, and  for lists where  a lot  of subscribers
cannot conveniently read lines longer than 80 bytes.

  Eric