Path: cayman!psinntp!psinntp!uunet!spool.mu.edu!agate!apple!mumbo.apple.com!gallant.apple.com!wiese.apple.com!mikew
   From: mikew@apple.com (Mike Wiese)
   Newsgroups: comp.sys.mac.comm
   Date: 14 Dec 92 07:45:12 GMT
   References: <1992Dec01.233335.3935@news.mentorg.com> <1992Dec08.014323.24162@news.mentorg.com> <1fqeetINNs2j@calvin.NYU.EDU>
   Sender: news@gallant.apple.com
   Organization: Apple Computer, Inc.
   Lines: 89
   X-Useragent: Nuntius v1.1.1d15
   X-Xxdate: Sun, 13 Dec 92 07:45:07 GMT

   In article <KEN.92Dec13223109@stthomas.Cayman.COM> Ken Siegel,
   ken@Cayman.COM writes:
   >Why did Apple do compression and correction in the CPU?
   >
   [ Ken's theory deleted ]

   I have my own theory. I had nothing to do with ARA so I could be totally
   off.

   Reason #1: Simplicity

   To get better throughput using a modem's built in compression or 
   error free mode, the Mac has to send data to the modem faster than the
   actual
   data rate between the modems. So the modem has to be able to throttle the
   Mac in case errors cause transmission delays or the data isn't
   compressing much.
   This requires the user to have a cable properly wired for hardware 
   handshaking. If you put most modems into hardware handshaking mode w/o
   the proper cable, they won't be able to receive commands to get out of
   that mode, which leads to user unfriendly software.

If you are doing ARA with a 14.4K V.32bis modem then you have to set your
port speed to at least 19.2 kbps. Since you have a speed mismatch you MUST
use flow control in order to get it to work correctly so this issue does not
go away. This is also incredibly trivial to do for most modems and does not
in ANY way make the script more complicated. All modems I am aware of do not
toggle flow control lines when receiving modem command responses as well they
should not have to since these are limited by the command buffer space not
the modem character buffer. Flow control is not an issue here.

   It also makes the ARA scripts much more complicated, in that they will now
   all have to deal with the situation of the answering modem not supporting
   MNP or V.42.

The mistake was choosing V.42bis compression on top of MNP-4 in the protocol
instead of punting the whole issue like PPP and letting the modems negotiate.
It is only in the speced version of ARA that this is a problem, not with
modems.

   With ARA doing the compression, it has much more knowledge about how
   things
   are going so it can handle timeouts more easily and intelligently. With
   such
   low bandwidth connections this is critical.

Were that it were only true. In fact, check out an ARA connection in progress
some time while doing a file transfer. You will note that ATP has no notion
of round trip delay estimation so it does not adjust to line speed. The
retransmissions are truly depressing. ARA most certainly does NOT handle
timeouts intelligently. It relies on MNP to do that which would have been
a lot easier in the modem since that is the engine that is actually staring
the most likely source of problems in the face, the phone line.
FYI, higher speed modems still have an issue with timeouts. It is the 
resynching of a V.32bis connection that occurs with a change in line quality.
This can take up to 10 seconds. There is no clean way to communicate that this
is happening back to the sending entity. It just gets flow controlled if there
is a problem. Some modems do drop CD when resynching but that is ambiguous.
In short, yet another argument for letting the modem do the error correction
and compression.

   Reason #2: END-to-END data integrity

   I don't care if the data gets between the two modems OK, I care if the
   data gets between the two Macs OK. This is NOT the same thing at all.
   Since
   I'm worried about losing data between the modem and ARA's buffer in RAM,
   I'm
   going to have to do some kind of error free protocol, no matter what the
   modem is doing. SO WHY SHOULD THE MODEM WASTE TIME DOING IT TOO? I'd
   rather
   the modem used all of the connection's bandwidth to ship bits across as 
   fast as possible.

   Why am I worried about losing data between the modem and the Mac? Because
   the
   serial port hardware in most Macs has only a 3 byte buffer. If the Mac is
   busy,
   (say with MacTCP) it may not be able to read from the  buffer in time at
   38400
   bps. A Mac Plus will have problems at slower speeds, but Apple has to
   support it.

All well and good, but ARA specs a checksum on every packet that is not
sent with a 0 in the checksum field across the link. So, there is a level
of data integrity. I admit it may not be as "robust" as CRC-16 speced for MNP-4
but frankly, it is fine given the error rates and types you would experience
on a serial cable.

   I'm also not sure I want to trust every modem manufacturer to implement
   compression and error free stuff correctly. There's no certification
   authority
   like Underwriter's Labs and I don't want to be in the certification
   business.
   They've been just sending data for years, so I'm not so worried about
   that.

So, you would rather trust Apple to do the right thing with modem protocols
than US Robotics ;-)....

   Summary: To increase the odds of things working all the time, ARA does
   the compression and error correction itself. 

   I would argue that it does not take up significant CPU time on the Mac
   to do those functions, compared to the time it takes to send the data.

Agreed at low speeds that byte latency on the wire is the limiting factor.
This has nothing to do with compression and/or correction on the mac. Remember,
if you are doing a lot of dial-in ports, you have to think of the multiplying
effects of the algorithms. 

   One problem with this is that by putting the modem in "direct" mode where
   it just does the modulation thing on the bits we send, we're not making
   very efficient use of the communication channel. The modem's RS-232 port
   sends data in "async serial" format, with one start bit and one stop bit
   for every byte. So for example 9600 bps only yields 960 bytes/s, not
   1200 like you  might think.

   IF you can set your modems to NOT use direct mode, then they can talk to
   each other using whatever protocol they choose, which may have less than
   the 20% overhead that async serial has. So you set your computer to talk
   to the modem (using async serial) faster than the modems data rate and
   the modem will use hardware handshaking to throttle you. According to
   someone at Telebit, the error free mode on their modems has less than
   20% overhead, so you should be able to get more than 960 bytes/s in 9600
   bps mode. But you still have the drawbacks I listed above.

The easiest way to dump the 20% overhead is to use synchronous communications.
If the mac had real serial ports then this would be possible but it does not
so it is a mute point.

   It may be worthwhile to experiment with your ARA setup to see if you can
   improve performance by enabling error free mode, but not compression, in
   your modems. If you do get better performance, I'd like to hear from you.

   mike
--
*******************************************************************************

 Ken Siegel                                           Internet: ken@cayman.com
 Cayman Systems, Inc                                     Phone: 617/494-1999
 26 Landsdowne Street                                      FAX: 617/494-9270
 Cambridge, MA 02139                                 AppleLink: D0523
Newsgroups: comp.sys.mac.comm