From: gsa@easyaspi.udev.cdc.com (gary s anderson)
Newsgroups: comp.protocols.nfs
Subject: Re: NFS packet size
Date: 5 Jan 93 16:34:17 GMT
Reply-To: gsa@easyaspi.udev.cdc.com (gary s anderson)

In article <peterd.726170584@pjd.dev.cdx.mot.com>, peterd@pjd.dev.cdx.mot.com (Peter Desnoyers) writes:
|> vjs@rhyolite.wpd.sgi.com (Vernon Schryver) writes:
|> 
|> >In article <1993Jan1.115015.17311@prl.dec.com>, boyd@prl.dec.com (Boyd Roberts) writes:
|> >> ...
|> >> > A dropped TCP fragment can result in a large amount of resending too.
|> >> 
|> >> True, but that all happens at the TCP/IP level which is far more
|> >> responsive than NFS's 1, 2, 4, 8 ... _second_ exponential backoff
|> >> when a datagram gets lost.
|> 
|> >In other implementations of NFS, the NFS timeout starts at 0.7 seconds
|> >not 1.0, and can be reduced to 0.1 seconds.  Please compare 0.1 seconds
|> >with the standard TCP timeouts.
|> 
|> It still doesn't compare very well, at least on paper. With one second
|> timeouts and 8K blocks, exponential retransmission ala NFS falls off
|> to 50% throughput at an Ethernet packet loss rate of 0.001, and to 25%
|> throughput at a loss rate of 0.004. With a 100 mS starting timeout,
|> throughput falls to 50% at a loss rate of 0.01, and 25% at 0.025 or
|> so. (all figures from eyeballing gnuplot graphs...)
|> 
|> With TCP on a local net, however, the retransmission timer should
|> adapt to somewhere in the range of say 3 to 6 packet times (correct me
|> if I'm wrong...), and so in the worst case (loss rate 0.025, timer = 6
|> packet times) the throughput would be about 87%.
|> 
|> >> We had a flakey drop cable here a while back.  When it was bad it would
|> >> take in the order of a minute to write an 8K block.  You could see the
|> >> fragments being dropped and the NFS retries.  I _never_ want to see
|> >> that again -- it was _awful_.
|> 
|> >It could have been as bad or worse if you had been using NFS or TCP.
|> >Your TCP connections would have been timing-out and being recreated,
|> >throwing away any successfully moved data, and forcing the systems to
|> >attempt the extra packet exchanges that are required to start a TCP
|> >connection, incurring even more grief.
|> 
|> If you lose a segment in TCP, you only need to get one frame through
|> correctly for the connection to make progress, while for NFS with 8K
|> blocks you need to get 6 frames through. In fact, a year or two ago we
|> had some network problems that killed NFS, but never dropped telnet
|> connections or X sessions. (in fact, you didn't really notice a
|> problem with TCP services - FTP performance was only slightly
|> degraded, while NFS throughput was terrible. )
|> 
|> It looks like there are several explanations for this:
|> 
|> - NFS works a block at a time, and so has an effective error rate
|> about six times higher (for 8K blocks) than TCP, since the entire
|> block gets retransmitted if any packet is in error. In particular, TCP
|> only needs to get the first packet of the window through in order to
|> keep from resetting the connection.
|> 
|> - Even if NFS didn't retransmit much more data than TCP (it does, due
|> to the larger effective packet size) it also spends most of its time
|> idle under error conditions, while TCP only waits a few round-trip
|> times (i.e. 10-20 mS) to retransmit.

I'm not sure about your version of TCP, but 1 sec tends to be a
nominal value for determining when to retransmit.  You must also
accommodate "delayed acks" (200 msecs comes to mind for this value)
when determining when to retransmit.

You will also discover that NFS has an interesting delay pattern when
using TCP because of "delayed acks" (i.e. the last packet
is not a full MSS).  Disabling Nagle's algorithm does eliminate this delay
(some implementations allow you to do this).

I guess you can turn off "delayed ACKS" and set timers to fire every few 
milli-seconds in order to get the behavior you just described.  My 
question:  Is this what your code is doing and, if so, is this part
of a recent BSD release??

|> 
|> - Even through TCP may have a large maximum window size (e.g. 60K), on
|> a LAN (with <<1 packet time latency) the actual window size is very
|> small, as is the adaptive retransmission timer.
|> 
|> 				Peter Desnoyers
|> -- 

If packet loss is a major problem in a net, one can lower the 
rsize/wsize to a value less than the MTU size in order to minimize the 
retransmission impact.

Finally, I thought the, NFS over TCP versus UDP, differences had been debated
enough times in this news group to satiate most individuals.  I believe
most (if not all) believe that TCP is generally superior in WAN's and
lossy networks and that UDP works "good enough" in most LAN environments.
Is there some major point that needs rehashing?