From: peterd@pjd.dev.cdx.mot.com (Peter Desnoyers)
Subject: Re: NFS packet size
Organization: Motorola Codex, Canton, Massachusetts
Date: Mon, 4 Jan 1993 18:03:04 GMT

vjs@rhyolite.wpd.sgi.com (Vernon Schryver) writes:

>In article <1993Jan1.115015.17311@prl.dec.com>, boyd@prl.dec.com (Boyd Roberts) writes:
>> ...
>> > A dropped TCP fragment can result in a large amount of resending too.
>> 
>> True, but that all happens at the TCP/IP level which is far more
>> responsive than NFS's 1, 2, 4, 8 ... _second_ exponential backoff
>> when a datagram gets lost.

>In other implementations of NFS, the NFS timeout starts at 0.7 seconds
>not 1.0, and can be reduced to 0.1 seconds.  Please compare 0.1 seconds
>with the standard TCP timeouts.

It still doesn't compare very well, at least on paper. With one second
timeouts and 8K blocks, exponential retransmission ala NFS falls off
to 50% throughput at an Ethernet packet loss rate of 0.001, and to 25%
throughput at a loss rate of 0.004. With a 100 mS starting timeout,
throughput falls to 50% at a loss rate of 0.01, and 25% at 0.025 or
so. (all figures from eyeballing gnuplot graphs...)

With TCP on a local net, however, the retransmission timer should
adapt to somewhere in the range of say 3 to 6 packet times (correct me
if I'm wrong...), and so in the worst case (loss rate 0.025, timer = 6
packet times) the throughput would be about 87%.

>> We had a flakey drop cable here a while back.  When it was bad it would
>> take in the order of a minute to write an 8K block.  You could see the
>> fragments being dropped and the NFS retries.  I _never_ want to see
>> that again -- it was _awful_.

>It could have been as bad or worse if you had been using NFS or TCP.
>Your TCP connections would have been timing-out and being recreated,
>throwing away any successfully moved data, and forcing the systems to
>attempt the extra packet exchanges that are required to start a TCP
>connection, incurring even more grief.

If you lose a segment in TCP, you only need to get one frame through
correctly for the connection to make progress, while for NFS with 8K
blocks you need to get 6 frames through. In fact, a year or two ago we
had some network problems that killed NFS, but never dropped telnet
connections or X sessions. (in fact, you didn't really notice a
problem with TCP services - FTP performance was only slightly
degraded, while NFS throughput was terrible. )

It looks like there are several explanations for this:

- NFS works a block at a time, and so has an effective error rate
about six times higher (for 8K blocks) than TCP, since the entire
block gets retransmitted if any packet is in error. In particular, TCP
only needs to get the first packet of the window through in order to
keep from resetting the connection.

- Even if NFS didn't retransmit much more data than TCP (it does, due
to the larger effective packet size) it also spends most of its time
idle under error conditions, while TCP only waits a few round-trip
times (i.e. 10-20 mS) to retransmit.

- Even through TCP may have a large maximum window size (e.g. 60K), on
a LAN (with <<1 packet time latency) the actual window size is very
small, as is the adaptive retransmission timer.

				Peter Desnoyers
--