Newsgroups: vmsnet.networks.tcp-ip.cmu-tek
Subject: RE: Long FTP file transfers slow down almost to stop
From: Mighty Firebreather <dragon@nscvax.princeton.edu>
Date: Mon, 23 Nov 1992 15:51:17 EST
Reply-To: Mighty Firebreather <dragon@nscvax.princeton.edu>
Organization: The Internet
Return-Path: <SYNFUL@CMUTEK.CC.CMU.EDU>
Errors-To: SYNFUL@CMUTEK.CC.CMU.EDU
X-Listname: CMU-TEK TCP/IP Mailing list <CMU-TEK-TCP@CMUTEK.CC.CMU.EDU>
CC: cmu-tek-tcp@cmutek.cc.cmu.edu

>	Mike LeVine <LEVINE%FIDLER.decnet@sunman.chinalake.navy.mil> writes:
>
>	I have FTP exhibiting the same problem in 2 different systems
>that I manage.
>	Configuration 1
>		Local isolated ethernet with only the following
>
>		VAX 8800 VMS 5.4 CMU-TEK 6.6_5(patched)
>		Silicon Graphics 4D/280 (1),4D/240(2),PI running IRIX 4.0.4
>		SUN sparcstation II
>		assorted DECSERVER 100/200's
>		NCD-14c X-term (1)
>
>	Configuration 2
>		Corporate network running DECNET,TCP/IP,Appletalk,etc
>		connecting via a SUN gateway to the internet
>
>		uVAX II VMS 5.4 CMU-TEK 6.6_5(patched)
>
>		Hundreds of various UNIX cpus,MAC's,VMS,etc scattered
>		over a large navy base.
>
>Symptom:
>	When doing an FTP of a large file (6-8k bytes min), the first few
>k bytes come accross fast enough, but then the rest just trickle in. Monitoring
>with SET HASH ON shows after the initial burst of 4-6 markers, nothing comes
>in for 10's of seconds, with maybe 1-4 markers and another long wait for more
>to come accross. Large files (like CMUTEKU5066.A_Z) eventually abort
>
>426-Connection closed; transfer aborted.
>426 Abort
>
>those that are able to complete give transfer rates of 100-200 bytes/sec.
>
>Is there some setup parameter or SYSGEN paramter that needs to be tweekd, or
>any hints as to where to look?
>

	I last saw this behavior with CMU-TEK 6.3.  It was tracked to its
lair by Chuck Lane and fixed long ago.  If the same thing has recurred, it
should show in the output of:

$ IPNCP NETSTAT /STAT

as an unduly large number of "Future Segments" being dropped.

	The workaround was to increase the value of Variable:FQ_MAX in
INTERNET.CONFIG.  My recollection is that a value of 10 or 15 worked well.
(YMMV) Just keep jacking it up until the number of dropped future segments
decreases to something "reasonable".

	I'm appending Chuck's original message following my signature.

*************************************************************************
*                                                                       *
*                        Here, there be dragons!                        *
*                      dragon@nscvax.princeton.edu                      *
*                                                                       *
*                                                Richard B. Gilbert     *
*************************************************************************

From:	lane@DUPHY4.Drexel.Edu 30-DEC-1988 05:16:00.00
To:	cmu-tek@DUPHY4.Drexel.Edu
CC:
Subj:	 TCP performance and FQ_MAX parameter

Date:     Thu, 29 Dec 88 14:48:39 EST
From: lane@DUPHY4.Drexel.Edu (Charles Lane)
Message-Id: <881229144837.52b@DUPHY4.Drexel.Edu>
Subject:  TCP performance and FQ_MAX parameter
To: cmu-tek@DUPHY4.Drexel.Edu
Cc: lane@DUPHY4.Drexel.Edu
Comment: Drexel University Particle Physics

I just ran into an interesting little problem in TCP tuning that
is probably a common occurance...To begin with, let me state that
I'm running V6.3 on a VMS 4.5 microvax II, and that the FQ_MAX was
set to 5 in the original INTERNET.CONFIG that was obtained from the
distribution tape.

I try FTPing something big into the microvax, and things start off
fine, but gets real slow after a while.  Some checking with NETSTAT/STAT
and NETLOG shows what's going on: the TCP packets typically have 472
bytes of data, and the IP_ACP advertises a window of maximum size 4096.

here's the packets coming in (I'll label them by packet number rather
than sequence number, since they're all the same length)

Them:   1       2       3       4       5       ....
Us:       ACK1    ACK2    ACK3    ACK4    ACK5    ....


now suppose that #1 dissappears in transit,
                                                    retransmit
T:  x   2   3   4   5   6        7   8 ...N            1          N+1  N+2...
         \___\___\___\___\___FQ   \___\___dropped       use FQ     \____\__FQ
                                 FQ full now
U:                                                      ACK1-6      ....

When 1 gets retransmitted, then packets 1-6 are delivered, but now
7-N have been dropped. After a while, 7-N will be retransmitted, but
not before more packets have been dropped. And the whole thing
repeats.  The result is an artificially `lossy' network connection.
I was seeing packet loss rates of ~30% due to this. And
sssssllllllooooowwwwww transfers.

The mechanism exists to control this: the receiving end should be able
to buffer packets up to the size of its advertised window. So when the
forward queue is full the window should be reduced. Or, the way that
I dealt with the problem: increase the FQ_MAX until the amount of data
that the forward queue can hold is a bit larger than the maximum
window size (FQ_MAX = 10 for 472 byte TCP packets) but one can see
that a relatively high rate of small packets could produce the same
effect even with FQ_MAX set to a large value.

So a word of warning to everyone out there: if you have lousy performance
and see dropped packets in NETSTAT/STAT, you should try increasing FQ_MAX.

                                        --Charles Lane
                                            lane@duphy4.drexel.edu