From: janet@cs.uwa.oz.au (Janet Jackson)
Subject: SUMMARY: HELP - how to get adequate cross-campus NFS performance?
Organization: Dept. Computer Science, University of Western Australia.
Date: Tue, 7 Apr 1992 03:46:30 GMT

[My original posting is quoted below.]

We have managed to fix this by setting rsize=4096,wsize=4096,timeo=10.
The secret to getting these values to actually work is to unmount and
remount the filesystem.  Using "-o remount,..." doesn't work.

Thanks to the various people who responded.  The main suggestion was
that PC-Route may not be up to the task, and that you need at least
a 12MHx 286 with 16-bit Ethernet card.  We have the former but not
the latter but it seems to be working OK now.

Janet Jackson
<janet@cs.uwa.edu.au>
Systems Administrator
Department of Computer Science
The University of Western Australia

>Situation:  two subnets hanging off our campus LAN, with probably about
>700m of cable in between.
>
>sun1-|                                                     |-sun2
>     |                                                     |
>     |-parkgate                                   wacsgate-|
>         |                                           |
>====================================================================
>
>sun1 and sun2 are Sun SPARCs running SunOS4.1.1.  (These are not our
>real hostnames, just a generic example.  We actually have lots of Suns
>all behaving the same.)
>
>parkgate and wacsgate are PCs running PC-Route.
>
> |
> |   are our subnets (Thin ethernet)
> |
>
>======= is the campus lan (mostly fibre optic but some slower stuff
>        I am told, plus various repeaters.  I don't do the cabling.)
>
>We are NFS mounting between sun1 and sun2.  However the performance is
>appalling.  Lookups are OK, but try to read or write a moderately sized
>file (24K is the smallest I've seen cause problems) and you get repeated
>messages of the form:
>
>NFS server sun2 not responding still trying
>NFS server sun2 OK
>NFS server sun2 not responding still trying
>NFS server sun2 OK
>
>indicating that major timeouts are occurring.  It doesn't hang forever - 
>eventually it manages the read or write.  What surprises me most
>is that the 'OK' message always comes immediately after the 'still trying'
>message.  The pairs of messages come at roughly 20-sec intervals - no matter
>what I set the NFS timeout period to.
>
>The NFS retransmission rate (timeout/calls) can get up to about 50%.  When
>not going through the campus LAN it's more like 0.05%.
>
>% nfsstat -rc
>
>Client rpc:
>calls    badcalls retrans  badxid   timeout  wait     newcred  timers
>3553     510      0        1        509      0        0        214   
>
>(Just now the retrans. rate is only about 14%; but I can make it far worse
>by not NFSing anything from the local subnet.  But note the 'retrans' value,
>which is always 0 - is this a bug or should it be telling me something?
>Also note 'badxid' is very low, which indicates that the servers are fast
>enough when the packets actually do get through.)
>
>Everything except NFS (rcp, for example) works beautifully.
>'ping' turnaround time between the two nets is 2 to 10 milliseconds.
>
>I used 'spray' to check for packets being dropped somewhere between the
>two Suns.  THEY ARE!  For packet sizes up to about 200 bytes, almost all
>get through.  For sizes around 512 bytes (the minimum NFS packet size,
>I believe) about 35% get dropped.  For sizes around 1K, about 65% get dropped.
>3K - 75%.  and so on up to 8K (the default NFS packet size) where almost
>100% get dropped.
>
>To cope with this, I've tried:
>
>- changing the NFS timeout value to all sorts of numbers.  The default 7ms
>actually seems best (ie, sometimes you only get one pair of messages)
>but performance is still unacceptable.
>
>- killing biod to make NFS requests single-threaded.  This doesn't seem
>to help.
>
>- trying to reduce the rsize and wsize (NFS buffer sizes).  However SunOS
>has dynamic resizing, which keeps increasing the size again,
>and I can't find any way to turn it off (perhaps in the kernel config?).
>SunOS seems to mostly settle for 3K read buffer and 512 byte write buffer,
>but sometimes it picks larger sizes.  Doesn't seem to help at all, anyway.
>
>Shall I give up and blame the hardware,  or is there more NFS tuning I
>can try?
>
>I'll be most grateful for any help.
>
>Please reply by email and if there's interest, I'll summarise.
>
>Janet Jackson
><janet@cs.uwa.edu.au>
>Systems Administrator
>Department of Computer Science
>The University of Western Australia
Newsgroups: comp.protocols.nfs