From: coolidge@speaker.wpd.sgi.com (Don Coolidge)
Subject: Re: NFS I/O Ops/seconds
Organization: Silicon Graphics, Inc.
Distribution: usa
Date: Wed, 22 Jul 92 22:29:57 GMT

Seems like some summary is in order.

1) The standard NFS benchmark is nhfsstones. It will probably
soon be replaced by LADDIS. Both are similar; in fact, the
default mix of operations used by each is (at least was) the
same. Reads/writes account for less than 35% of the mix.

2) Sun (among others) has from time to time reported very
high IOPS numbers by changing the mix to have even fewer reads
and writes. This is not dishonest...it just means that the
reporting requirements for the test should make some mention
of the mix.

3) Auspex has gotten well over 1000 IOPS with the standard
nhfsstones operations mix, but that was indeed with
4 connected Ethernets, and using their own NFS load generator,
not nhfsstones (they may well have also gotten wonderful numbers
with nhfsstones, but I haven't seen any).

4) LADDIS reccommends adding another Ethernet for every 200 IOPS.

5) However, that's not really necessary...our own tests have shown
that nhfsstones are at least as client-bound as they are server-bound
and media-bound. We've seen well over 400 IOPS over a single Ethernet
using nhfsstones with the default mix and with UDP checksumming
enabled, between a Crimson server and a single Crimson client. We
also see well over 500 IOPS with the same setup, but with two
Crimson clients on the single Ethernet. Multiple Ethernets
give an additional boost. So does use of FDDI instead of Ethernet
(a big boost).

6) To be able to make reasonable NFS performance comparisons, I
think that at least the following need to be specified in the
performance report (there may well be more; this is just off the
top of my head):

	- Network topology (number of nets; medium being used; 
			    number of clients per attached net)

	- Server:	Number and clock speed of CPUs
			Medium Interface model and revision
			O/S version
			Number, kind, and size of disks attached
			Is there an NFS accelerator, either external
			   or intrinsic?
			Any other specialized hardware?
			How much RAM?

	- Clients:	Number and clock speed of CPUs
			Medium Interface Board model and revision
			O/S version
			Any acceleration onboard?
			Any specialized hardware?
			How much RAM?

	- General:	UDP checksumming enabled?
			Mix of operations used
			Benchmark - nhfsstones, LADDIS, other?

I've seen too many bogus comparisons made where The Competition's
server was tested with underpowered, under-RAMed clients, or where
different versions of the O/S were used on different clients, or
where the mix was different, or the test, or...

LADDIS is supposed to allow us to compare apples and apples. But it
seems to have gotten a tad too political to do so, as many of the
items I mentioned above were suggested and rejected as part of
the output report. Too bad.

Anyhow, if you want to be absolutely sure of your vendor 
performance comparisons, run the tests yourself. There's
just too much information that gets left out of reports -
for all practical purposes, any such marketing performance
reports are worthless.

Don Coolidge
coolidge@speaker.wpd.sgi.com
 
Xref: sparky comp.protocols.nfs:1988 comp.protocols.tcp-ip.ibmpc:4543