From: beepy@tabitha.Eng.Sun.COM (Brian Pawlowski)
Newsgroups: comp.protocols.nfs,comp.sys.sun.hardware
Subject: Re: NFS I/O Ops/seconds
Date: 22 Jul 1992 16:59:11 GMT
Organization: Sun Microsystems Inc., Mountain View, CA
NNTP-Posting-Host: tabitha
Summary: nhfsstone PRE-LADDIS iops nfs

In article <1992Jul22.061146.15641@u.washington.edu>, kint@rio.engr.washington.edu (Rick Kint) writes:
> The Sun rep commented that a typical Ethernet allows 300 NFS ops per second,
> so if you're on a single wire any server bandwidth beyond that is wasted. 
> 
> 	I have no feel whatsoever for the numbers that go into this figure,
> can anyone in netland (a) explain this in words of two syllables or less, and
> (b) does anyone know the figures for current systems?

I decided to post this in addition to mailing. I have
far exceeded two syllables. Sorry.

The number "300 NFS ops/s per ether" is a rule-of-thumb
(that's not quite the right word) maximum NFS ops/s as
generated by nhfsstone (Legato) or PRE-LADDIS (SPEC)
benchmark. I'll assume an equivalence of "NFS ops/s"
and "NFS I/O ops/s". [Is there a difference?]

It assumes a mix of which the more interesting features
are:

	34% Lookup
	22% Read
	15% Write
	13% Getattr

And something like 85% of the write's are 8KB, the amount
of data touched is in excess of 15MB (dependent on number
of clients). Couple hundred files, etc. All having implications
on effectiveness of caching under the presented load.

I'm truly generalizing above, by putting nhfsstone and
PRE-LADDIS in same breath. From a 1,000 feet they're similar 
enough to generalize about.

The "300 ops/s" number is approaching ethernet maximum
utilization. For nhfsstone from Legato, I have faint
recollections of this being in excess of 50-60% ethernet
utilization with peaks going higher (as shown on a
Sniffer in monitor mode).

nhfsstone is the original benchmark from Legato. SPEC is
evaluating a derivative benchmark called PRE-LADDIS. If
you were going to get serious about benchmarking and comparisons,
then you have to be precise about which benchmark
you're talking about. On one recent version of PRE-LADDIS
(each version has been changing during the "beata test",
recent changes have been significant to load characteristics)
I saw about 330 ops/s on an ether. Someone told me they
saw 340 ops/s on an ether. At this point you have very
busy ethernet.

What do you do with this information? That's the hard part.
How closely does your network resemble the PRE-LADDIS 
workload model? Probably not very. If you assume it
does, and your load peaks are 300 and below, you can
convince yourself that you can get by with a server
with capacity to handle 300 ops/s and one ethernet
interface (no need to segment net and have multiple
interfaces to feed server). But doing an analysis of your
network is not simple, and applying nhfsstone results
to draw conclusions for your workload is not trivial
(that is my personal opinion).

Both nhfsstone, and more so PRE-LADDIS, are configurable and allow 
you to simulate other workloads. So for the serious
investigator, you can conduct experiments to compare
different server configurations for your specific workload.
After you have characterized your workload to your satisfaction
such that you feel your conclusions will be valid.

I'm not sure I answered your "why 300 ops/s question" yet:-)

It's just an observed point on or about which ethernet
approaches maximum utilization when subjected to a load
generated by PRE-LADDIS or nhfsstone (Legato).

Both benchmarks are available (the PRE-LADDIS one is an
evaluation version from SPEC with restrictions on use of
data, etc). In the last InterOp proceedings there are
slides describing LADDIS work which could explain some
things. 

You asked for numbers for various servers... Can't do that,
and the PRE-LADDIS beta evaluation agreement specifically
precludes public disclosure of results.

Brian Pawlowski

Xref: sparky comp.protocols.nfs:1956 comp.protocols.tcp-ip.ibmpc:4430