From: beepy@ennoyab.Eng.Sun.COM (Brian Pawlowski)
Newsgroups: comp.protocols.nfs
Subject: Re: NFS I/O Ops/seconds
Date: 26 Jul 1992 08:45:47 GMT
Organization: Sun Microsystems Inc., Mountain View, CA
Distribution: usa
NNTP-Posting-Host: ennoyab
Summary: nfs nhfsstone laddis

[A public follow-up].

I agree with Rick Jones. My take on the various status I've seen from
both Sun and other vendor reps is that most of what was mentioned for
reporting rules in Coolidge's posting is not a problem.

Reporting rules are simple: You must report all configuration details
which affect NFS server performance. SPEC is nailing down specifically
for NFS what the reporting rules are, so there is not misunderstanding,
along with guidelines--I believe--for vendors who may have non-standard
configurations. The August SPEC meeting in Lisle will continue this
work.

"Run rules" are more contentious. Reporting rules are where you get to
say how you set up and ran the tests. Run rules, on the other hand, are
the "rules" you must adhere to when running the tests. This has been
the source for alternating lively and heated discussion. I have
disagreements with counterparts. Variety is the spice of life.

Also, to my highly informed knowledge, Sun has never engaged in
reporting numbers for nhfsstone and PRE-LADDIS where we have modified
the mix. Some field people may have done this in an attempt to match a
customer's measured mix (that's why the knobs were put in in the first
place).

Different versions of nhfsstone under construction (*before* it was
called PRE-LADDIS) where the benchmark load characteristics varied from
release to release has confused some prior reporting. That's why the
LADDIS guys and SPEC got a little rigid regarding reporting results
from the beta evaluation version (i.e., you can't)--things were already
confused enough.  The whole point behind this SPEC process of trying to
hammer out an agreed to NFS (server) benchmark is to bring some sanity
and conformity to the process. I believe SPEC is helping in this
regard--getting players in the NFS server market to hash out their
issues in a server benchmark to provide a tool for the industry.

You mentioned that SPEC or LADDIS process is political. LADDIS
evaluation is being complicated by different participant vendor
concerns. My hope is that with sufficient feedback from the beta
evaluation, and SPEC's structure of "balance of powers", that LADDIS
will not suffer (and actually improve) as part of process.  I firmly
believe that the changes occurring since January when it was funneled
into SPEC have been for the better. It's not perfect (what is).

I've given up looking for perfection.

Is SGI a SPEC member? (I do believe so) If so, I *strongly* encourage
you to raise your objections to your SPEC rep! Now is the time.  If you
want, I'll put forward any concerns you have. I'll simply add them to
my voluminous list. Anyone should feel comfortable evaluating
PRE-LADDIS and getting comments back to either Bruce Keith
at DEC (SPEC PRE-LADDIS Project Leader) or any SPEC representative.
[Do we need a repost of the PRE-LADDIS info?]

Most of my concerns--which are somewhat nebulous--center on "workload
model".  Does PRE-LADDIS (or for that matter nhfsstone) define
sufficiently an NFS workload to justify its use as a benchmark?  I tend
to believe "yes". I and several other engineers (outside of Sun)
recently decided that PRE-LADDIS is a very powerful tool to aid in NFS
server analysis. Regardless of the SPEC outcome, or acceptance of
PRE-LADDIS as the mother-of-all NFS server benchmarks, the work as it
now stands (evolved from nhfsstone from Legato) provides a great tool
to accomplish analysis of NFS servers. I use it regularly to subject
servers to a reproducible load to analyze effectiveness of changes to
software or hardware.  Way cool.

Rusty Sandberg did a great job on nhfsstone. Mark Wittle (DG),
Ken Teelucksingh (DEC) and John Corbin (Sun) did major efforts
to get PRE-LADDIS (highly evolved from nhfsstone) to where
it is today. Thanks.

Brian Pawlowski
Newsgroups: comp.protocols.nfs