From: eugene@amelia.nas.nasa.gov (Eugene N. Miya)
Subject: [l/m 10/1/92] NIST source and .orgs	(11/28)		c.be FAQ
Keywords: who, what, where, when, why, how
Organization: NAS Program, NASA Ames Research Center, Moffett Field, CA
Date: Tue, 12 Jan 93 12:25:13 GMT
Reply-To: eugene@amelia.nas.nasa.gov (Eugene N. Miya)

11	NIST source and .orgs				< This panel >
12	Measurement environments
13	SLALOM
14
15	12 Ways to Fool the Masses with Benchmarks
16	SPEC
17	Benchmark invalidation methods
18
19	WPI Benchmark
20	Equivalence
21	TPC
22	RFC 1242 terminology (network benchmarking)
23
24
25	Ridiculously short benchmarks
26	Other miscellaneous benchmarks
27
28	References
1	Introduction to FAQ chain and netiquette
2
3	PERFECT
4
5	Performance Metrics
6	Temporary scaffold of New FAQ material
7	Music to benchmark by
8	Benchmark types
9	Linpack
10

NISTLIB/NBSLIB -- the mail server daemon
It exists from an ONR project of indeterminate origin.

nistlib@cmr.ncsl.nist.gov
	send index
includes LLNL loops (aka Livermore Loops, Livermore Fortran Kernels (LFK),
	NAS kernels, Los Alamos benchmarks, parcbench, whetstone, dhrystone,
	mendez, and many others
no results -- they do not wish to get sued.


Reviews of various benchmarks:

LFK, Livermore Fortran Kernels, Livermore Loops, MFLOPS.
Frank McMahon's program is one of the more rigorous.
A personal favorite in some ways.
Problems include the sheer volume of information.


There are 14 and 24 loops versions.  The 24 loops version is the
"official one."  Do not trust vendors bearing 14 loop results.
Frank has decided that the output of his programs goes back into the
program in a Comment form.  This helps with the database communication problem.


David Bailey's NAS Kernels.
Uses higher order kernels than either Dongarra and McMahon.
Uses trig functions which can behave in strange ways.
Requires the use of 64-bit arithmetic.
Has interesting consequences for DOUBLE PRECISION COMPLEX types.
These programs are less portable than the less complex loops.
One advantage is an attempt to measure compiler differences.
There is an optional Level which people may code to improve optimization.


ORGs.

Performance mailing list: (oriented toward performance analysis
using quantitative modeling (Mean-Value Analysis [MVA], etc.)
performmance@vuse.vanderbilt.edu

ACM/SIGMETRICS (Perforance Evaluation Review)
	Check the ACM.

CMG is concerned with all aspects of computer system measurement
and capacity planning. CMG has an annual meeting in North America.
This year's meeting is in Reno (Bally's Grand), Dec. 7 - 11, 1992.

CMG can be contacted at (312) 527-6652 FAX: 312 527-6656
			401 North Michigan Avenue
			Chicago, IL 60611-4267

CMG has had a definite bent toward large IBM mainframe systems
but that has been changing.  It also has a bend toward system tuning.
Keywords: Measurement,Capacity Management,System Management

                   ^ A  
                s / \ r                
               m /   \ c              
              h /     \ h            
             t /       \ i          
            i /         \ t        
           r /           \ e      
          o /             \ c    
         g /               \ t  
        l /                 \ u
       A /                   \ r
        <_____________________> e   
                Language