HDTenEightyP/NetNews huggingface dataset:
From: cabhhmm@cli74aa.NoSubdomain.NoDomain (Michele Morin 5115 A408)
Subject: Re: General description of Benchmarks
Organization: EDF Direction des Etudes et Recherches
Date: Fri, 8 Jan 1993 09:51:29 GMT


Hello Thomas,

I'm working about benchmarks and benchmarking for about 10 years now.

One year ago, I've read an article :Scientific benchmark characterizations
It is interesting because teh subject is treated in many points of view. 
It was one of 14 papers in a special issue of PARALLEL COMPUTING : 
BENCHMARKING OF HIGH PERFORMANCE SUPERCOMPUTERS, Vol 17, N 10 & 11 
The Managing Editors are in Munchen : Prof. Dr. Feilmeier, Junker  & Co.
Institut fur Wirtschafts und Versicherungsmathematik GmbH
Leonhard-Moll-Bogen 10
W-8000 Munchen 70
I hope you find what you want !
         
    Michele Morin
    Electricite de France
    michele.morin@der.edf.fr
             

Newsgroups: comp.benchmarks
From: eugene@amelia.nas.nasa.gov (Eugene N. Miya)
Subject: [l/m 10/14/92] SLALOM				(13/28) c.be FAQ
Keywords: who, what, where, when, why, how
Organization: NAS Program, NASA Ames Research Center, Moffett Field, CA
Date: Tue, 12 Jan 93 12:25:18 GMT
Reply-To: eugene@amelia.nas.nasa.gov (Eugene N. Miya)

13	SLALOM							<This panel>
14
15	12 Ways to Fool the Masses with Benchmarks
16	SPEC
17	Benchmark invalidation methods
18
19	WPI Benchmark
20	Equivalence
21	TPC
22
23
24
25	Ridiculously short benchmarks
26	Other miscellaneous benchmarks
27
28	References
1	Introduction to FAQ chain and netiquette
2
3	PERFECT
4
5	Performance Metrics
6	Temporary scaffold of New FAQ material
7	Music to benchmark by
8	Benchmark types
9	Linpack
10
11	NIST source and .orgs
12	Benchmark Environments

%A J. Gustafson
%A D. Rover
%A S. Elbert
%A M. Carter
%Z U. of Iowa, Ames Lab.
%T SLALOM: The First Scalable Supercomputer Benchmark
%J Joural of Parallel and Distributed Computing
%V 12
%N 4 
%D August 1991
%P 388-401
%r Supercomputing Review
%d November 1990

%A John Gustafson
%A Diane Rover
%A Stephen Elbert
%A Michael Carter
%T The Design of a Scalable, Fixed-Time Computer Benchmark
%R IS-5049/UC-32
%I Ames Lab, Iowa State University
%D 1990

And other sources.

Archive-directory: tantalus.al.iastate.edu:/pub/Slalom/ [129.186.200.15]
Tantalus.al.iastate.edu has the SLALOM sources in various languages and other
things including the latest reported results in
pub/Slalom/Reports/CurrentReport.

                   ^ A  
                s / \ r                
               m /   \ c              
              h /     \ h            
             t /       \ i          
            i /         \ t        
           r /           \ e      
          o /             \ c    
         g /               \ t  
        l /                 \ u
       A /                   \ r
        <_____________________> e   
                Language
 
Newsgroups: comp.benchmarks
From: vaghela@laxatv.reo.dec.com (Umesh Vaghela)
Subject: Re: [l/m 4/7/92] Performance metrics (5/28)	c.be FAQ
Keywords: who, what, where, when, why, how
Reply-To: vaghela@laxatv.reo.dec.com (Umesh Vaghela)
Organization: Digital Equipment Corporation
Date: Tue, 5 Jan 1993 15:48:58 GMT


could someone post benchmarks for Sparc vs Alpha
thanks.

--
+-------------------------------------------------------------------------+
|                            Umesh Vaghela                                |
|                Digital Equipment Corporation - Reading, Berks, UK       |
|                                                  _ _ _ _ _ _ _          |
| 	Userid : vaghela@marvin.enet.dec.com	  | | | | | | | |TM	  |
|          	 vaghela@laxatv.reo.dec.com       |d|i|g|i|t|a|l|         |
|          	 vaghela@wikkit.enet.dec.com      |_|_|_|_|_|_|_|         |
|									  |
|All views & opinions expressed here are entirely my own and NOT digital's|
+-------------------------------------------------------------------------+
Newsgroups: comp.benchmarks
From: eugene@amelia.nas.nasa.gov (Eugene N. Miya)
Subject: [l/m 4/6/92] New FAQ scaffold (6/28)		c.be FAQ
Keywords: who, what, where, when, why, how
Organization: NAS Program, NASA Ames Research Center, Moffett Field, CA
Date: Wed, 6 Jan 93 12:25:10 GMT
Reply-To: eugene@amelia.nas.nasa.gov (Eugene N. Miya)

6	Temporary scaffold of New FAQ material			<this panel>
7	Music to benchmark by
8	Benchmark types
9	Linpack
10
11	NIST source and .orgs
12	Benchmark Environments
13	SLALOM
14
15	12 Ways to Fool the Masses with Benchmarks
16	SPEC
17	Benchmark invalidation methods
18
19	WPI Benchmark
20	Equivalence
21	TPC
22
23	RFC 1242 terminology (network benchmarking)
24
25	Ridiculously short benchmarks
26	Other miscellaneous benchmarks
27
28	References
1	Introduction to FAQ chain and netiquette
2	Benchmarking concepts
3	PERFECT Club/Suite
4
5	Performance Metrics

Terminology
	Evaluation
	Analysis
		Mean Value Analysis
	Measurement
Model
Equivalence
	Languages
	Surrogates

Minimal benchmarks
	Kernels
	Tests
Simulations of benchmarks
Results data base
Source code
	Version control
Reduciability
Representativeness
	Functionality, how do you know it works? (correctness)
Testing

Workloads.

Output
	Analysis
	Interpretation
	Statistics, Composites, Averages, means, variance,
		Normalized numbers
	Graphics

Who
Virtual work
Application characterization

Kinds of benchmarkers
People who have to take numbers by faith.
People who reproduce but do so blindly.
People with the time and energy to do so.

                   ^ A  
                s / \ r                
               m /   \ c              
              h /     \ h            
             t /       \ i          
            i /         \ t        
           r /           \ e      
          o /             \ c    
         g /               \ t  
        l /                 \ u
       A /                   \ r
        <_____________________> e   
                Language
 
Xref: sparky comp.benchmarks:1921 comp.arch:12199
From: vac+@cs.cmu.edu (Vincent Cate)
Newsgroups: comp.benchmarks,comp.arch
Subject: FTPable Performance Results
Date: 9 Jan 93 22:13:44 GMT
Article-I.D.: cs.C0Lxqx.6p7.1
Followup-To: comp.benchmarks
Organization: School of Computer Science, Carnegie Mellon


On gatekeeper.dec.com in DEC/DECinfo/performance-summary there are some 
performance summary papers.  The following two papers are dated 
Nov 20 1992 and seem to have current HP, Sun, and RS-6000 machines as 
of that time:

   alpha-axp-server.ps
   alpha-axp-workstation.ps

These include results from many benchmarks (~dozen).

Do these papers in fact have the latest machines from the other vendors?

What other FTPable and current benchmark papers are out there?

I noticed that DEC does not list any IBM (or Amdahl etc) mainframes
in there "Competitive Systems" graphs.  :-)  

Does anyone have numbers for the old multi-chip-cpu mainframes? 

  -- Vince

PS  Will 1992 be remembered as the year "killer micros" assassinated
    the multi-chip-cpu mainframes?  (followup to comp.arch)
 
Newsgroups: comp.benchmarks
From: eugene@amelia.nas.nasa.gov (Eugene N. Miya)
Subject: [l/m 3/17/92] music to benchmark by	(7/28)	c.be FAQ
Keywords: who, what, where, when, why, how
Organization: NAS Program, NASA Ames Research Center, Moffett Field, CA
Date: Thu, 7 Jan 93 12:25:14 GMT
Reply-To: eugene@amelia.nas.nasa.gov (Eugene N. Miya)

7	Music to benchmark by					<This panel>
8	Benchmark types
9	Linpack
10
11	NIST source and .orgs
12	Measurement Environments
13	SLALOM
14
15	12 Ways to Fool the Masses with Benchmarks
16	SPEC
17	Benchmark invalidation methods
18
19	WPI Benchmark
20	Equivalence
21	TPC
22
23
24
25	Ridiculously short benchmarks
26	Other miscellaneous benchmarks
27
28	References
1	Introduction to the FAQ chain and netiquette
2
3	PERFECT
4
5	Performance Metrics
6

Benchmarking takes time.  Might as well listen to something useful.
Hey, engineers and scientists need some general education requirements and
to see the generally inteconnectedness of computing to the rest of the
world.

The Photographer by Philip Glass.
[Also "How to Get away with Second Degree Murder."]

Einstein at the Beach by Philip Glass.

"Democratic Process"
"Cowboy Lips"
"Art For Art's Sake" 
all off the "The Bobs" album on Kaleidoscope Records "K-18"
(P.O. box O, El Cerrito, CA, 94530)
[Excellent!  You are RIGHT!  SIGGRAPH 1984 as a good example]
jason@cs.utexas.edu

Dmitri Shostakovich
Aram Khachaturian
Sergei Prokofiev

as well as major cello concerti throughout the ages.  Paul Chihara's work
before he started writing for movies (if you can find it) is also good.
 
Or, if you're benchmarking a 16-processor Cray Y-MP C90, Chopin's Minute Waltz.

                   ^ A  
                s / \ r                
               m /   \ c              
              h /     \ h            
             t /       \ i          
            i /         \ t        
           r /           \ e      
          o /             \ c    
         g /               \ t  
        l /                 \ u
       A /                   \ r
        <_____________________> e   
                Language
 
Newsgroups: comp.benchmarks
From: morse@quark.mpr.ca (Daryl Morse)
Subject: Re: FTPable Performance Results
Organization: MPR Teltech Ltd., Burnaby, BC, Canada.
Date: 12 Jan 93 08:47:33


In article <1954@niktow.canisius.edu> pavlov@niktow.canisius.edu (Greg Pavlov) writes:
Path: mprgate.mpr.ca!cs.ubc.ca!utcsri!rutgers!ub!niktow!pavlov
   From: pavlov@niktow.canisius.edu (Greg Pavlov)
   Newsgroups: comp.benchmarks
   Date: 12 Jan 93 05:05:54 GMT
   References: <C0Lxqx.6p7.1@cs.cmu.edu>
   Organization: Canisius College, Buffalo NY. 14208
   Lines: 19

   In article <C0Lxqx.6p7.1@cs.cmu.edu>, vac+@cs.cmu.edu (Vincent Cate) writes:
   > 
   > On gatekeeper.dec.com in DEC/DECinfo/performance-summary there are some 
   > performance summary papers.  The following two papers are dated 
   > Nov 20 1992 and seem to have current HP, Sun, and RS-6000 machines as 
   > of that time:
   > 
   > Do these papers in fact have the latest machines from the other vendors?
   > 
>   "Results" for the newest HP machines are incomplete, probably because they
>    were unavailable at the time.

>    SGI/MIPS was totally ignored.  Given DEC's virtual abandonment of its 
>    MIPS-based line,  my suspicion is that this was not simply an oversight.

Look again, there are some SPEC and X11Perf numbers for SGI boxes,
though not that many.  As for why non-Mips platforms received more
attention in the summary, I really don't think it's "symbolic proof"
that DEC is dumping the DECstation line so much as a reflection of
where Mips-based platforms are in the current market (i.e., not really
anywhere). Sun, HP, and IBM are DEC's competition and rightfully
should be the focus of any useful comparision.

--
Daryl Morse                     | Voice  : (604) 293-5476
MPR Teltech Ltd. 		| Fax    : (604) 293-5787
8999 Nelson Way, Burnaby, BC    | E-Mail : morse@mpr.ca
Canada, V5A 4B5                 |        : mprgate.mpr.ca!morse@uunet.uu.net
From: jmd6@unix.brighton.ac.uk (Morgoth.)
Newsgroups: comp.benchmarks
Subject: Info on Performance Monitors
Date: 10 Jan 93 15:03:16 GMT
Organization: University of Brighton, UK

Performance Monitors

I'm A final year student on a BSc in Computer Graphics at Brighton Uni in
the UK. I am just about to start on my final year project which is to be 
locking into performance monitors. For this I intent to try to write a 
performance monitor and investigate other peoples and the issues involed
around performance monitoring.
So far my research has mainly considered the technical issues of writting
such an application, but I am now widerning this looking for articles on
performance  monitoring and information on performance monitoring products.


Primary I am intrested in the UNIX monitoring field, but my research must
also cover other operating systems. I'm not a complete novice on this subject
having written a basic performance monitor (in UNIX shell scripts) for ICL
DRS6000, which was being marcketed by IT Southern in the UK.

So can any one give me a few pointers as to articles and revies covering 
performance monitoring? 

What is a good introduction to the subject?

Electronic articles are welcome as they are less dificult for me to get hold of than some journels and books.

Jon Durrant
Newsgroups: comp.benchmarks
From: ludwigt@Informatik.TU-Muenchen.DE (Thomas Ludwig)
Subject: General description of Benchmarks
Originator: ludwigt@sunjessen21.informatik.tu-muenchen.de
Organization: Technische Universitaet Muenchen, Germany
Date: Thu, 7 Jan 1993 17:53:47 GMT


Hello folks,

Its the first time I use this possibility of communication.

My question is if anybody knows an article about benchmarks.
That means general discussion of benchmarks, the use, the aims, ...

I hope you understand what I mean.

					Thanks in advance
					Thomas "Big L" Ludwig
Newsgroups: comp.benchmarks
From: eugene@amelia.nas.nasa.gov (Eugene N. Miya)
Subject: [l/m 3/23/92] Linpack			(9/28)	c.be FAQ
Keywords: who, what, where, when, why, how
Organization: NAS Program, NASA Ames Research Center, Moffett Field, CA
Date: Sat, 9 Jan 93 12:25:10 GMT
Reply-To: eugene@amelia.nas.nasa.gov (Eugene N. Miya)

9	Linpack					<This panel>
10
11	NIST source and .orgs
12	Measurement Environments
13	SLALOM
14
15	12 Ways to Fool the Masses with Benchmarks
16	SPEC
17	Benchmark invalidation methods
18
19	WPI Benchmark
20	Equivalence
21	TPC
22
23
24
25	Ridiculously short benchmarks
26	Other miscellaneous benchmarks
27
28	References
1	Introduction to FAQ chain and netiquette
2	Benchmarking Concepts
3	PERFECT
4
5	Performance Metrics
6	Temporary scaffold of New FAQ material
7	Music to benchmark by
8	Benchmark types

The LINPACK benchmark is a very simple LU decomposition of a dense linear
system (Gaussian elimination) by Jack Dongarra the developer of the
LINPACK library and the netlib numerical software server.

Ref: Dongarra's article in CACM on Netlib.
And ACM SIGARCH Computer Architecture News and SIGNUM Newsletter.

Advantages:
Simple fairly portable FORTRAN.  One of the shorter benchamrks.
It is small enough to be carried on disk or in Jack's laptop without
consuming too much porting time.
A good attempt at experiment control, has stringent execution requirements.

Dongarra also records the compiler options used to invoke the Fortran
compilers.  Record keeping is good.  Reports are quickly available
electronically and published with some frequency in Supercomputing Review.

Disadvantages:
Diminishing parallelism during the decomposition (as in all Gaussian
elimination).  It is perhaps too simple.  Execution time is on the shorter
side.  It cannot characterize less numeric applications.

NETLIB benchmark index (Linpack benchmark)
netlib@ornl.gov
	send index from benchmark
includes linpack (100x100,300x300,1000x1000).

What is interesting about these sizes is that they are not
the powers of 2 which characterize many benchmarks.  Powers of 2
can bias in favor of some architectures and bias against other
architectures.

                   ^ A  
                s / \ r                
               m /   \ c              
              h /     \ h            
             t /       \ i          
            i /         \ t        
           r /           \ e      
          o /             \ c    
         g /               \ t  
        l /                 \ u
       A /                   \ r
        <_____________________> e   
                Language
 
Newsgroups: comp.benchmarks,comp.lang.ada
From: kdm@Rational.COM (Kent Mitchell)
Subject: FTP site for Ada comp/execution benchmarks
Summary: benchmarks
Organization: Rational
X-Newsreader: TIN [version 1.1 PL8]
Date: Tue, 5 Jan 1993 16:29:48 GMT

I'm looking for an FTP site with code for benchmarking Ada compiler
compilation speed and code for testing execution performance of the
generated code.

Any suggestion would be apreciated.

Kent Mitchell                    My views are my own.
Technical Consultant
Rational
3320 Scott Blvd.
Santa Clara, Ca
Newsgroups: comp.benchmarks
From: pavlov@niktow.canisius.edu (Greg Pavlov)
Newsgroups: comp.benchmarks
Subject: Re: FTPable Performance Results
Date: 12 Jan 93 05:05:54 GMT
Organization: Canisius College, Buffalo NY. 14208

In article <C0Lxqx.6p7.1@cs.cmu.edu>, vac+@cs.cmu.edu (Vincent Cate) writes:
> 
> On gatekeeper.dec.com in DEC/DECinfo/performance-summary there are some 
> performance summary papers.  The following two papers are dated 
> Nov 20 1992 and seem to have current HP, Sun, and RS-6000 machines as 
> of that time:
> 
> Do these papers in fact have the latest machines from the other vendors?
> 
  "Results" for the newest HP machines are incomplete, probably because they
   were unavailable at the time.

   SGI/MIPS was totally ignored.  Given DEC's virtual abandonment of its 
   MIPS-based line,  my suspicion is that this was not simply an oversight.



   greg pavlov
   pavlov@fstrf.org
Newsgroups: comp.benchmarks
From: eugene@amelia.nas.nasa.gov (Eugene N. Miya)
Subject: [l/m 3/17/92] Measurement environments		(12/28) c.be FAQ
Keywords: who, what, where, when, why, how
Organization: NAS Program, NASA Ames Research Center, Moffett Field, CA
Date: Tue, 12 Jan 93 12:25:18 GMT
Reply-To: eugene@amelia.nas.nasa.gov (Eugene N. Miya)

12	Measurement environments			<This panel>
13	SLALOM
14
15	12 Ways to Fool the Masses with Benchmarks
16	SPEC
17	Benchmark invalidation methods
18
19	WPI Benchmark
20	Equivalence
21	TPC
22
23
24
25	Ridiculously short benchmarks
26	Other miscellaneous benchmarks
27
28	References
1	Introduction to FAQ chain and netiquette
2
3	PERFECT
4
5	Performance Metrics
6	Temporary scaffold of New FAQ material
7	Music to benchmark by
8	Benchmark types
9	Linpack
10
11	NIST source and .orgs

Benchmarking environments
Taxonomy described by: A, B, C, D, E
Examples:
	A Cray Y-MP or better with integrated hardware performance monitor
	B Cray-2, Convex C-1/C-2...
	C VAX-11/780, IBM PC, IBM 370-class
	D Apple II, Timex Sinclair
	E Incomplete hardware, chip, module, board-level

An A environment should have state of the art hardware and
software to aid benchmarking.  Hardware should be minimally or non-intrusive.
This category should change with time.  The SS-1 should have a
more programmable HPM, this would lower the Y-MP down into
a B environment (can stay A if no other HPM type hardware becomes
predominant).  This is akin to high-precision Cesium clocks.

A B environment is a "good" environment to do benchmarking.
It has as a minimal, a near cycle-time clock, profiling software,
etc.  The environment at this level or higher may also be more hostile
for measurement.  Good performance is likely to be an issue, so the
compilers will have good optimizers.

A C environment is an "average" benchmarking environment.
This is just typical: a 60 or 100 Hz clock.  Some profiling software.

A D environment is a difficult environment to do benchmarking.
This environment may not have a clock or software to help the poor
benchmarker.  Timing by wrist watch or maybe oscilliscope.
Software? What software?

An E environment:
Another difficult environment to do benchmarking.  It might have software.
It might be an incomplete computer: a processing using of a larger
multiprocessor: it touches on the issue of composition.

If dealing with a simulator:
there maybe an F category for simulated timing.

                   ^ A  
                s / \ r                
               m /   \ c              
              h /     \ h            
             t /       \ i          
            i /         \ t        
           r /           \ e      
          o /             \ c    
         g /               \ t  
        l /                 \ u
       A /                   \ r
        <_____________________> e   
                Language
From: betsys@cs.umb.edu (Elizabeth Schwartz)
Newsgroups: comp.org.eff.talk
Subject: Re: "Civil Disobedience" et al.
Date: 6 Jan 93 23:04:58 GMT
	<van-060193103022@gerard.eff.org>
Followup-To: comp.org.eff.talk
Organization: University of Massachusetts at Boston

I for one get frustrated by the tendency to categorize to the extreme
in this medium. It seems that every tiem a group or a publication gets
interesting, some folks start yelling for the discussion to move to a
different group. Sure, there's a place for strict definition of a
subject... in the strictly technical groups. A group and a publication
dedicated to civil liberties on the emerging electronic frontier is
certainly a place where opinions from past definers of our freedoms
should be welcome!
  I value diversity and hearing things which I might not have
otherwise heard. Keep up the good work! 
--
System Administrator                  Internet: betsys@cs.umb.edu
MACS Dept, UMass/Boston               Phone   : 617-287-6448
100 Morrissey Blvd                    Staccato signals
Boston, MA 02125-3393                      of constant information....
Newsgroups: comp.org.eff.talk
From: Clayten_Hamacher@mindlink.bc.ca (Clayten Hamacher)
Subject: Re: Beneficial Virus
Organization: MIND LINK! - British Columbia, Canada
Date: Tue, 12 Jan 1993 06:34:13 GMT

>>One possible problem would be a malicious virus masquerading as a
>>valid marker file.  Then any file compressed with the beneficial
>>virus would activate the malicious virus - some sort of check would
>>have to be made that the virus was the *RIGHT* virus.
>
>With all the attendant troubles of verification and the slowdown it
>will present.

Do you check all the rest of the programs on your system? They are just as
likely to have 'evil twins' lurking around.

>Note that this would change the creation date of every executable
>every time the program was run.  Even on a single-user MS-DOS system,
>this can have some undesirable results.

Like what? Your half-witted virus checker goes off and shuts down the
computer?
In almost every OS I've ever used there has been a way to fake the
modification
date, in unix you just write a new date, in MS-DOS if nothing else you just
record the current time, set the clock to the old modification date, write
the
file, set the clock to the time you just recorded plus about two seconds..
And
decent virus could do it so it's not a very good way to tell if you're
infected.

>>Having considered the infected file, let's consider the virus in operation.
>>It is assumed that the virus has only been placed in memory after it has
>>been verified that the marker was present, and it is okay for it to run.
>
>Does the virus get removed from memory after the executable stops?  If not,
>then what happens when another executable is run?

You do understand the meaning of the word 'virus' don't you? If it finds the
marker (ie it's ok to execute) then it does what the name suggests and
infects
the next executables that you run. If it doesn't find that permission then it
quits operation after unpacking the program.

--

Clayten_Hamacher@Mindlink.bc.ca                 Land of the rising snow.
Newsgroups: comp.org.eff.talk
Subject: Re: legal question re ano
From: matt.clement@grapevine.lrk.ar.us (Matt Clement) 
Date: 11 Jan 93 02:08:00 GMT
Reply-To: matt.clement@grapevine.lrk.ar.us (Matt Clement) 
Distribution: world
Organization: The GrapeVine BBS *** N. Little Rock, AR *** (501) 753-8121

>If lawyers were doctors then hospitals would be places where one of
>them injects your arm with a disease and the other with the cure, and
>both will expect to be paid.


How true!


-- Matt
---
 * WorldComm BBS  301-657-8313/8958
 * PostLink(tm) v1.04  WORLDCOM (#1860) : RelayNet(tm)
                                                                                                 
Newsgroups: comp.org.eff.talk,comp.software-eng,misc.jobs.misc