From: jwreilly@mipos2.intel.com (Jeffrey Reilly)
Newsgroups: comp.benchmarks
Subject: Re: SPEC : questions, copyright, summary - a database of spec's ?
Date: 16 Nov 92 16:20:18 GMT
Organization: Intel Corporation, Santa Clara, CA USA


In article <1992Nov16.143948.27382@crc.ac.uk> you write:
>i would like to complie a list of all (!) SPECmark ratings, both INT & FP,
>even THRUput, if possible.  basically i would like to be able to produce a 
>list that i could post every-so-often when a new spec-mark for a new machine
>appears.  this would be of interest when the dec-alpha marks came out, to
>give an example.  this however, raises a coupla questions...
>
>is it legal ?  am i infringing SPEC's copyright ?  

Well, I am not a lawyer :-) but from what I understand the benchmarks results are
not copyrightable (the terminology and the reporting format from the SPEC
Newsletter may be). For example, both companies and magazines frequently publish
tables of SPEC data (for example, performance brief's from DEC, IBM, Sun and
magazines such a Microprocessor Report, Andrew Allison's RISC Forum, EE Times.

>
>how should i report them ?  i assume that i would include supplementary
>info along with each benchmark result, such as cpu (with mhz rating), 
>cache size (?), memory (is it relevant?), compiler used?  i assume that
>it is necessary to chop the results up into specint92, specfp92, and 
>specthruput, (and also spec89marks, if relevant).  i would probably also
>separate the multi-cpu from the single-cpu results.

Typically, SPEC suggests that enough information be reported to allow someone to
easily reproduce the results. Cache size and memory are important (though
admittedly if you throw enough hardware at the problem, eventually you reach a
point where adding more doesn't help). The choice of a compiler makes a big
difference (as an extreme, look at the difference between the results published
by SUN, HP, etc. before and after the introduction of the KAP Preprocessor).

The OS may make a difference in SPEC SDM results.

Also, note that configurations may vary between a reported SPECint92, SPECfp92
and SPECrate_int92, SPECrate_fp92.

Another key point to make is you should make sure you mention the source of the
information.

All and all, it depends on how much work you want to put into it. It is a lot of
information that changes and is added to quite rapidly from many sources (SPEC
Newletter, company hardware and software announcements, etc.)

SPEC is currently planning to include a year end summary of performance for it's
December Newsletter.
>
>i have read the FAQ pertaining to spec-stuff.  perhaps i am trying to create
>a store of spec-marks that could be attached to a FAQ, although this is not
>my intention.  it is intented to be stand-alone (!) from a faq.  do you think
>anyone from spec would be upset.  and no, i can't be bothered to go and
>ask them myself.  any comments welcomed (flame-proof suit on...) mainly be
>postings, email also.
>

Well, if you ever want to bring anything up with SPEC, we're always willing to
listen... :-)

Personally, I think its a useful idea and I think most other people would too
(particularly if it is kept current and correctly).
>thanks...

Good Luck!

Jeff
Intel's Representitive To SPEC
Jeff Reilly			| "There is something fascinating about
Intel Corporation		| science. One gets such wholesale returns
jwreilly@mipos2.intel.com	| of conjecture out of such a trifling
(408) 765 - 5909		| investment of fact" - M. Twain
All opinions are my own....
Xref: sparky comp.benchmarks:1707 comp.arch.storage:780