From: walter@zarquon.Eng.Sun.COM (Walter Bays)
Newsgroups: comp.benchmarks
Subject: Dhrystone, SPECint, and Nostradamus
Date: 10 Sep 1992 17:35:46 GMT
Organization: Sun Microsystems
Distribution: world
Reply-To: walter@zarquon.Eng.Sun.COM
NNTP-Posting-Host: zarquon

A year ago there was a column in a respected magazine by a respected
industry consultant purporting to show a simple formula to "accurately"
predict SPECmarks from (Dhrystone) MIPS and MFLOPS.  As proof he showed
a list of machines with predicted versus actual SPECmarks.  As I
remember, the error was something like from 2% to 24% -- on that very
set from which the formula was derived by least-squares fit. 

He also predicted that the Sun 3/80 had a negative SPECmark meaning.
(Does that mean it's *really* **slow**?  Or that it completed the
benchmarks before he pressed RETURN?)  Then the consultant computed the
formula for a couple of other DEC machines with the same architecture
(MIPS) as part of the original set, and the error was something like
12% to 40%.

But now I have discovered an even better SPECmark predictor among the
lesser-known writings of Nostradamus who, writing in the 13th century,
foretold the development of computers, the universal use of the
Dhrystone benchmark, the creation and widespread adoption of SPEC,
Reinhold Weicker the creator of Dhrystone becoming a major contributor
to SPEC, and joint ventures between IBM and Apple.

Nostradamus foretold that Sir Arthur Conan Doyle would write the
Sherlock Holmes stories, that they would be printed in anthology by
Baring-Gould, and that encoded in their text would be the all the
secrets of computer performance evaluation.  To foretell the SPECmark
of a computer, you take the first three digits of its' model number and
turn to that page in the book.  Scan to the first word beginning with
the first letter of the manufacturer's name and begin reading.  Encode
each letter as a number, A=1, B=2, etc.  Take the first seven numbers,
corresponding to the seven planets of the solar system.  Multiply each
by the corresponding number in this list, and sum the products:
-0.266, 0.191, -0.367, 0.052, -0.145, 0.654, 0.881.  The result is the
SPECmark.

To test the accuracy of this medieval mystic, I applied his formula to
SPEC results published in the Winter 1990 newsletter.

Data General Aviion 310         page 310, "dark over this matter"
Data General Aviion 5010        page 501, "Douglas, said the inspector"
MIPS RC 3260                    page 326, "moved to rooms in"
MIPS RC 3240                    page 324, "much has been written"
Sun 4/490                       page 490, "subject with a wave"
Sun 4/330                       page 330, "streak not only of"
Sun SPARCstation 1              page 1,   "some who will read"

The letter codes are then used to compute SPECmarks.  As you can see,
the maximum error of the Nostradamus method is 0.1%.  This method is
far less laborious and error prone than actually running benchmarks.
And the accuracy is better than the dice roll, tea leaves, linear
regression, tarot card, or chicken entrail methods.

                ---Planetary Codes-------------- Predicted Actual
DG Aviion 310    4    1   18   11   15   22    5    9.71    9.7
DG Aviion 5010   4   15   21    7   12    1   19   10.1    10.1
MIPS RC 3260    13   15   22    5    4   20   15   17.3    17.3
MIPS RC 3240    13   21    3    8    8    1   19   16.1    16.1
Sun 4/490       19   21    2   10    5    3   20   17.6    17.6
Sun 4/330       19   20   18    5    1   11   14   11.8    11.8
Sun SS 1        19   15   13    5   23    8   15    8.41    8.4

Unfortunately, the text of the original manuscript is obscured in later
passages, so we do not know how to predict SPECint92 nor SPECfp92.  Oh
well, I guess we'll have to keep running benchmarks after all.



---
Walter Bays                                    walter.bays@eng.sun.com
Sun Microsystems, 2550 Garcia Ave., MTV15-404, Mountain View, CA 94043
(415) 336-3689                                      FAX (415) 968-4873