From: abuhr@owlnet.rice.edu (Aaron M. Buhr)
Subject: Re: Intel vs. Motorola performance
Organization: Rice University
Date: Wed, 13 Jan 1993 03:31:49 GMT

In article <1993Jan13.001422.29177@klaava.Helsinki.FI> lindblad@klaava.Helsinki.FI (Jarkko Lindblad) writes:
>Aaron M. Buhr (abuhr@owlnet.rice.edu) wrote:
>>       Please, if you're going to say ANYTHING about Intel CPU vs. Motorola
>> CPU performance, back up your claims with third-party benchmarks available
>> (same source) for both systems, ideally the SpecInt92 and SpecFP92 figures.
>
>Same source does not make benchmarks any more valid that they really
>are. Have you ever hear on optimizations?

    The Spec code was originally chosen as being indicative of the kinds of
things real applications can do, to the extent that in order for
optimizations to have a significant effect on the Spec figures, they would
have to also have a similar effect on real-life code.  The Spec figures are
compiled from quite a few different code sources, and in order to have a
significant effect on the final Spec figures you'd have to have substantial
optimizations for all the different code fragments, or a big increase for a
few.  In addition, the Spec code is updated regularly when an unfair (read
not-real-world) optimization is found.  

>
>> This subject has been flamed back and forth over and over and over, ad
>> nauseum...   Not that you can't say that about quite a few other arguments
>> here on c.s.a.a, but it seems that in this case there's a fairly simple,
>> equitable solution, namely the Spec figures.
>
>Spec's are not more valid that other benchmarks.

    There are quite a few people more knowledgeable about this subject than
you or I that do believe the Spec benchmarks are at least somewhat
indicative of computer performance, and certainly better than predecessors
such as dhrystones, etc. .  Or are you going to contend now that you know
more about the subject than the Spec originators?  You know more about
computer benchmarking than a host of Computer Science and Electrical
Engineering Ph.D.s?

    Admittedly, benchmarks are not going to give you a perfect comparison,
but they do seem to reflect real world performance fairly well, especially
for the tasks similar to those that are in the Spec test suite.  Your
_specific application_ performance may differ from the Spec figures,
but if you run a variety of different applications overall performance
should approach the Spec figures.


>If Specmarks would be the absolute truth of _CPU_ performance, then
>why do different PC-configurations (with 486) get different results,
>or why do similar PC-configurations get different results?

    You got me there.  I had forgotten that many of the tests involved are
so real-world that the system surrounding the CPU can have a significant
effect on the final performance figures.  That does essentially disqualify
the Spec figures for CPU comparison.  There may be some individual tests
within the Spec test suite that may be more applicable for CPU comparison,
though (rather than using the figures for the whole test suite).


    All your statements about different instruction timings and such are
useless without an an analysis of the frequency of use of such instructions.
Sure, the 68040 may perform a multiply in 5 clocks as opposed to the 80486's
25 (or whatever), but if the instruction is only used once every 1000
instructions, it's not going to be a significant difference.  I am going to
have to go check up on relative instruction timings.  If the 68040 does have
shorter instruction execution times than the 80486 on all comparable
instructions, I'll grant that the 68040 is faster.  Otherwise your "proof"
is inconclusive.


Aaron Buhr
abuhr@owlnet.rice.edu