From: lindblad@cc.helsinki.fi
Newsgroups: comp.sys.amiga.advocacy
Subject: nap42487s benchmarks are always perfect or are they?
Date: 13 Apr 92 20:43:18 GMT
Organization: University of Helsinki

In article <1992Apr12.235751.26074@ux1.cso.uiuc.edu>, nap42487@uxa.cso.uiuc.edu (Nishith A. Patel) writes:
> I'll tell you wy the benchmarks are accurate. One,you seem to know exactly
> what my FP benchmarks do. Why don't you send me a copy of the source code
> and we ca look at them and find out that they are not FP tests at all, just
> integer tests. The Microprocessor report just does not know the difference
> between integer and FP, and just randomly grouped some under integer
> (yes, that is integer, i-n-t-e-g-e-r) and some under FP. (F-P). The world
> is lucky we have you here as no one else can determine what tests do what,
> probably not even the people who made them.

I suppose this is for me?
	Why the fuck should I send you copy of source code to anything? You
still don't get it. I sent you bencmarks that clearly proved you that
benchmarks don't tell you the thruth.
	The world probably is luckier to have me who atleast do know that
benchmarks don't tell you much about CPU performance but more about system
performance and especially about compilers efficiency to detect bencmarks and
optimize to certain processor. But you just have to insist that benchmarks tell
you the absolute truth.
	The people who have made the bencmarks sure do know that they don't
tell you much but as people want them and you can even sell magazines (and
computers) with benchmarks they're still used.

> Those benchmarks were not published by IBM or something, they were from the
> microprocessor report trying to compare CPUs.

I don't care who publishes these benchmarks - the results of course vary, but
nothing changes the _fact_ that benchmarks don't tell much about CPu
performance no matter how good they are (even if no compiler manages to do
illegal optimizations we do have to think about the fact that compilers are not
equalent.
 
> You posted some benchmarks using I/O. Gee, you don't think a workstation
> will do better at I/O than a clone, do you? After all, clone makers

Yes, I posted some benchmarks with I/O on them, and then? If we're thinking in
your way that should not matter as you say that benchmarks all allmighty and
tell you the absolute truth no matter how false they are.

> make sure they have the highest throghput and best graphics speed, disk 
> access, etc so they can rival workstations. Not quite. As Jerry pointed out,
> the Apollo has good compilers that are optimized for Dhrystone (This is a 
> common thing to do with compilers, optimize them for the Dhrystone benchmark).

There might be hope for you - you now seem to understand that it all depends
one hell of a lot on the efficiency of the compiler. Now you must understand
that your benchmarks are as much false as the ones I offered you (and even told
you that neither ones don't tell the truth).

> I can just as easily show you a 486 system beating an 040 on the Dhrystone
> benchmark (I won't post any, as I do not remember the exact numbers for any
> of them). Try a Dhrystone on an Amiga with 040 and a 486 using commonly used
> compilers for both (say, Borland C++ or MS C 7.0 vs SAS- Sorry to break
> your hearts, but BLAZEC doing infinte dhrystones is NOT available for the
> Amiga, as many sadly believe).

Shit. Shit. Shit. You still don't get it: I said that benchmarks do not tell
the truth and that the results depend very much on compiler, CPU settings,
system and other things. On the last paragraph you said that the 68040 compiler
detects dhrystones etc. (which is very probably true to some point) but on this
paragramps you evidently claim that the compilers _you_ choose are absolutely
equivalent - how do you know that MS C 7.0 don't detect dhrystones and do some
illegal optimizations and that SAS C optimizes correctly to 68040? You don't
and still you claim all kinds of stupid things. Why don't you finally
understand that it's the compiler that is the weakest link when you're
benchmarking?

> And how about what I said earlier. About running tw similar apps on an
> Amiga and 486? Someone posted some stupid shit earlier about using 
> SUPERSUPERDUPERCAD on IBM vs. SUCKCAD on Amiga. That is not what I said.

Amiga with 68040?
	I did not post anything like that, but if you've not noticed it's you
who tend to write lots of shit.

> I said, use AutoCAD 386 and ANY cad program on the Amiga. Then try a series
> of tests. If anyone actually does this, don't post, unless you post every

And you really think I could afford to buy CADs for any platform? I guess I
could of course use Universitys PCs and I guess they also have AutoCads, but I
could not buy Ax000+68040 card + CAD as I'm a poor student.
	I guess you now mean that 386 beats 040?
	And now you claim that 386 is faster than Amiga because some program
may be more optimized and probably uses different algorithms? It would be
easier if you do benchmarks with your 386 and buy an A3000 and find out the
slowest CAD for Amiga and publish the results - I guess that would give you the
picture you want us to see.

> I believe in fair benchmarks, impartially and taking account all of them.

There is no such thing as "fair benchmarks" when comparing CPUs from different
families. If you want to do any comparisions that have any value you _have to_
study both CPUs, study from books and magazines.

After studying tell me (or rather, tell me now so that I can have a good laugh)
why 68040 is slower on integers that 486 in your mind even as it only has one
single unified cache, more register shuffle, slower instructions, no copyback
mode. Please tell me this.

I have asked you that thing now for about 5 times and I still miss the answer.


> Thesystems should be using the CPU only, with similar levels of things
> like memory bottleneck (no 40 ns vs 120 ns). If the tests are true and
> comprehensive, not just a few selected ones where the 040 is particularly
> well suited, then benchmarks do give you a pretty good idea. Because
> all people use computers for anyway is to run programs. Nobody runs anything
> in theory, Jarkho).

First, I'm Jarkko, not Jarkho...:-)

Second - how can you compare the performance of two CPUs in practise if there
is so many different variables? Can you find a PC that has absolutely equal
memory system to A3000? Can you find a PC compiler that does produce _exactly_
as much optimized code for 486 as say SAS C 5.10 optimizes for 68040? Does this
compiler detect benchmarks exactly as well (or badly) as the SAS compiler? How
can you be sure that the compilers in fact do code with similar amount of
optimization? Or how can you be sure that AutoCAD and some CAD for AMiga have
exactly similar optimizations and use same algorithms? You can't be and
therefor you can not get truthful results by benchmarking but you can only get
quite truthful results by comparing the CPUs in theory.


You seem not to know what to think about benchmarks: first you claim that they
are the absolute truth, then you say that my benchmarks are all false because
of this and that, and then you continue saysing how _your_ benchmarks are
perfect and tell the absolute truth and you fail to see that your own
benchmarks have all the same faults my benchmarks have.

> Nish Patel  nap42487@uxa.cso.uiuc.edu
> A


--
Jarkko Lindblad
lindblad@cc.helsinki.fi
Newsgroups: comp.sys.amiga.advocacy