From: eoahmad@ntuix.ntu.ac.sg (Othman Ahmad)
Subject: Surprised:gcc&dhryston benchmarks
Organization: Nanyang Technological University - Singapore
X-Newsreader: Tin 1.1 PL4
Date: Thu, 10 Sep 1992 13:19:08 GMT

As promised in my previous post, I intend to conduct gcc-2.2.2 and dhryston-1.1
benchmarks on different machines:

Dec Station 5000 running on Ultrix, using mips:
Dhrystone(1.1) time for 500000 passes = 12
This machine benchmarks at 39370 dhrystones/second
       19.3 real        12.8 user         0.0 sys  
GNU C version 2.2.2 [AL 1.1, MM 19] DECstation running ultrix compiled by CC.
        4.5 real         2.4 user         0.2 sys  
 time /usr/users/othman/lib/gcc-lib/mips-dec-ultrix/2.2.2/cc1 dhry.cpp -mgpopt -quiet -dumpbase dhry-1.1.c -O2 -version -o dhry-1.1.s &> test1.dat

80386-DX 25Mhz uncached with 16Mbyte ram and conners 200Mbyte IDE hard-disk:
Dhrystone(1.1) time for 500000 passes = 61
This machine benchmarks at 8086 dhrystones/second
       63.29 real        62.97 user         0.16 sys
time /usr/local/lib/gcc-lib/i386-386bsd-bsd/2.2.2/cc1 dhry.cpp -quiet -dumpbase dhry-1.1.c -O2 -version -o dhry.s &> test2.dat
GNU C version 2.2.2 (80386, BSD syntax) compiled by GNU C version 2.2.2.
        5.00 real         3.76 user         1.05 sys

		cc1		dhry
386/mips	1.56		4.92

If we use cc1 version 1.41, the time for mips is only 0.8s so 386/mips=4.7
which is closer to to ratio for dhry.

I'm really surprised that my cheap 386 is only 56% slower than the much more
expensive mips based workstation.

What are the possible reasons?
i) dhrys is a very small program that runs very well in cached systems, 
because it can fit in all the cache and just loops for 500000 times, so 
favouring cached systems,

ii) cc1 is a much larger program so cannot fit inside caches.
iii) harder to optimise mips code compared to 386, I may verify this later.

Conclusion:
dhryston 1.1 is not yet useless because it actually gives a reasonable
processor speed indication for small programs. Sometimes this is what we
want.

For this reason I'm not convinced of the need for dhryston-2.1 which consists
of 2 parts which makes it difficult to transport and compile. The true solution
to redundancy is to input data from a separate file or "pipe", not by splitting
the code.

The benchmarks which I showed also took the disk access time. To reduce the
effect of disk I/O on the results, pipes should have been used.

My recommendations to Spec Mark consortium is to find small programs that
characterise a computer system characteristics, e.g. cache, RAM, disk,
Character, Graphic, serial, interrupt-response etc.

Then for each software, what portions of those characteristics do they 
consume. Cross-correlation with predicted results using those basic 
characteristics for a software, and the actual results of running user 
software should be done, to validate the "usefulness of those programs".

Some of those small programs are 
dhryston for integer small programs,
lorenze for floating point small programs,
iozone for portable I/O,

I believe this is the methods used by engineers in calibrating and fine
tuning circuits, by measuring voltage,current,power, complex impedence,
distorting etc.

The benchmark results are not extensive. Any effort to verify or even
counter propose will be most welcomed. In other words 
substanciated flames are very welcomed.

quotation of published works will be even more appreciated. I do not have to
waste time doing bits and pieces.

--
Othman bin Ahmad, School of EEE,
Nanyang Technological University, Singapore 2263.
Internet Email: eoahmad@ntuix.ntu.ac.sg
Bitnet Email: eoahmad@ntuvax.bitnet

Newsgroups: comp.benchmarks