From: stp@bernina.ethz.ch (Stephan Paschedag)
Newsgroups: comp.os.os9
Subject: Ultra C test results (2nd edition)
Date: 22 Nov 92 18:21:17 GMT
Organization: Swiss Federal Institute of Technology (ETH), Zurich, CH

Hi !

From the answers I got to my first posting about comparision of the available
C compilers for OS-9/68k I've extended my tests. I've compiled the same
programs with new options (sieve, dhrystone, whetstone) and I've added some
new tests (espresso, primes which is another version of the sieve benchmark).

  - why is sieve slower when compiled for a 68020 ?
    this seems to happen on a 68030, but not on a 68040.

  - rayshade could not be compiled with Ultra C because the compiler
    generated invalid assembly source ('fadd.l a0,fp0'). This bug could
    be avoided with -O0, but then the program didn't work anymore !

  - Ultra C does not have special math support for the 68040. A system-
    state traphandler has to be used for 68881/2 emulation, which is VERY
    slow. This is the reason why the whetstone benchmark is 10 times faster
    on a 68040 when compiled with gcc (gcc has a special math library for
    the 68040 chip).

  - when dhry V2.0 was compiled using -j (icode linking) the executable didn't
    work anymore (Error #000:102, Bus Error).
    I couln't reproduce this error, maybe the next problem explains this.

  - compiling dhrystone V2.0 with '-tp020 -O7 -r -t100 -s0 -j' failed :
    "be68020: **** internal error - sym not found ****".
    The same happened for compress with '-O7 -tp020 -t5 -j'.
    It seems that this is caused by a combination of the '-j' and '-t' switches

  - compiling espresso with Ultra C and the -j option didn't work. The backend
    either terminated with BusError (NULL pointer access) or with an error
    "I-code buffer exhausted". It didn't help to use the -b option. I think
    that it depends on the memory contents at startup of be68k/be68020 how it
    terminates.

  - when I tried to compile the modules of espresso with -t5 iopt crashed on
    2 files. For one I got a strange error message :
    "read error at -1308643328 in file ucc0800.i"
    When I tried to compile the second module iopt did hang (I've waited
    30 minutes before I interrupted it)

  - for espresso Ultra C generates the slowest code of all three compilers.
    all tries to generate faster code failed (see above)

  - the dhrystone benchmark seems to be the only one that runs faster when
    compiled with Ultra C, for all others gcc (in some cases also cc) seems
    to generate faster code. I'm wondering if Microware has optimized their
    compiler for this benchmark ?

  - where can I find a documentation about the parameters to the -s and -t
    options. compilations using '-t100 -s0' take MUCH longer than compilations
    using '-t5'; why ?

  - whet could not be compiled with 'cc -O=2', c68/c68020 always crashed with
    "**** reg free ****". When compiled with 'cc -O=2 -k2f" c68020 terminated
    with "**** Stack Overflow ****"

  - compiling lharc with Ultra C failed because iopt crashed (BusError).

  - I've not succeeded compiling the jpeg image compression package due to
    some very strange errors from cfe. The preprocesses sources looked like
    this :

       static void
       select_file_type (compress_info_ptr cinfo)

    and I got the following error message :

       "jcmain.c", line 95: **** ) expected ****
        voidselect_file_type (compress_info_ptr cinfo)
                                                 ^

    I'm wondering if a space characted got lost between 'void' and 'select_...'
    If you replace 'compress_info_ptr', which is a typedef', by a simple type,
    the compilation doesn't fail.

  - the new linker l68 (Ed 83) supplied with the Ultra C package also seems
    to have bugs. I got strange errors all similar to this :

    l68: error - symbol name '_jmptbl', defined by psect 'cstart_a',
        file '/dd/lib/cstart.r' already defined by psect 'cstart_a',
        file '/dd/lib/cstart.r'

  - the new r68 (Ed 128) fails to translate correctly 'movem (a7)+,#$1234'

  - xengine could only be compiled when an extra math.l was added to the list of
    libraries because l68 complained about too long offsets in netdb.l.
    It seems that this happened because the new libraries for Ultra C are much 
    bigger than the old ones.

  - xengine shows once more how important it is to have FAST math support for
    the 68040 chip (gcc '-m68040' is much faster than 'gcc -m68020 -m68881'
    or 'ucc -tp020')

  - why is compress so much slower when compiled with Ultra C ??


Result : gcc still generates faster and more compact code than Ultra C.
         Ultra C is a step into the right direction, but still needs many
         improvements. It seems that some parts of the Ultra C system are
         still quite buggy (did Microware give it to beta testers ??)

If there is somebody who gets different results, please let me know. If there
are options for Ultra C that force it to generate faster code, please tell me !


Stephan

------------------------------------------------------------------------------
Program   : sieve  (source : 2058 bytes, 135 lines)
Machine   : MVME147S-1 (68030 25MHz, 1 User, no other active processes)
Date      : 92/10/06
Compilers : ucc V1.0, gcc V1.42, cc V3.2

compiler/options                         comp time   exec time     size
====================================== ============ =========== ==========

ucc -O7                                      31        3.94        25130
ucc -O7 -j                                  392        3.94        24266
ucc -O0                                      18        7.73        25314
ucc -tp020 -O7                               29        4.25        23510
ucc -tp020 -O7 -j                           389        4.24        23190
ucc -tp020 -O0                               17        7.73        23706
ucc -tp020c -O7                              30        4.25        23462

gcc -O                                       17        3.73        18850
gcc -O -ob                                   17        3.70        18806
gcc                                          13        5.93        19266
gcc -O -mc68020                              17        3.73        23030
gcc -O -ob -mc68020                          17        3.69        22986
gcc -mc68020                                 13        5.93        23446

cc -O=2                                      11        4.82        18732
cc -O                                        10        4.86        18776
cc -O=2 -k2wcw                               12        4.82        22960
cc -O -k2wcw                                 10        4.86        23004

------------------------------------------------------------------------------

Program   : sieve  (source : 2058 bytes, 135 lines)
Machine   : KWS cpUnit040 (68040 25MHz, 1 User, no other active processes)
Date      : 92/11/22
Compilers : ucc V1.0, gcc V1.42, cc V3.2

compiler/options                         comp time   exec time     size
====================================== ============ =========== ==========

ucc -O7                                       7        1.49        25418
ucc -O7 -j                                   78        1.48        24786
ucc -O0                                       5        2.24        25602
ucc -tp020 -O7                                7        1.46        23810
ucc -tp020 -O7 -j                            77        1.44        23586
ucc -tp020 -O0                                5        2.25        24006
ucc -tp020c -O7                               7        1.45        23762
ucc -tp020 -O7 -t5                            7        1.43        23810
ucc -tp020 -O7 -t100                         10        1.46        24550
ucc -tp020 -O7 -t100 -s0

gcc -O                                        3        1.22        18854
gcc -O -ob                                    4        1.16        18810
gcc                                           3        2.04        19270
gcc -O -mc68020                               4        1.23        23034
gcc -O -ob -mc68020                           4        1.16        22990
gcc -mc68020                                  3        2.05        23450

cc -O=2                                       3        1.41        18736
cc -O                                         2        1.49        18780
cc -O=2 -k2wcw                                3        1.41        22964
cc -O -k2wcw                                  3        1.49        22990

------------------------------------------------------------------------------

Program   : dhry V1.0
Machine   : KWS cpUnit040 (68040 25MHz, 1 User, no other active processes)
Date      : 92/11/08
Compilers : ucc V1.0, gcc V1.42, cc V3.2

compiler/options                        dhry/sec      size
====================================== =========== ==========

ucc -tp68k -O7                            15663      22926
ucc -tp68k -r -O7                         16537      22542
ucc -tp020 -O7                            20779      21426
ucc -tp020 -r -O7                         22299      18942
ucc -tp020c -r -O7                        22206      21070
ucc -tp020c -r -O7 -t=100                 32820      22934

gcc -O -o68 -ob                           18099      15274
gcc -O -o68 -ob -mnostack-check           21650      15130
gcc -O -o68 -ob -m68020                   21577      18034
gcc -O -o68 -ob -mnostack-check -m68020   26622      17890
gcc -O -ob -mnostack-check -m68020
    -fomit-frame-pointer -fforce-mem
    -fcombine-regs                        30477      17774

cc -k0wcw -q -O=2                         12432      15482
cc -k0wcw -S -q -O=2                      13570      15342
cc -k2wcw -q -O=2                         17259      18238
cc -k2wcw -S -q -O=2                      19716      18098

------------------------------------------------------------------------------

Program   : dhry V2.0
Machine   : KWS cpUnit040 (68040 25MHz, 1 User, no other active processes)
Date      : 92/11/08
Compilers : ucc V1.0, gcc V1.42, cc V3.2

compiler/options                        dhry/sec      size
====================================== =========== ==========

ucc -tp020 -O7                            17247      23260
ucc -tp020 -O7 -r                         18920      22920
ucc -tp020c -O7 -r                        18900      22892
ucc -tp020 -O7 -r -t5                     23431      23044
ucc -tp020 -O7 -r -t100 -s0               26657      23612
ucc -tp020 -O7 -r -t0 -s100               18679      22924
ucc -tp020 -O7 -r -t5 -j                  27042      25306

gcc -O -ob -m68020 -fcombine-regs
    -fomit-frame-pointer -fforce-mem      20800      19082
gcc -O -ob -m68020 -fcombine-regs
    -fomit-frame-pointer -fforce-mem
    -mnostack-check                       25918      18938

cc -k2wcw -O=2                            14256      19536
cc -k2wcw -O=2 -S                         16425      19292

------------------------------------------------------------------------------

Program   : whet
Machine   : KWS cpUnit040 (68040 25MHz, 1 User, no other active processes)
Date      : 92/11/08
Compilers : ucc V1.0, gcc V1.42

compiler/options                        whet/sec      size
====================================== =========== ==========

ucc -O7 -tp020                           4633000     22986
ucc -O7 -tp020 -t5                       4738750     23230

gcc -O -mc68040                         45745600     21436
gcc -O -mc68040 -fcombine-regs
    -fforce-addr                        46332000     21358

gcc -O -m68020 -m68881                   4689900     16730

------------------------------------------------------------------------------

Program   : whet
Machine   : NDR Computer (68030 25MHz, 68882 25MHz)
Date      : 92/11/08
Compilers : ucc V1.0, gcc V1.42

compiler/options                        whet/sec      size
====================================== =========== ==========

ucc -O7 -tp020                          12180200     22986

gcc -O -mc68040                          8631100     21436

gcc -O -m68020 -m68881                  13372500     16730

------------------------------------------------------------------------------

Program   : primes (sieve)
Machine   : KWS cpUnit040 (68040 25MHz, 1 User, no other active processes)
Date      : 92/11/14
Compilers : ucc V1.0, gcc V1.42

compiler/options                         comp time   exec time     size
====================================== ============ =========== ==========

ucc -bc -O7                                  4         1480ms      21204
ucc -bc -O7 -tp020                           4         1440ms      19844
ucc -bc -O7 -t=5                             4         1500ms      21292
ucc -bc -O7 -t=100                         634         1460ms      28148

gcc -O                                       2         1170ms      13868
gcc -O -mc68020                              2         1170ms      13868

------------------------------------------------------------------------------

Program   : espresso (logic minimization)
Machine   : KWS cpUnit040 (68040 25MHz, 1 User, no other active processes)
Date      : 92/11/21
Compilers : ucc V1.0, gcc V1.42, cc V3.2

compiler/options                         comp time   exec time     size
====================================== ============ =========== ==========

ucc -O7 -tp020                             790         6.95       145848

gcc -mc68020 -ob -fcombine-regs
    -fforce-mem                            167         5.86       114596

cc -O=2 -k2wcw                             127         6.27       133678

------------------------------------------------------------------------------

Program   : xengine (X11R4 server compiled with gcc)
Machine   : KWS cpUnit040 (68040 25MHz, 1 User, no other active processes)
Date      : 92/11/22
Compilers : ucc V1.0, gcc V1.42, cc V3.2

compiler/options                          speed       size
====================================== =========== ==========

ucc -tp020 -O7 -t5                      122.4 rpm    484028

gcc -m68020 -m68881 -O                  123.4 rpm    471998

gcc -m68040 -O                          135.2 rpm    475334

cc -k2f -O=2                            123.1 rpm    474334

------------------------------------------------------------------------------

Program   : gnu-compress 4.0 (times for a 600k file on RAM-disk)
Machine   : KWS cpUnit040 (68040 25MHz, 1 User, no other active processes)
Date      : 92/11/22
Compilers : ucc V1.0, gcc V1.42

compiler/options                            compress    decompress
========================================== =========== ============

ucc -O7                                       33.10        22.80
ucc -O7 -t5                                   32.57        22.97
ucc -O7 -t5 -tp020                            20.60        14.09
ucc -O7 -tp020 -j                             21.40        14.65

gcc -O                                         8.36         5.38
gcc -O -m68020                                 7.98         5.74

cc -O=2 -k2wcw                                 8.09         5.79
Newsgroups: comp.os.os9