From: vancleef@netcom.com (Henry van Cleef)
Subject: Three compilers compared on Unix 3.2v4 (long)
Keywords:  cc rcc gcc
Organization: Netcom--posted from Andover, Mass.
Distribution: biz
Date: Sat, 2 Jan 1993 23:31:30 GMT

I decided to put three compilers to the test on my 3.2v4 system.  GNU
tar 1.11.1 looked like a good test vehicle for this test.  It has a good
mix of programming styles and tasks, and includes GNU's regex.c with
with its huge macros.  The three compilers tested were:

1. GNU gcc 2.3.3, with flags -posix.  GCC 2.3.3 had been cross-compiled 
with itself several times, last compilation was with -O.  

2. (Microsoft) cc, with flags -posix.  This is the sls update
compiler, not the original one.

3.  (AT&T) rcc, with flags -D_POSIX_SOURCE -D_NO_PROTOTYPE -Di386
-DNULL=0

Retest with optimization. Tests 4 5 6 are the first 3 with -O added.

This was done with a shell script to keep things reasonably constant.
The test machine was a 386-33, 64k cache, 16mb core ram, Micropolis 1624
670mb disk, Adaptec 1542b controller, idle except for this script. 
Configurations were "out of the box" except as noted.  Headers did NOT
have fixinc.sco run on them.  <stdio.h> uses the 3.2.2 devsys macros for
putc/getc, and <sys/errno.h> has the ifdefs preventing inclusion of
error numbers disabled.  

The shell script makes tar, moves it to the user root directory.  It then
tars up the tar-1.11.1 directory to the user root and untars it into
another directory.  A test routine compares (cmp) all the files in the
tar-1.11.1 directory with those in the untar test directory as a quickie
q/a test.  Here is the data, gathered by the shell script.  

compile times, unoptimized compile
GCC-2.3.3		(Micro) cc		(ATT) rcc
real     3:02.3 	real     4:18.3	 	real     1:50.0
user     2:10.8 	user     3:13.5	 	user     1:07.9
sys        22.5 	sys        31.6		sys        20.4
tar up times for directory contents (tar size shown)
1177600 test.tar.0 	1146880 test.tar.1 	1167360 test.tar.2
real       10.4 	real        9.9 	real       10.5
user        0.1 	user        0.0 	user        0.0
sys         2.0 	sys         2.1 	sys         2.0
untar times for tar just generated
real        9.5 	real        8.4 	real        8.5
user        0.1 	user        0.1 	user        0.0
sys         2.2 	sys         2.2 	sys         2.7
Object file sizes from ls -l and size
tar.0	157652 
131384 + 11624 + 7316 = 150324
tar.1	151128 
115056 + 23828 + 7320 = 146204
tar.2	1167360 
107808 + 25728 + 6628 = 140164

compile times with optimization (same as above plus -O flag)
GCC-2.3.3		(Micro) cc		(ATT) rcc
real     3:41.9 	real     5:46.8 	real     2:48.5
user     2:53.8 	user     4:40.2 	user     1:57.9
sys        22.7	 	sys        30.8 	sys        24.0
tar up times for directory contents (tar size shown)
1146880 test.tar.3 	1126400 test.tar.4 	1157120 test.tar.5
real       10.1 	real        9.4 	real       10.1
user        0.0 	user        0.1 	user        0.1
sys         2.3 	sys         1.8 	sys         2.0
untar times for tar just generated
real        8.4 	real        7.9 	real        9.9
user        0.0 	user        0.1 	user        0.1
sys         2.3 	sys         2.2 	sys         2.3
Object file sizes from ls -l and size
tar.3	1146880 
109284 + 11624 + 7316 = 128224
tar.4	1126400 
97584 + 23828 + 7320 = 128732
tar.5	1157120 
101096 + 25728 + 6628 = 133452

What does one make of all this?  I'll discount the notion that there is
any meaningful difference between the performance data for the tar
objects.  Tar itself is very dependent on i/o performance.  What stands
out to me is that rcc can hold its own.  The compile times for the three
compilers can't necessarily be taken at face value.  Rcc does not
perform the full function prototype lint checking that the other two
compilers do.  Cc even on an unloaded system is more likely to be i/o
bound than gcc.  Nevertheless, cc's 50% greater use of system resources,
both in user and sys space, is a very real price to pay.

Conceptually (and demonstrated by the figures above) a tar program ought
not need performance optimization.  Gcc is the one of the three
compilers where there is a real payback in text size for optimizing.  The
4:1 ratio between cc's resource use optimized and rcc's unoptimized is
outstanding, considering the small differences between binary and load
image sizes.  

SCO doesn't support GCC, and GNU's support of SCO has never been their
strong suit---indeed, it took a good deal of wizardry to get 2.3.2 to do
anything on 3.2v4 besides squawk about the headers.  I ran these tests
to get some feel for tradeoffs in compiler selection for porting.  And
of course, I am not discussing a lot of the tradeoffs here. I'll post
this for what it's worth.      





-- 
Hank van Cleef  vancleef@netcom.com vancleef@tmn.com  Andover, Mass.
Unix systems consultant---kernel, device drivers, networking, X11
SysV, BSD, Sun, Ultrix, AIX, SCO. Porting a specialty. 
	     The Union Institute  History of Science