From: hedrick@athos.rutgers.edu (Charles Hedrick)
Newsgroups: comp.sys.sun.admin
Subject: Re: Sun 690MP performance problems
Date: 29 Aug 92 22:16:20 GMT
Organization: Rutgers Univ., New Brunswick, N.J.

iglesias@draco.acs.uci.edu (Mike Iglesias) writes:

>Earlier this year, we purchased a 4 processor Sun 690MP system to replace
>an aging Sequent Symmetry system as our general Unix timesharing service
>and campus news server (using nntp and c-news).  We have 2 IPI disks on the
>system, with swapping on both and the users space taking up most of one of
>the disks.  News is currently on a Sun 600mb SCSI disk connected to a
>SCSI/Ethernet SBUS card. 

>Since we cut over to the 690MP, we have had unsatisfactory performance on
>the 690MP, both for the timesharing users and the people reading news via
>nntp. 

I assume you understand the basic problem: 4.1.x is not a full
symmetric multiprocessor kernel.  Only one processor can be in kernel
code at a time.  In typical timesharing use, it's common to spend at
least 50% of the CPU in system code.  If you try to do that on a
4-processor 670 or 690, you'll lose, because you can only get 25%
system time (one of four processors).  This doesn't necessarily mean
that vmstat will never show more than 25%, since when you get into
this situation, the processors are going to start waiting, and that
may well be reported as system time.

Sun should not have sold a 4-processor system for heavy general
timesharing usage.  We use a 4-processor system as our main multiuser
system in the computer science dept, but when usage gets heavy, there
are always lots of Lisp and other compute-bound jobs just using the
CPU.  We had been getting slightly slowdowns in response during
periods of heavy load (though I think I'm the only one that noticed
it), but the most recent MP performance patch seems to have fixed
that.  However when we get the system, we knew what its limitations
were.  I configured it with NC400's for each Ethernet, to get NFS
processing out of the kernel (since this is a major file server for
us), and made sure that our users knew it was fine to use this system
for their long-running CPU-bound jobs.  (We normally try to keep those
jobs off our interactive machines -- this is one case where I *want*
CPU-bound jobs.)  We've been lucky.  I'm really pleased with the
system.

There are at least two more on campus.  One is used for CAD work.
Their software is set up so that it can use all 4 processors, and is
primarily CPU-bound.  A perfect application.  The third is a general
campus timesharing system, used much as yours is.  I'm nervous about
it.  So far no one has reported trouble, but I think it's at risk.  If
it's OK, I suspect that's because we don't have large numbers of
users, and there are a lot of people doing large numbercrunching jobs
on it.  But I think it's in danger.  We do understand that, and will
either go to Solaris 2.0 or to a single-processor Viking as soon as
possible.  In general I'd have to say that we're very happy with the
4-processor systems.  But if we had tried to use one to replace our
general-purpose campus VMS system, we would have ended up with a
disaster.

One thing you might try is removing two of the processors.  It's much
less common to get a 2-processor system into trouble.  Of course if
you have to have the full power of all 4 processors to handle your
load, you lose.

I think Sun's "license to speed" promotion caused a lot of this.  It
caused lots of people who would otherwise have gotten 1 or 2 processor
systems -- which would have worked fine -- to end up with 4 processors
-- which do not work in all applications.  The one thing you can say
for the promotion is at least you can't claim you *paid* for something
you can't use.