From: fmrco!msommer@uunet.uu.net (Mark Sommer)
Subject: System Mode Swamp on Sparc 2
Organization: Fidelity Management & Research Co.
Distribution: usa
Date: Thu, 7 Jan 1993 14:35:47 GMT

A program of mine, for some unknown reason, spends an inordinate
amount of time in system mode. Any insights or suggestions would be
welcome. 

My program calls a Fortran function to solve a large quadratic
programming problem. The program is extremely computationally
intensive and it runs very large (>10MB), but my machine has 48MB of
memory and lots of swap space (>80MB), so I would have expected it to
spend most of its time in user mode. In fact, according to many
indicators (such as gprof), it spends most of its time in kernel mode.
I've included information I think might be relevant.

This is what ps says about my program:

  ampere% ps -u
  USER       PID %CPU %MEM   SZ  RSS TT STAT START  TIME COMMAND
  msommer  22245 83.3  9.510232 4436 p1 R    15:07  27:51 portopt -p /tmp/port-222

I was first alerted to the problem when I "time"d a script (porto)
that runs my program (portopt). (Presumably, the "time" command
tabulates for all porto's child processes.) The 164 page faults
and 0 swaps seem reasonable to me:

  ampere% time porto > opt
  187.240u 3576.250s 1:17:39.44 80.7% 0+2251k 11+47io 164pf+0w

Pstat says I'm using all my physical memory, but have lots of virtual
memory left:

  ampere% /etc/pstat -s
  20260k allocated + 10172k reserved = 30432k used, 49844k available

According to "trace -p", my program doesn't do any system calls during
the bulk of its runtime.

Here's what vmstat says as portopt is running. Notice, there doesn't
seem to be an unusual amount of paging or swapping, yet the cpu's "sy"
column is pegged:

 procs     memory              page               disk       faults     cpu
 r b w   avm   fre  si so  pi  po  fr  de  sr s0 d1 d2 d3  in  sy  cs us sy id
 1 0 0     0  2272  10  0   0   0   0   0   0 46  0  0  0 214 264  44  6 94  0
 1 0 0     0  2272   5  0   0   0   0   0   0  3  0  0  0 114 163  34  7 93  0
 1 0 0     0  2272   5  0   0   0   0   0   0  0  0  0  0  80 190  42  5 95  0
 1 0 0     0  2272  10  0   0   0   0   0   0  0  0  0  0  60 181  39  5 95  0
 1 0 0     0  2272   0  0   0   0   0   0   0  0  0  0  0  65 198  43  5 95  0
 1 0 0     0  2272   0  0   0   0   0   0   0  0  0  0  0  72 260  53  6 94  0
 1 0 0     0  2272  10  0   0   0   0   0   0 46  0  0  0 203 180  41  7 93  0
 1 0 0     0  2272   5  0   0   0   0   0   0 13  0  0  0 131 152  36  8 92  0
 1 0 0     0  2272   0  0   0   0   0   0   0  0  0  0  0  73 137  34  6 94  0
 1 0 0     0  2272   0  0   0   0   0   0   0  0  0  0  0  56 141  34  6 94  0
 1 0 0     0  2272   0  0   0   0   0   0   0  0  0  0  0  51 114  31  6 94  0
 1 0 0     0  2272   0  0   0   0   0  16   0  0  0  0  0  42  92  26  4 96  0
 2 0 0     0  2272  10  0   0   0   0   0   0 47  0  0  0 182 154  37  5 95  0
 1 0 0     0  2272   5  0   0   0   0   0   0  2  0  0  0 126 171  39  6 94  0
 1 0 0     0  2272   5  0   0   0   0   0   0  0  0  0  0  77 178  41  6 94  0
 1 0 0     0  2272  10  0   0   0   0   0   0  0  0  0  0  64 184  40  3 97  0
 1 0 0     0  2272   0  0   0   0   0   0   0  0  0  0  0  51 163  37  4 96  0
 1 0 0     0  2272   0  0   0   0   0  16   0  0  0  0  0  60 193  41  3 97  0
 1 0 0     0  2272  10  0   0   0   0   0   0 47  0  0  0 164 140  34  7 93  0


Can anyone explain what's going on?
--
Mark Sommer
Fidelity Management and Research