From: jmb@patton.wpd.sgi.com (Jim Barton)
Subject: Re: Socket programming and select()
Organization: Silicon Graphics Inc.
Date: Mon, 12 Oct 92 16:01:26 GMT

In article <1992Oct8.132715.10324@aio.jsc.nasa.gov>, you write:
...
|> Timing tests show that select() behaves erratically if we want to do
|> this quickly (less than 20 milliseconds)...
...
|> We are wondering:
|> 
|>    1) Is this behavior indicative of a BUG in select()?

Not necessarily. You don't say what else is going on on the system at the same
time. For instance, if several compute jobs are running, they may have a 
higher priority than yours, or they may have just started a time slice when
the byte is delivered to the receiver. The time slice is 30 ms, which 
could result in a large chunk of the variation. One thing to keep in mind
is that the kernel is largely driven by a 10 ms clock for synchronization,
and most timing, stream pushes, socket pushes, wakeups, etc., rely on this
clock. You also don't mention if the receiving machine is currently paging
or handling heavy disk or network traffic, all of which have an effect.

Not being exactly sure of your setup, I'm also not clear if you've taken
network delays into account. These can be highly variable. For example, an
Ethernet segment that's noisy (runs too long, bad transcievers, etc.) can
result in dropped packets, and TCP retries take awhile.

|> 
|>    2) Does the select call use gettimeofday() to determine when it's 
|>       waited long enough (this seems possible since 10ms intervals are the 
|>       only useful settings for the timeout values)? 

See the answer above. Select translates the timeval structure into kernel
clock ticks, thus only a 10 ms resolution is possible.

|> 
|>    3) Can anyone suggest a better way for determining if the graphics
|>       are ready on a remote process/machine?

Not really. However, one way to reduce the variability on the receiving side
is to set the receiving process at a high non-degrading priority, basically
make it a "real-time" process (see schedctl(2)). Giving a process such
a priority changes the way the kernel handles it - for example, when that
process becomes "ready-to-run" (i.e., the byte arrived), it will be
dispatched immediately instead of waiting for the current time-slice to
end. These priorities were designed for your kind of problem.

Also, one should be careful about memory. If paging activity is expected,
the process waiting for the byte should be locked in core (see plock(2) or
mpin(2)) so that it always remains loaded and ready to run.

Finally, if your compute job is the one that checks for the remote byte
to arrive, perhaps it would be best to create a separate thread (see
sproc(2)) to simply wait for the byte to arrive, and then send a signal,
change a shared memory variable, or use some other mechanism to communicate
readiness.

I know of other customers who have done similar things and
succeeded. Sometimes, though, you have to go to great lengths to manage the
network. For example, a dedicated ethernet. I have also seen raw ethernet
access used (to avoid TCP/IP overhead), and even custom ethernet drivers
to shave that last few microseconds off the response time.

-- Jim Barton
   Silicon Graphics Computer Systems
   jmb@sgi.com
Newsgroups: comp.sys.sgi