From: stevo@elroy.Jpl.Nasa.Gov (Steve Groom)
Newsgroups: news.software.b,news.software.nntp
Subject: INN problems: sendbatch, innwatch, NOFILE_LIMIT
Date: 12 Jan 1993 23:03:36 GMT
Organization: Image Analysis Systems Group, JPL
Distribution: world
NNTP-Posting-Host: judy.jpl.nasa.gov

I recently upgraded elroy.jpl.nasa.gov to INN 1.2 and encountered
several problems.  The first few involve site/sendbatch.  The
next is a request to support better notification when innd is
throttled because of some problem (disk full, etc).  The last is a
query about how to deal with the "Too many open files" problem
on SunOS 4.1.x.

I know many people don't do UUCP batching any more, but we do.
I had some trouble getting site/sendbatch to not
overflow our outgoing UUCP spool.  It would just keep spewing UUCP batches
as long as the input batchfiles were there.  It turns out that all
the appropriate disk space checks are in the sendbatch script, but they don't
work.  There are a couple of awk commands invoked by using ${AWK},
but that variable is not set anywhere in that script even though it is used.
The fix was to duplicate the line in that script where SED is defined
(and picked out of the config file), changing it all to AWK.  That seemed
to fix it, but I was surprised that such a significant problem existed.
Before I fixed it, the script choked on the awk code but just ignored it,
so it proceeded to fill my disks - a couple of times before I figured it out.

The next problem with site/sendbatch is that the UUSPOOL directory
must actually be the mount point for that filesystem, and not the
actual spool directory.  The problem is that the disk space checks use awk
to match the df output against UUSPOOL, so unless they match
the check won't work.  I just changed my UUSPOOL to be the mount
point, but it would be nice if it could just do a df on the specific
filesystem, strip the df output header, and use whatever's left as the
answer, as in "df ${UUSPOOL} | tail +1 | awk ......"  I don't know how
portable this is, though.

The next problem with site/sendbatch is that although there is
apparently support in that script to limit how many bytes to batch,
the calculated value is never actually passed into batcher.
There is also the problem of the BYTESQUEUED and DEFBYTES values
representing the size BEFORE compression.  DEFBYTES is adjusted
for compression, but BATCHBYTES (derived from BYTESQUEUED) is not.
After adding the adjustment of BATCHBYTES, I inserted "-B ${BATCHBYTES}"
on the line where batcher is invoked, and this part started working.

During all of these problems, I kept discovering that innwatch (or
innd itself) had dutifully throttled innd.  The problem is, it didn't bother
to tell me.  There is so much noise spewing out through syslog right now
I will never see any of those messages until I get something in place
to filter out the REAL errors.  This is probably just a configuration issue
on my part, but I have a request.  Would it be possible to support
notification by mail to the news admin whenever one of these serious
conditions was encountered?  I keep discovering problems because some
user calls and says he's got some strange error message, which innd
is telling him but that I haven't noticed yet.  I noticed that innwatch
has a comment in it that no notification is done except through syslog,
but I think it would be wise to support a more direct method of notification
instead of just logging the errors.  Any takers?

The last problem is one that is documented briefly in the install notes,
and that is the Sun bug with "Too many open files".  This
hit me twice this morning, and I don't know what caused it, but when
it happened innd just throttled itself and sat there.  How do I
set up my system so that this doesn't happen?  When it does happen,
what is the "proper" way to recover?  If I set NOFILE_LIMIT to a fixed
number less than the threshhold of disaster, will innd properly
stay under that limit, or will I just have a different type of
disaster when innd tries to cross it?

On the good side, overall I am impressed with how tight everything seems.
There are obviously some holes and some weaknesses, but it looks
like it hangs together very well.  Thanks, Rich.

-steve
-- 
Steve Groom, stevo@elroy.jpl.nasa.gov		Oxymoron of the year:
Jet Propulsion Laboratory, Pasadena, CA		      "OpenVMS"