X-Vms-Mail-To: uu%"BANYAN-L@VM1.CC.UAKRON.EDU",HARAKKA_J
Newsgroups: bit.listserv.banyan-l
Date:         Thu, 15 Oct 1992 22:11:41 EET
From:         Juha Harakka SALCOM Finland Oy <harakka_j@SALCJKL.SALCOM.FI>
Subject:      Mystical server hangs (lengthy)

Hello all,

I have got a though problem and hope that someone could give some
insight to the problem.

One of our customer have started to seee problems in their VINES network.
Their network consist of two servers on two different sites that are
bridged together with 64 Kb line.  On the 'main' site they currently
have 20+ users and the other one have some 15 users.

The problem is that the server at the main site have started to
mystically 'hang' every now and then.  When it hangs, nobody can log in
(actually one can't see anything on the Z: drive), but if one looks
at the console, all services seem to be running OK. However, if one
looks the network management menu, the other server can't be seen.
So it seems that the server stops to communicate with the rest of world.
But on the other hand, when user tries to log in, She/he DON't get
Z: drive from the other server as I understand would be the case, if the
server were totally unavailable.  Only solution seems to be to reboot
the server.

The server is Compaq SystemPro/LT with 16 Megs and is connected to Ethernet
with Intel card that looks like a Novell NE3200 EISA card (they had
the problem allready when using 3Com Etherlink II).  There's quite a many
services on the server: 13 or so file services, 8 print services (which
are queues to PCPRINT printers on workstations), Mail service, SMTP Gateway,
STDA and NetBIOS (and TCP/IP routing).  We calculated the amount of memery
that those services should require (from the 4.11 release note) and ended
up a little bit under 6 Megs.  So memory shouldn't be problem, right?

I investigated the server logs after the 'hang' and found from system
error log at the time of 'hang' following error messages:

Console message at Thu Oct 15 10:25:21 1992

[16;1Hprocess_rx(): cannot get mbuf

Can somebody explain what this means? Out of some buffers, I suppose?

In the SS-service log there's messages:

10:14:42 [256] SS: SvrError: 163
10:53:17 [512] Starting termination sequence

So the last error before reboot (and before those process_rx()... )
is error 163 for which VNSERR.EXE gives the following explanation:


********   Help Description   ********

COM163 -- TIMEOUT EVENT.

   The time limit on a socket receive or wait request expired
   before a message was received.

   This error also occurs when a reliable IPC send request
   does not complete within a reasonable amount of time.
   Each Remote Procedure Call also uses a similar time limit
   for receiving the reply.  The call returns this error
   if the service takes too long processing the request.



End of help...

Meanwhile in VANGuard log:

10:11:11 [3] Login user=MHONKO@ETU@VAPO, stamp=3581
10:11:11 [3]    Confirm Port=00 03 00 20 24 3d 80 27 00 03 00 00 00 00 00 00
10:11:11 [3] @ (BANTH,3-0,02608ca7ed33)
10:25:23 [256] SvrError called. Event Code: 162 Client Port: 00 03 00 20 24 3d
00 01 02 10 00 00 00 00 00 00

The user login above (according the customer) is one of the failed logins
and the time is roughly sama as the time of the Timeout-error above.
Also 10:25:23 is the same time as the timestamp for those process_rx() errors.

Explanation for error code 162 is as follows:


********   Help Description   ********

COM162 -- NO BUFFER SPACE AVAILABLE.

   The communication layer did not have enough memory space
   to process the request.  This error can result from local
   conditions, which include a temporary heavy communication
   load, a communications buffer size configured at too low
   a value, or wasteful use of socket calls.  You may need
   to increase the communications buffer size for the server
   or PC, or close unneeded sockets.

   A process can also receive this error when sending data
   to a process whose input buffer is full.  The receiving
   process in this case is too busy to handle the data flow.
   You should retry the request later.  If the condition
   persists, the receiving process could be hung.



End of help...

That led me to check the Communication Limits values.  The Server Operation
Guide tells how to calculate the total communications buffer size, IF you
know the max concurrent SPP connections and max concurrent open sockets.
But to determie these "...you should be familiar with all communications
aspects of your server, including the number of users and services."  But
nothing how to ACTUALLY calculate those values.  Assuming that one user
connecting to one file service requires (at least) one SPP and reserving
some for the services to communicate with each other, I came to conclusion
that the number of SPP's could be the problem.  Tonight we changed the
number of SPP's from the default 300 to 400 and the number of sockets
from 200 to 300 (which is maximum) also increasing the total amount of
communication buffers from 60 000 to  70 000 bytes (and now we wait if
the problem still persists).  Does this make any sense?

Is there still something to check out?

Any help and insight greatly appreciated (the customer is getting
nervous; today they had to boot the server three times!).

					- juha -



----Juha Harakka, SALCOM Finland OY, Box 37, SF-40321 Jyvaskyla      -
---Tel: +358-41-685 333 & +358-49-643 319, Fax: +358-41-677 944     --
--X.400:c=fi;admd=fumail;o=salcom;s=Juha.Harakka,BIX: jharakka     ---
-Internet: Juha.Harakka@salcom.fi PSImail: 244324100014::Harakka_J----
Comments: Gated by NETNEWS@AUVM.AMERICAN.EDU