From: Chuck.Phillips@FtCollinsCO.NCR.COM (Chuck Phillips)
Newsgroups: comp.sys.hp,comp.unix.internals,comp.lang.c
Subject: Re: Decreasing brk value
Date: 7 Oct 92 08:45:43 GMT
Organization: NCR Microelectronics, Ft. Collins, CO
In-reply-to: heidi@ucthpx.uct.ac.za's message of 30 Sep 92 10:37:25 GMT

>>>>> On 30 Sep 92 10:37:25 GMT, heidi@ucthpx.uct.ac.za (Heidi de Wet) said:

Heidi> ... a lot of dynamic memory is being allocated.  The problem is that
Heidi> after it's been freed, the process is left with a high brk value, and
Heidi> when a couple of applications have done this the system starts falling
Heidi> over due to lack of available memory.  Processes sometimes have as
Heidi> much as 80MB standing idle.

Without access to the application source, specificly to the part where memory
is temporarily malloc()ed and free()d, there isn't a whole lot you can easily
do, short of restarting the application.  You're in a particulary bad
situation in that you're linking in vendor's code (without source I assume)
which does its own malloc()ing.

If you *do* have access to source, check out GNU's mmalloc(), an mmap()-pool
based allocator.  (Part of the latest GDB distribution.)  If the copyleft
gets in your way, almost any PD malloc() implementation could be converted to
use mmap(), instead of s/brk() memory with some effort.  (I would recommend
using the GNU API -- unless your're worried about a "look-and-feel" lawsuit.
;-)  If you're willing to forgo a lot of convenience, you can call mmap()
directly without bothering with a malloc()-like interface.

The mmap(), et al, API is suprisingly portable; almost every modern UNIX
variant has it, e.g., SunOS 4.1, HP-UX A.08.07, AGEIS, SVr4, OSF/1.  AGEIS
hadn't implemented the Copy-On-Write (COW) feature when I last checked, but
COW isn't necessary for simple memory allocation.  Although it wasn't fully
implemented in the initial BSD 4.3 release, I *suspect* newer versions of BSD
have completed the implementation by now.

Database applications in particular could realize performance gains by using
calling mmap() directly instead of of duplicating the functionality only to
have the OS call mmap() anyway without the application's knowlege or control.
In your specific case, memory needed for a large query could be allocated
from one mmap() pool during the query, then freed in one quick (compared to
many free()s) munmap() upon completion, returning the memory to the OS in the
process.

If the mmap() API allowed an application to switch between COW and
pre-modified pages, and further, allowed the application to apply COW changes
to the original on demand, it could really reduce the overhead required to
implement the rollback/norollback algorithms needed to assure DB consistancy
in multi-user DBs.  (Hint, hint. :-)  If someone has a work-around to achieve
this using the present API, *please* post.


Heidi> Decreasing the brk value manually seems to be the answer, but dynamic
Heidi> memory is being used elsewhere as well, and malloc will fall over.

Even if no memory in the freed area is actually used, malloc() probably
includes the now-missing memory in its list of available memory.


	Good luck!
--
Chuck Phillips  MS440
NCR Microelectronics 			Chuck.Phillips@FtCollinsCO.NCR.COM
2001 Danfield Ct.
Ft. Collins, CO.  80525
Newsgroups: comp.sys.hp