From: cpcahil@virtech.uucp (Conor P. Cahill)
Subject: Re: Looking for source for instrumented malloc()
Keywords: malloc_debug
Organization: Virtual Technologies Inc.
Date: Wed, 7 Oct 1992 11:34:53 GMT

In article <elliott.718402570@cs.wisc.edu>, elliott@veronica.cs.wisc.edu (James Elliott) writes:
> Does anyone have source to a version of the malloc library that
> supports debugging (i.e. heap integrity checking after each allocation
> and free, and on demand at other times)? The malloc available to me in
> Ultrix seems to lack these features, and they would be very helpful to
> me in tracking down a bug that has manifested itself in code I'm
> working on.

I recently (early Sept) posted a copy of dbmalloc which performs the tasks
that you require in comp.sources.misc.  We also support a commercial product
called SENTINEL which performs these tasks, but it is not yet available
for Ultrix (however, it will be available by the end of the year).

Anyway, here is a comparison of dbmalloc, SENTINEL, and Purify (a product 
that is only available on Sun OS at this time).  If you would like additional
information send email to info@vti.com.



This document compares the pertinent features between three memory
allocation debugging tools (SENTINEL, dbmalloc, and Purify).  Both
SENTINEL and Purify are commercial products, while dbmalloc is a free
product that was posted to the comp.sources.misc USENET newsgroup.


*) licensing

	dbmalloc is freely redistributable and only has a few restrictions
	on its use including a) you can't remove the copyright, b) you
	can't sell it as part of a commercial product or base a commercial
	product intended to perform the same function upon it.

	SENTINEL is licensed by system (the old kind of licensing) and
	has no copy protection/license manager built in to it. There is
	no limit on the number of users on the system that can link with
	the product.  Once the program is linked, there are no limitations
	on the distribution of the program.

	Purify is licensed by the user.  A user is a) somebody linking
	with the product or b) someone using a program linked with the
	product.  All accesses to the product by the same user at the same
	display on the same system count as a single licensed user. 

	Purify manages this limitation via the use of a license manager
	(you need one of those host-id based passwords to install the
	software).

*) pricing	

	dbmalloc is free (costs $0)

	SENTINEL costs approx $195 for 386 UNIX, $395 for Sun OS,
	and $495 for HP/UX for the first copy.  After you buy that 
	copy, the 2nd through 10th copies are discounted 30%, the 11th
	through 20th copies are discounted 60%.  After 20 copies, you 
	have bought a site license for that particular architecture. 
	Therefore, a site license for SunOS would cost approx $4488
	($395 + $251.30 * 9 + $158 * 10).

	Purify costs approx $2,750 for one license, $11,750 for five 
	licenses ($2,350 per license for 2-5 licenses), $20,500 for
	ten licenses ($2,050 per license for 6-10 licenses) and
	$39,000 for 20 copies (something like $2200 per user when you
	buy 10 or more.

*) portability

	dbmalloc works on just about any UNIX platform (although some
	tweaking of the code might be necessary)

	SENTINEL currently supports 3 hardware platforms and additional
	ports are in progress and will be announced when available.

	Purify currently only works on Sun Sparc platforms.  I do not
	know what the plans are for other platforms.

*) how they work

	dbmalloc and SENTINEL work by replacing the system malloc library
	and many of the libc data area access functions (like memcpy, 
	strcpy, etc).  This allows them to capture quite a bit more than
	a standard malloc debug library.

	Purify pre-processes the object code for the program and inserts
	calls to its own functions for every data area access.  These 
	functions maintain a set of bits for each memory location which
	indicate whether or not the data area is accessible and whether
	or not it has been initialized.

*) installation

	dbmalloc comes in source form.  You must configure it for your
	system and compile it.  The Configure script will automatically
	configure the software for most host environments.

	SENTINEL and Purify come as binary ready-to-use distributions and
	can be installed anywhere on the disk

*) tech support

	dbmalloc has no technical support, but you do have the source.

	SENTINEL and Purify both come with a 1 year of phone/email 
	based technical support/product upgrades

*) documentation

	dbmalloc comes with a 40 or so page man page which describes
	most of the functionality 

	Although SENTINEL and Purify both come with approx 150 pages of
	documentation the output from the programs has enough detail 
	so that you would probably not refer to the manual once the 
	product is installed (other than to look up a configuration
	parameter or two)

*) how you use them

	dbmalloc will work by just linking the library, although if you
	use it this way you only get a message that says you overran a 
	buffer.  If you re-compile your code, you will also get a message
	detailing the location of the offending line of code.  This need
	to recompile has caused some problems because dbmalloc does it's
	magic by #defining all of the malloc functions and many of the 
	libc functions.  This causes a problem when you have local 
	definitions of the functions or a variable defined with the same 
	name.

	SENTINEL works by just adding -lsent to the link step in building
	the program.  There is no need to recompile the software.

	Purify works by prepending the link statement with "purify" so
	instead of saying "cc -o t t.o" you would say "purify cc -o t t.o"

*) configuration

	dbmalloc is mostly configured at compile time, although it does
	have a series of run-time environment variables that can be used
	to configure some of its options.

	SENTINEL uses a layered configuration methodology.  A system 
	wide configuration file is read, followed by $HOME/.sentinel, 
	followed by .sentinel  in the current directory, followed by
	environment variables.  Each level takes precedence over the
	prior level.  

	Purify uses a method similar to SENTINEL.

*) error reports
	
	dbmalloc has a short message identifying the problem and if the
	offending code was recompiled with the malloc.h included with 
	dbmalloc, it will usually have the file name and line number of
	the offending line of code.

	SENTINEL and Purify include a more detailed message about the 
	problem and where the problem occurred, including a full stack
	context (including function name, and  file name/line number if
	the program was compiled with -g) of where the error occurred,
	where the data area was allocated (if it is an allocated data
	area) and where the data area was freed (if it had been freed)

*) detecting malloc boundary overwrites
*) access to freed data areas
*) duplicate frees

	Both dbmalloc and SENTINEL trap most calls to libc and check to
	see if the data area to be accessed is properly within the 
	boundaries of a malloc region.  On these calls (some of the most
	frequent overwrite sources) you will get an immediate report
	on the problem before it occurs.  Other problems such as an array
	index beyond the end, will not be trapped immediately but will be
	trapped the next time that data area is accessed or when the
	program exits.  Some of thes problems, especially read access
	problems can go undetected.

	Purify catches all data area overwrites immediately.

*) uninitialized data area access

	dbmalloc fills data areas with known values that are not typical
	data values and usually cause a problem when a program attempts to
	use them.

	SENTINEL also fills areas, but it also recognizes pointers from 
	uninitialized areas and generates a warning explicitly from them.

	Purify catches all access to uninitialized allocated data areas
	at the point that they are being accessed.

*) memory leak detection

	dbmalloc has support for tracking leaks.  The programmer must call
	the leak detection routines at symetric locations within the 
	program being debugged and dbmalloc will report all data that was
	allocated between the checkpoints. Each and every leak (even those
	from the same location) is included in the report reguardless of
	size.

	SENTINEL has automatic support for tracking leaks and will
	generate a report on the leaks present when the software exits. 
	The leak report includes all data areas that have been allocated
	by the program but have not been freed.  SENTINEL realizes that
	there are some allocations that are considered "permanent" and
	should not be included in a leak report and therefore allows you
	to stamp an allocation as "not a leak". SENTINEL accumulates all
	leaks from a particular location in the code and reports this as
	a single leak with the accumulated size as the size of the leak.
	A configuration parameter is available to set the threshold for
	leak reports. A second configuration parameter is available which
	disables the accumulation of leaks and reports each leak
	individually.

	Purify has automatic reports for leak detection and the reports 
	are similar to those of SENTINEL.  However, Purify uses a
	different mechanism for determining whether or not a data area
	has been leaked.  At the time a leak check is performed, Purify
	scans the entire address space and looks for pointers to allocated
	data areas. Whenever it finds a pointer to the beginning of the
	data area it figures that this is not a leak since it could still
	be referenced.  If it finds a pointer to the middle of a segment
	it reports that as a potential leak since it can no longer be
	freed and part of it can no longer be accessed.

*) limiting output

	dbmalloc reports each and every error it encounters.

	SENTINEL will report every error, but give you a configuration
	parameter that will allow you to only receive a single report 
	for a particular line of code, or a particular stack context. 
	That way, if you have a bug in a memcpy() call that is in the
	middle of a for loop, you will only get the error once, not every
	iteration of the loop.  SENTINEL also provides you with the
	capability to disable some or all of the error reporting using
	configuration parameters.

	Purify allows you to suppress each of the types of errors and to 
	attach a function name to the error that you wish to suppress.
	For example, you can disable all array bound write errors within
	the function thisfunc() with the following configuration
	parameter:

		suppress abw thisfunc


*) watching data areas for changes
	
	One of the hardest problems to debug is when a data area gets 
	trashed and you don't know where it is happening.  Most symbolic
	debuggers provide a capability to watch a data area and report
	on the instruction that causes the problem.  The one problem with
	this feature is that it slows down the execution of the program
	by several orders of magnitude (just about enough to make it 
	totally useless).

	dbmalloc has no support for watching data areas

	SENTINEL provides the capability to watch for data area changes
	within the program being debugged (yes, this capability can be
	initiated from the debugger) and does the watching from within
	the context of the program being debugged, not from the context
	of the debugger.  Using this mechanism, a watchpoint will run 
	at almost the full speed of the normal execution of the program.

	Purify provides the same functionality, although as with out of
	bounds memory accessing, the report of a data change will always
	be immediate, while with SENTINEL it may be delayed until the next
	entry into the SENTINEL library (usually very soon thereafter).
	
*) stack checking

	dbmalloc does not perform any stack manipulation

	Both SENTINEL and Purify check the stack whenever data areas
	are allocated and/or free'd.  If an error is detected in the
	stack at that point in time, SENTINEL will tell you about the
	problem, giving you as much of the current stack context as it
	can, and it will give you a copy of the last stack context 
	that it succesfully obtained.  This should help you track down
	the location of the problem.

*) stack functions

	dbmalloc has no support for stack manipulation

	SENTINEL provides the developer with a set of routines that can be
	used to access and print a stack context whenever they want.  This
	allows the developer to include a full stack context dump in their
	assertion errors or other "internal" errors.  This may preclude
	the necessity to generate or look at a core file when debugging a 
	internal error.

	Purify has no support for such a feature.

*) portability warnings

	dbmalloc can be configured to warn about non-portable uses of
	malloc/realloc and free that could cause a problem if you ported
	the software to a non-ANSI-C system.

	SENTINEL will warn about these non-portable uses automatically (of 
	course, these warnings can be disabled with a run-time configuration
	parameter if you don't want to see them.

	Prufiy does not have support for such a feature

*) debug output
	
	dbmalloc includes a file "debug.h" which defines a general purpose
	mechanism to include debug output statements in the code but have 
	them disabled unless you recompile the code with a -DDEBUG flag.

	SENTINEL takes this one step further and provides a mechanism that
	can support the same functionality plus it gives you the
	capability to enable/disable the debug output at run-time as
	opposed to just compile time.  This is similar to the -x flag for
	uucp which enables multiple levels of debug output that is not
	normally seen without the debug flag.

	Purify has no support for such a feature.

*) production support

	dbmalloc can be included in a beta or production version of your
	product, but it is always enabled and therefore may generate error
	reports for your users that you don't want them to see

	SENTINEL can be included in a beta or production version of your
	program and by default it will be disabled (it will not check any
	errors, nor give any output).  You can get your end-user to enable
	it by simply setting a configuration parameter or environment 
	variable (this can also be done by your program so you can add a
	run-time parameter to the program like -d which in turn enables 
	the SENTINEL debugging capabilities).

	Purify requires a license whenever a program that was linked with
	it is run, therefore you cannot ship a purified program to a 
	customer unless they also have a Purify license.

*) language support

	dbmalloc supports C and to a lesser extent C++ (it doesn't know how 
	to translate C++ names to readable names, if you use new/delete for
	your allocation/free mechanism, it won't be able to record to
	location of where the allocation/free took place.

	Both SENTINEL and Purify fully support C and C++.  I have seen some
	claims from users that say that they have "purified" fortran
	program, but Purify doesn't claim to support doing that.

*) Performance

	We have the following statistics based upon a program that
	allocates, reallocates and frees data in a tight loop (it keeps a
	table of 1,000 allocations and randomly chooses one to free and
	allocate again, or to reallocate).  I beleive this is the worst
	case for both Purify and SENTINEL because both programs must
	perform a stack trace at every malloc and free (to record where
	the action took place).

	The test program allocates 1,000,000 data areas of varying sizes
	from 80 bytes to 90,000 bytes keeping 1,000 allocations alive at
	any point in time and randomly selecting one of the 1,000 slots for
	reallocation and/or freeing. (While the word random is used here,
	all invocations of the program use the same seed and therefore
	generate the same random sequence of sizes on the same system).

		SUN SPARC 2 w/ Solaris 1.0.1

				User		System		Program
				CPU		  CPU		 Size 

	standard libc		 65.9		  0.4		1.54MB
	Purify			169.3		 51.2		1.56MB
	dbmalloc     		314.8		  0.9		4.66MB
	dbmalloc no fill*	 75.5		  0.9		4.66MB
	SENTINEL     		327.9		 53.2		2.54MB
	SENTINEL no fill*	139.9		 50.6		2.54MB
	SENTINEL no stack* 	261.0             0.6           2.51MB
	SENTINEL no stack/fill*  67.0             0.6           2.51MB
	SENTINEL disabled*	 56.0		  0.5		2.51MB

		
		HP 9000/847 w/ HP-UX 8.07 (with 30 users online)
		
				User		System		Memory
	standard		 20.0		  0.1		0.86MB
	Purify			  NA 		  NA 		NA
	dbmalloc     		214.7		  0.8		1.81MB
	dbmalloc no fill*	 38.1		  0.5		1.81MB
	SENTINEL     		205.8		  1.1		1.38MB
	SENTINEL no fill*	145.3		  2.1		1.38MB
	SENTIENL no stack*	 95.3		  0.4		1.38MB
	SENTINEL no stack/fill	 32.8		  0.2           1.38MB
	SENTINEL disabled*	 29.8		  0.3		1.38MB


		Intel 80486 w/ Interactive UNIX 3.0

				User 		System		Memory
	standard		 21.1		  0.8		2.56MB
	Purify			  NA 		  NA 		  NA 
	dbmalloc     		115.3		  1.8		1.92MB
	dbmalloc no fill*	 46.6		  0.8		1.92MB
	SENTINEL     		168.1		  1.5		1.29MB
	SENTINEL no fill*	 92.6             0.5           1.29MB
	SENTINEL no stack*	139.0             2.1           1.33MB
	SENTINEL no stack/fill	 64.7		  0.6           1.33MB
	SENTINEL disabled*	 54.9             1.0           1.33MB

	*  SENTINEL and dbmalloc provide limited action modes which give
	   better performance at the expense of some of the error checking
	   capabilities.  The following modes are supported:

		no fill	 - data area filling is disabled.  This reduces
			   the likelyhood that small data overruns outside
			   of sentinel control will get caught and excludes
			   any uninitialized read errors.
		no stack - do not record the stack of where allocations are
			   made and freed.  Only the stack of where the
			   error is detected will be recorded.
		disabled - do not perform any extra error checking.

	   dbmalloc only supports the "no fill" mode (other than its normal
	   mode).
		
	NA - not available on this architecture

	-----------------------------------------
	PERFORMANCE TEST 2 - memory access

	This test involves running the following program with each debugging
	environment:

		main()
		{
			long i;
			char * t, *u;
			t = (char * ) malloc(300);
			u = (char * ) malloc(300);

			for(i=0; i < 1000000; i++)
			{
				memset(t,'\0',300);
				memset(u,'\0',300);
				memcmp(t, u, 300);
			}
		}

	This program was compiled without optimization in each environment.


	ENVIRONMENT	TOOL           USER CPU	  SYSTEM CPU

	Sun Sparc 2:	normal cc        49.8 	    0.0
			Purify     	104.4	    0.4
			dbmalloc	359.9       0.1
			SENTINEL	114.2       0.8

	HP 9000/847:	normal cc        27.4 	    0.0
			Purify     	   NA        NA
			dbmalloc	528.3       1.3
			SENTINEL	131.1       0.3

	Intel 80486:	normal cc 	 89.0	    1.3
			Purify		   NA        NA
			dbmalloc	421.1       0.9
			SENTINEL	 97.8       0.8


*) contact info

	The current version of dbmalloc (PL14) was posted to
	comp.sources.misc, Volume 32 issues 6 to 15, dated 4 Sep 92,
	archive name dbmalloc, release 1.14; it can thus be gotten
	from wherever comp.sources.misc are archived. If you don't have
	access to such an archive I can email the sources to you.


	Further information about SENTINEL is available from:

		Virtual Technologies Inc
		46030 Manekin Plaza, suite 160
		Dulles, VA 20166
	
		(703) 430-9247

		info@vti.com


	Further information about Purify is available from:

		Pure Software Inc.
		2111 Grant Road
		Los Altos, Ca 94024

		(415) 903-5100

		info@pure.com


-- 
*** SENTINEL(tm) The ultimate Debugging Environment - email for more info ***

Conor P. Cahill              (703)430-9247            cpcahil@virtech.vti.com
Virtual Technologies, Inc.  46030 Manekin Plaza          Dulles, VA 20166 
Newsgroups: comp.unix.programmer