From: hacrat@catfish.lcs.mit.edu (Nate Osgood)
Subject: g++2's vector memory allocation and malloc.o
Organization: MIT Laboratory for Computer Science
Distribution: gnu
Date: Sun, 27 Sep 1992 19:40:30 GMT

Hi.  I'm using g++ 2.2.2 on Sun4 hardware, running SunOS release
4.1.2.  I've noticed something that I believe is not a bug in g++ but
rather an incompatibility between g++ 2.2.2 and the standard
 routines for use in debugging heap-based allocation problems.  i'm
wondering whether other people on this group might have encountered
this problem before and/or would be able to describe how to get around
it or be able confirm its character (as a NON bug)


  in particular, after hacking around at the assembly language level
i've noticed that g++ 2.2.2's memory layout for vectors of objects
differs substantially from its layout for single objects -- and also
differs substantially from the layout in pre-release 2.0 versions of
g++. more specifically, it seems (from causual observation of the data
structures involved) that the standard format for data structures
allocated by malloc and "new" is of the basic format


4 Bytes of length information
4 additional bytes
Beginning of the data structure


thus, when one gets back a pointer to some location A, there are 8
bytes of data preceding the referent of the pointer in the heap.  (so
one has "actually" allocated 8 + the size of A bytes).  g++ version
1.41 also seems to stick to this convention for both vector and
non-vector objects.


g++ 2.2.2 seems to have a somewhat "trickier" strategy.  for
non-vector objects, the format seems to be the same.  for at least some vector
objects, it seems that another format is used:


4 Bytes of length information
4 additional bytes
4 bytes giving the NUMBER OF ELEMENTS in the vector
4 additional bytes (i think this is a magic word of some sort -- see below)
Beginning of the data structure


thus, when one gets back a pointer to some location A, there are 16
(instead of 8) bytes of data preceding the referent of the pointer in
the heap.  (so one has "actually" allocated 16 (instead of 8) + the
size of A bytes)


it appears that the format for vectors described above is rejected
when one is trying to debug the program by linking to the "malloc.o"
library routines that allow run-time checking of heap integrity.  it
would seem that the the DEALLOCATION mechanism supplied with g++
 has to be aware of which format of data structure it has been handed
in the form of a pointer -- it seems that the "free" routine and its
associated checking machinery is not familiar with the memory layout
format and balks.  more precisely, it seems that the "free" routine's
machinery for checking the sanity of its operation believes that the
length of the allocated block is ALWAYS 8 bytes before the pointer
given.  when this "free" routine is given a pointer to a vector object
as described above (one associated with the second layout), it balks
because it thinks it is being told that the LENGTH of the object in
bytes is the NUMBER OF ELEMENTS of that object (i.e. it looks for the
length at offset -8 rather than -16 from the pointer).

 when the program is compiled using g++ 1.41, everything works fine,
and when the program is NOT linked to the debugging routines, things
SEEM to work fine.

	my primary concerns are that perhaps the problem does not lie
in the checking mechanism, but that incorrect deallocation ALWAYS
takes place (it's just that the checking mechanism CATCHES it).  i
also am a bit puzzled as to why this new format of heap object layout
was adopted -- i figure there must be some obvious reason, but am not
sure what it is.  

	the example below illustrates the problem.  if you link it
with malloc.o, it should report a bad block size of 1 (because there
is 1 element in the vector being allocated).   the error should look
something like this:

catfish % a.out
free: bad block size (1) at 0xf7fff974

IOT trap


I have also included below a snippet from the assembly language code
for the example, showing where the call to operator new "secretly"
creates the extra information (# of elements, and the 4 other bytes)
in the extended format above.  note that if the line

	  Derived *p = new Derived[1];

	is changed to 

	  Derived *p = new Derived;

	the problems go away, since the standard layout for objects is
used.



	any comments on this issue would be ENORMOUSLY appreciated


	Thanks!
	Nate Osgood

---------------------------




class Foo 
{
 public:
  Foo() {}
  ~Foo() {}
};
 
main()
{
  Foo *p = new Foo[1];
 
  delete p; 
}


--------------
	call ___builtin_new,0
	nop
	add %o0,8,%l0		<=  here we're incrementing the returned 
					address by 8 to store this extra info
	mov 1,%o0
	st %o0,[%l0-8]         <= put the # elts in the extra info
	mov 0,%o0
	sub %o0,%l0,%o1	       <= i think this is a "magic word"
				  that serves to indicate the second format
				  note that it is -(addr. of the pointer
  				  to the element)
				
	st %o1,[%l0-4]
	mov %l0,%l1




-- 
-------------------------------------------------------------------------------
Nate Osgood			| ARPA: hacrat@catfish.lcs.mit.edu
CS Graduate Student		| 
MIT LCS Room NE43-636		| UUCP: ...!uunet!catfish.lcs.mit.edu!hacrat

Newsgroups: gnu.g++.bug