From: ouster@sprite.Berkeley.EDU (John Ousterhout)
Newsgroups: comp.lang.tcl
Subject: Re: Core dump when delete chars in text widget
Date: 1 Oct 1992 20:58:21 GMT
Organization: U.C. Berkeley Sprite Project
Distribution: world
NNTP-Posting-Host: tyranny.berkeley.edu

In article <1992Sep26.194123.24976@rchland.ibm.com>, shmdgljd@rchland.vnet.ibm.com (Jay Schmidgall) writes:
|> I've been getting core dumps when deleting characters in the text widget.
|> 
|> call stack is usually Rebalance, TkBTreeDeleteChars, tkText.DeleteChars().
|> 
|> codecenter tells me that on
|> 
|> tkTextBTree.c:722   nodePtr = line2Ptr->parentPtr;
|> 
|> line2Ptr is pointing to freed memory; unfortunately, I haven't been able
|> to track down where it is being freed.
|> 
|> Of course, if I recompile tcl/tk with TCL_MEM_DEBUG defined, everything
|> works fine.  I've also tried using a local malloc debug package which
|> did basically the same things as the tcl memory stuff; the core dump still
|> occurred but the information I got didn't help much.
|> 
|> I'm on AIX 3.2.2, x11r4.  Has anyone else seen this problem?
|> 
|> -- 
|> : jay          jay@vnet.ibm.com    My opinions and ideas, not my employer's.
|> : shmdgljd@rchland.vnet.ibm.com    (c) Copyright 1992.  All rights reserved.

I've found and fixed two different problems with the code in tkTextBTree.c,
one of which is responsible for the symptoms above.  The other bug also causes
core dumps during text deletion, but only with some memory allocators and
only if a line being deleted contained both tags and marks in a particular
configuration.  This one has been fixed as well.  I'm enclosing a patch
below that should fix both of these problems.  Please let me know if you
continue to have problems with text after applying the patch, since that
would mean that there's at least one more bug out there.  If you do run into
problems, the best thing is for you to send me a short wish script producing
the problem, plus a stack trace from your system so I can see where the
error is happening (in case I can't reproduce it here).  Jay deserves thanks
for providing enough information for me to track down the above problem even
though I couldn't reproduce it here;  thanks also to Luke Emery for helping
to track down the second problem.

*** /tmp/,RCSt1478026	Thu Oct  1 13:44:09 1992
--- tkTextBTree.c	Thu Oct  1 13:40:02 1992
***************
*** 555,562 ****
  	     * automatically and be dropped, which is the right behavior.
  	     */
  
! 	    for (annotPtr = linePtr->annotPtr; annotPtr != NULL;
! 		    annotPtr = annotPtr2) {
  		if (annotPtr->type == TK_ANNOT_TOGGLE) {
  		    AddToggleToLine(line2Ptr, ch2, annotPtr->info.tagPtr);
  		    ChangeNodeToggleCount(nodePtr, annotPtr->info.tagPtr, -1);
--- 555,562 ----
  	     * automatically and be dropped, which is the right behavior.
  	     */
  
! 	    for (annotPtr = linePtr->annotPtr, linePtr->annotPtr = NULL;
! 		    annotPtr != NULL; annotPtr = annotPtr2) {
  		if (annotPtr->type == TK_ANNOT_TOGGLE) {
  		    AddToggleToLine(line2Ptr, ch2, annotPtr->info.tagPtr);
  		    ChangeNodeToggleCount(nodePtr, annotPtr->info.tagPtr, -1);
***************
*** 564,570 ****
  		    ckfree((char *) annotPtr);
  		} else {
  		    annotPtr2 = annotPtr->nextPtr;
- 		    TkBTreeRemoveAnnotation(annotPtr);
  		    annotPtr->linePtr = line2Ptr;
  		    annotPtr->ch = ch2;
  		    TkBTreeAddAnnotation(annotPtr);
--- 564,569 ----
***************
*** 720,727 ****
  	prevLinePtr->nextPtr = linePtr;
      }
      ckfree((char *) line1Ptr);
-     nodePtr = line2Ptr->parentPtr;
      if (line2Ptr != line1Ptr) {
  	if (nodePtr->children.linePtr == line2Ptr) {
  	    nodePtr->children.linePtr = line2Ptr->nextPtr;
  	} else {
--- 719,726 ----
  	prevLinePtr->nextPtr = linePtr;
      }
      ckfree((char *) line1Ptr);
      if (line2Ptr != line1Ptr) {
+ 	nodePtr = line2Ptr->parentPtr;
  	if (nodePtr->children.linePtr == line2Ptr) {
  	    nodePtr->children.linePtr = line2Ptr->nextPtr;
	} else {
Newsgroups: comp.lang.tcl