Repository: nltk/nltk

Issue:
ValueError when using ParentedTree in 2.0.1rc4
When using NLTK version 2.0.1rc4 (as packaged by Arch Linux), the following code results in an error:

```
>>> nltk.tree.ParentedTree("(FOO (BAR baz) (QUUX blorfle))")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.7/site-packages/nltk/tree.py", line 1035, in __init__
    super(ParentedTree, self).__init__(node_or_str, children)
  File "/usr/lib/python2.7/site-packages/nltk/tree.py", line 821, in __init__
    self._setparent(child, i, dry_run=True)
  File "/usr/lib/python2.7/site-packages/nltk/tree.py", line 1115, in _setparent
    raise ValueError('Can not insert a subtree that already '
ValueError: Can not insert a subtree that already has a parent.
```

The same code does not give an error with version 2.0.1-rc2 (the next-most-recent version available as a binary package in Arch:

```
>>> nltk.tree.ParentedTree("(FOO (BAR baz) (QUUX blorfle))")
ParentedTree('FOO', [ParentedTree('BAR', ['baz']), ParentedTree('QUUX', ['blorfle'])])
```



Discussion:
Introduced in 8284903e2a80925fb681fc2176914193c32cbce6 – having a look now!

Disregard attempted fix. I'll leave this for someone that understands what the hell happens under the hood in tree.py.

As of 2.0.2, this seems to have been resolved -- the example code above no longer raises an error.  I don't really know what change is responsible for the fix, though...only that it now works again.

Thanks, closing this now.

On further investigation, 200eea8 in combination with 1309882 is actually an anti-fix, since `_setparent` is now never called.  So now in 2.0.2, the following code returns `None`, bogusly.

```
t = nltk.tree.ParentedTree("(FOO (BAR baz))")
t[0]._parent
```


Resolving patch:
diff --git a/nltk/tree.py b/nltk/tree.py
--- a/nltk/tree.py
+++ b/nltk/tree.py
@@ -813,6 +813,21 @@ class AbstractParentedTree(Tree):
       - ``_delparent()`` is called whenever a child is removed.
     """
 
+    def __init__(self, node_or_str, children=None):
+        super(AbstractParentedTree, self).__init__(node_or_str, children)
+        # If children is None, the tree is parsed from node_or_str, and
+        # all parents will be set during parsing.
+        if children is not None:
+            # Otherwise we have to set the parent of the children.
+            # Iterate over self, and *not* children, because children
+            # might be an iterator.
+            for i, child in enumerate(self):
+                if isinstance(child, Tree):
+                    self._setparent(child, i, dry_run=True)
+            for i, child in enumerate(self):
+                if isinstance(child, Tree):
+                    self._setparent(child, i)
+
     #////////////////////////////////////////////////////////////
     # Parent management
     #////////////////////////////////////////////////////////////
@@ -1023,6 +1038,15 @@ def __init__(self, node_or_str, children=None):
         self._parent = None
         """The parent of this Tree, or None if it has no parent."""
         super(ParentedTree, self).__init__(node_or_str, children)
+        if children is None:
+            # If children is None, the tree is parsed from node_or_str.
+            # After parsing, the parent of the immediate children 
+            # will point to an intermediate tree, not self. 
+            # We fix this by brute force:
+            for i, child in enumerate(self):
+                if isinstance(child, Tree):
+                    child._parent = None
+                    self._setparent(child, i)
 
     def _frozen_class(self): return ImmutableParentedTree
 
@@ -1133,6 +1157,15 @@ def __init__(self, node_or_str, children=None):
            contain duplicates, even if a parent contains this tree
            multiple times."""
         super(MultiParentedTree, self).__init__(node_or_str, children)
+        if children is None:
+            # If children is None, the tree is parsed from node_or_str.
+            # After parsing, the parent(s) of the immediate children 
+            # will point to an intermediate tree, not self. 
+            # We fix this by brute force:
+            for i, child in enumerate(self):
+                if isinstance(child, Tree):
+                    child._parents = []
+                    self._setparent(child, i)
 
     def _frozen_class(self): return ImmutableMultiParentedTree
 


Test patch:
diff --git a/nltk/test/tree.doctest b/nltk/test/tree.doctest
--- a/nltk/test/tree.doctest
+++ b/nltk/test/tree.doctest
@@ -697,13 +697,13 @@ operations:
     >>> ptree[0,0].remove(make_ptree('(Q p)'))
     Traceback (most recent call last):
       . . .
-    ValueError: list.index(x): x not in list
+    ValueError: ParentedTree('Q', ['p']) is not in list
     >>> ptree.remove('h'); pcheck(ptree)
     ok! (A (B (C (D )) g))
     >>> ptree.remove('h');
     Traceback (most recent call last):
       . . .
-    ValueError: list.index(x): x not in list
+    ValueError: 'h' is not in list
     >>> # remove() removes the first subtree that is equal (==) to the
     >>> # given tree, which may not be the identical tree we give it:
     >>> ptree = make_ptree('(A (X x) (Y y) (X x))')
@@ -1012,13 +1012,13 @@ multiple parents.)
     >>> mptree[0,0].remove(make_mptree('(Q p)'))
     Traceback (most recent call last):
       . . .
-    ValueError: list.index(x): x not in list
+    ValueError: MultiParentedTree('Q', ['p']) is not in list
     >>> mptree.remove('h'); mpcheck(mptree)
     ok! (A (B (C (D )) g))
     >>> mptree.remove('h');
     Traceback (most recent call last):
       . . .
-    ValueError: list.index(x): x not in list
+    ValueError: 'h' is not in list
     >>> # remove() removes the first subtree that is equal (==) to the
     >>> # given tree, which may not be the identical tree we give it:
     >>> mptree = make_mptree('(A (X x) (Y y) (X x))')