Record mutatedElement as parent on remove

Observers weren't getting called when elements were removed, since the element no longer had parents. Record the parent as the mutated element in these cases.