Don't attempt to remove node's that have been already been detached