Removing from parent if removed from child set.

This solves issue #7.