Check $element is not parent when deactivating
Fixes #67