Revert "Remove useless check"

Reverts hypothesis/h#1723

I found a case when the test is needed: when we click on the card of a deleted annotation.