Check for annotation before removing it