Remove markers for annotations when destroying an annotator instance