Add support for removing markers when removing annotator from an element.