Properly handle changes to annotations.