Remove from annotations groups which are removed.
Improved wording a bit. I hope. :-)