Optimize annotation activation handling.

Reusing the cached selection is important.