Fix highlight and unhighlight inconsistency.

The unhighlight method retrieved the point differently from highlight,
and incorrectly.  It now matches the behavior of highlight.
Short-circuit when removing all highlights.

There's no need to continue in the function when all highlights are
being cleared.