Simplify highlight visibility toggling

Just a small refactoring to make the code that toggles highlight visibility a little easier to understand. /cc @aron