When unhighlighting, filter down to the highlight nodes first. (Is this faster?)