Only add/remove on hasClass