Only update color when user intend to do so.