Remove attention from some labels