If it's causing pain, not erupting fully, causing crowding, ruining orthodontic work, and more, they might need to be removed.