Before revision:  Then we use the pseudo-label for the task-specific loss and masked language modelloss to pre-train.
After revision:  Then we use the pseudo labels to supervise the task-specific loss and masked language modelloss to pre-train.