Before revision:  In this paper, we investigate whether it is still helpful to add the specific task's loss in pre-training step .
After revision:  In this paper, we investigate whether it is still helpful to add the specific task's loss in pre-training step and the fine-tuning step. Towards this goal, we propose a learning framework that making best use of the unlabel data on the low-resource and high-resource labeled dataset.