Question: What experimental phenomena are presented?
Answer: The advantage of pre-training gradually diminishes with the increase of labeled data, Fixed representations yield better results than fine-tuning in some cases, pre-training the Seq2Seq encoder outperforms pre-training the decoder