Detailed Instructions: In this task, you will be presented with a context from an academic paper and a question separated with a
. You have to answer the question based on the context.
Problem:Probing into the assorted empirical results may help us discover some interesting phenomenons:

The advantage of pre-training gradually diminishes with the increase of labeled data BIBREF14, BIBREF17, BIBREF18.

Fixed representations yield better results than fine-tuning in some cases BIBREF24.

Overall, pre-training the Seq2Seq encoder outperforms pre-training the decoder BIBREF24, BIBREF17, BIBREF15, BIBREF16.
Question: What experimental phenomena are presented?
Solution:
The advantage of pre-training gradually diminishes with the increase of labeled data Fixed representations yield better results than fine-tuning in some cases pre-training the Seq2Seq encoder outperforms pre-training the decoder