Rethinking Why Intermediate-Task Fine-Tuning Works (2021)

Abstract: Supplementary Training on Intermediate Labeled-data Tasks (STILT) is a widely applied technique, which first fine-tunes the pretrained language models on an intermediate task before on the target task of interest. While STILT is able to further improve the performance of pretrained language models, it is still unclear why and when it works. Previous research shows that those intermediate tasks involving complex inference, such as commonsense reasoning, work especially well for RoBERTa-large. In this paper, we discover that the improvement from an intermediate task could be orthogonal to it containing reasoning or other complex skills — a simple real-fake discrimination task synthesized by GPT2 can benefit diverse target tasks. We conduct extensive experiments to study the impact of different factors on STILT. These findings suggest rethinking the role of intermediate fine-tuning in the STILT pipeline.

## introduction pretrained language models @xcite @xcite have contributed to great progress in natural language understanding (nlu). stilt @xcite @xcite @xcite can further improve their performance on downstream nlu tasks by redesigning the training pipeline, introducing an intermediate-task finetuning phase before fine-tuning the pretrained models on the target task of interest (figure 1 ). nevertheless, this approach is not necessarily beneficial, and its effectiveness depends highly on the intermediate task applied. to study when and why stilt works, @xcite conduct large-scale experiments based on roberta-large @xcite 2019) with different intermediate-target task pairs. they focus on studying what kind of intermediate tasks are helpful overall and which linguistic skills a model learns from the intermediate phase. they show the difficulty to have a generally useful intermediate task and conclude that those containing complex reasoning and inference, such as cosmosqa @xcite and hel-laswag @xcite , tend to enhance various target tasks. however, this ignores the fact that hellaswag is a synthetic dataset, and roberta tends to capture the data artifacts when fine-tuned on hellaswag @xcite . in this paper, we demonstrate that intermediate tasks' enhancement could be irrelevant to providing complex reasoning or special linguistic skillsa simple real-fake discrimination task synthesized by gpt2 @xcite can benefit diverse target tasks, including those commonsense reasoning tasks. these observations suggest rethinking the role of the intermediate-finetuning phase in the pipeline. our main contributions are as follows. foot_0 ## a good intermediate task we first define what a good intermediate task means. stilt is known for two benefits @xcite : 1) improving target tasks' best performance, and 2) stabilizing the fine-tuning process of the target tasks, notably reducing the degenerate fine-tuning runs @xcite @xcite . @xcite only focus on the first property, we study both benefits by extensive hyperparameter trials. summarized in ## rethinking: two simple baselines hellaswag @xcite is a commonsense reasoning multiple-choice task, which contains a premise narrating an event and four plausible next scenarios (options) in each data example (figure 4 ). all its negative options are generated by the machine given the premises; consequently, this dataset is known to contain superfluous artifacts @xcite . despite the limitation, @xcite show that hellaswag is one of the most potent intermediate tasks for roberta-large in their large-scale experiments 2 . they then attribute 2 they study roberta with 110 intermediate-target task such wide improvement on target tasks to the complex commonsense reasoning it requires. on the contrary, we first ablate the common sense from hellaswag, seeking to understand if simple intermediate tasks are enough to enhance the performance of various target tasks. we propose two baselines as intermediate tasks. the first one is to remove the premises from hel-laswag, denoted as hellaswag-p, so that each data example only contains four options without contexts. therefore, the model does not require common sense and reasoning skills to predict the follow-up anymore. it only needs to identify which option is not generated by the machine. secondly, we build a synthetic dataset that mimics the creation of hellaswag, denoted as synthesis gp t 2 . the main difference is that, unlike hellaswag, our premises and the correct endings are not from particular sources containing commonsense. we use wikipedia as the source corpus since it has already been seen by the model in the pretraining phase. specifically, given a sentence from wikipedia, we split it into two parts. the first half becomes the premise, and the last half becomes the positive choice. we then use pretrained gpt2 to generate three negative choices conditioned on the premise. the decoding strategy is nucleus (top-p) sampling @xcite , where @xmath0.9. more descriptions can be found in the supplementary materials. our goal is to use these two simple baselines to point out some underestimated factors when studying why stilt works. while previous work @xcite attempts to relate the linguistic skills between the intermediate and target task, we suspect that the linguistic knowledge further provided by the intermediate task could be less contributive than previous belief, as plenty of research combinations and show that in many cases, the intermediate tasks are not helpful, or even hurtful in some cases. ## analysis in this section, we further study different factors that may influence the effectiveness of stilt. ## conclusion we discover that a generally beneficial intermediate task to roberta can be as simple as a synthetic real-fake discrimination task, and provide observations on different factors that influence stilt's best and mean effectiveness. therefore, we suggest rethinking why intermediate-task fine-tuning works, particularly under low-resource settings.