Pre-training Is (Almost) All You Need: An Application to Commonsense Reasoning (2020)

Abstract: Fine-tuning of pre-trained transformer models has become the standard approach for solving common NLP tasks. Most of the existing approaches rely on a randomly initialized classifier on top of such networks. We argue that this fine-tuning procedure is sub-optimal as the pre-trained model has no prior on the specific classifier labels, while it might have already learned an intrinsic textual representation of the task. In this paper, we introduce a new scoring method that casts a plausibility ranking task in a full-text format and leverages the masked language modeling head tuned during the pre-training phase. We study commonsense reasoning tasks where the model must rank a set of hypotheses given a premise, focusing on the COPA, Swag, HellaSwag and CommonsenseQA datasets. By exploiting our scoring method without fine-tuning, we are able to produce strong baselines (e.g. 80% test accuracy on COPA) that are comparable to supervised approaches. Moreover, when fine-tuning directly on the proposed scoring function, we show that our method provides a much more stable training phase across random restarts (e.g x10 standard deviation reduction on COPA test accuracy) and requires less annotated data than the standard classifier approach to reach equivalent performances.

## introduction recent advances in natural language processing have been made using sequential transfer learning over large pre-trained transformer models. from these models, most nlp tasks can be addressed by adding a classifier on top of the transformer embedding outputs @xcite . ## related work in (trinh and le, 2018), researchers have shown that a rnn language model pretrained on a large amount of data can be used to efficiently score sentences in a zero-shot setting. they used the winograd schema challenge (wsc-273) dataset @xcite which mostly consists of a pronoun disambiguation task that requires commonsense reasoning. in their approach, the pronoun to disambiguate is replaced by the different candidates. then, each version of the sentence is scored using the likelihood of the sequence under the forward autoregressive factorization. they showed that targeting the likelihood of the tokens placed after the candidate words performs better than a full-sentence likelihood estimation. this result highlights the fact that the choice of the targeted sub-sequence for the likelihood estimation has an important impact on the overall performance of the model. more recently, analysis of relational knowledge contained in pre-trained bert models has been the subject of different studies @xcite . results have shown evidences that bert models memorize reasoning about entity names and commonsense knowledge, making mlm models appropriate candidates to commonsense oriented tasks. from a supervised learning perspective, @xcite proposed to replace the traditional cross-entropy loss with a margin-based one one the copa dataset. the authors argued that crossentropy based methods are not adapted for plausibility ranking tasks since they force the scores to adopt extreme values (near 0 or 1). in contrast, a margin-based objective function appeared to be a natural way to rank a set of hypotheses. both approaches were compared using the [cls] token of the bert-base model and a separated-sentence input format. the margin-based objective function surpassed the cross-entropy one by increasing the test set accuracy from 73.4% to 75.4%. adopting a token level scoring approach (kocijan et al., 2019) used a bert model with a mixture between a margin-based and a mlm loss on wsc-273 to score the different pronouns to disambiguate. this approach allows the authors to improve the previous state of the art by 8.8%. despite being the closest method to the one proposed in this paper, our approach differs from three points: ## datasets the commonsense reasoning datasets that we focus on are copa @xcite , swag @xcite , hellaswag @xcite and commonsenseqa @xcite . all these datasets share the premise-hypothesis task format. copa copa (choice of plausible alternatives) @xcite is a commonsense causal reasoning task where two candidate hypotheses are given. copa itself is composed of two sub-tasks: effect samples and cause samples. the effect and cause samples have respectively implies and implied by relation with the correct hypothesis. the full-text format of copa is built by using the conjunction words because (resp. so) as middle conjunctions for cause (resp. effect) samples. concerning the separated-sentence format, we reverse the premise and hypothesis order for cause samples in ## experiments in this section we first apply our scoring method in a zero-shot setting on the four aforementioned datasets. then we fine-tune our scoring method while varying the percentage of the training data used and compare it to approaches that use a randomly initialized classifier head. we use roberta large @xcite for our pretrained model as roberta large fine-tuned with a classification layer on top has very competitive results on those datasets. our implementation use pytorch and the huggingface transformers library @xcite . ## conclusions in this work, we presented a new method for plausibility ranking tasks, specifically targeting commonsense ranking problem. we define a scoring function that leverages the mlm head of large pre-trained bidirectional transformer models. we establish strong results in a zero-shot setting on four commonsense reasoning datasets, comparable to supervised approaches. we then fine-tune such model using a margin-based loss on the proposed scoring function, and provide a comparative study with state of the art randomly initialized head methods. our study demonstrates that the direct use of mlm over custom head yields increasingly superior performance gain when decreasing training data size. the proposed approach outperforms state-of-the-art training methods in terms of both test accuracy and training stability. future works include applying such scoring method on broader classification tasks like natural language inference and sentiment analysis. we also think that our token-level scoring method could be used during the self-supervised pretraining phase to extend traditional next sentence prediction and sequence ordering tasks, bringing more commonsense knowledge in the model.