O n L osses for M odern L anguage M odels (2020)

Abstract: BERT set many state-of-the-art results over varied NLU benchmarks by pre-training over two tasks: masked language modelling (MLM) and next sentence prediction (NSP), the latter of which has been highly criticized. In this paper, we 1) clarify NSP’s effect on BERT pre-training, 2) explore fourteen possible auxiliary pre-training tasks, of which seven are novel to modern language models, and 3) investigate different ways to include multiple tasks into pre-training. We show that NSP is detrimental to training due to its context splitting and shallow semantic signal. We also identify six auxiliary pre-training tasks – sentence ordering, adjacent sentence prediction, TF prediction, TF-IDF prediction, a FastSent variant, and a Quick Thoughts variant – that outperform a pure MLM baseline. Finally, we demonstrate that using multiple tasks in a multi-task pre-training framework provides better results than using any single auxiliary task. Using these methods, we outperform BERTBase on the GLUE benchmark using fewer than a quarter of the training tokens.

## introduction when @xcite released bert, a transformer network @xcite trained using a 'masked language model' (mlm) task and a 'next sentence prediction' (nsp), it redefined the nlp landscape, establishing itself as the state-of-the-art (sota) on many natural language understanding (nlu) benchmarks including the glue @xcite , squad @xcite , and swag @xcite benchmarks. many models inspired by bert have since surpassed its performance. however, in contrast to the original bert paper, many obtained better results by excluding the nsp task. some, such as xlnet @xcite and roberta @xcite , rely solely on a mlm variant, while others @xcite @xcite incorporate one or more different auxiliary loss functions. to our knowledge, there is no published work comparing or fully exploring auxiliary tasks for modern language models. with multi-task learning's long history in transfer learning @xcite @xcite , its use in language understanding models deserves further exploration. in this paper, we study existing and novel auxiliary tasks in a bert paradigm to guide future research in an informed manner. specifically, we test and provide insight on: 1) nsp's effect on bert pretraining; 2) the result of 14 other auxiliary tasks on bert pre-training; 3) how to combine multiple tasks in bert pre-training; and 4) the advantages of multi-task learning in bert pre-training. although all experiments in this paper are conducted using bert, we believe the results are applicable to bert's successors (e.g. xlnet, roberta, ernie...) and future models. the code is available at https://github.com/stephao/olfmlm . ## related work as with most deep learning, language representations require large datasets. while there exists corpora of labelled text, the vast majority of language data exists as raw, unlabelled text. accordingly, many language embedding methods, and all those described below, rely solely on unsupervised or self-supervised tasks. ## results in this section, we present the results from an array of different tests. due to the stochastic nature of the training, we would ideally run each test numerous times. however, this is prohibitively expensive due to the computational costs. building from raffel et al. (2019)'s experimental approach, we instead calculate the standard deviation for 5 independent trainings of the baseline mlm-only model, the mlm + nsp model, and our cmtl+ model. we find σ m @xmath0.198, σ n @xmath1.222, σ cm t l+ = 0.273, and use the highest, σ = 0.273, as an estimate for the standard deviation across all experiments. see appendix b for more detail. this is comparable to raffel et al. (2019)'s estimated standard deviation of 0.235. in each table, we boldface all average glue scores that are two estimated standard deviations above the mlm baseline. for the average glue score, we follow wang et al. ( 2018 ) and average the macro averages of each task. this is different than averaging the numbers in a row as we only report one metric per task. ## discussion our results support several recent papers: we sup2020 )'s proposal that nsp prediction is a semantically shallow and often solvable through lexical overlap and that using a task that requires understanding the ordering of contiguous text provides a stronger semantic signal; and we uphold @xcite 's idea that a language model should be trained in a multi-task setting. further, we offer novel methods and insights. providing a signal to reduce the embedding distance between semantically similar sentences, as in our fastsent or quickthought variants do, produces a strong boosts to downstream tasks, with the hypothesis that they produce more distinguishable embeddings. providing a signal that relays word importance, such as tf-idf and tf, likewise produces substantial benefit to bert pre-training. we show strong evidence that a mlm variant loss should always be included when multi-task learning. finally, we demonstrate the value of multi-task learning for language model pre-training; combining multiple beneficial tasks leads to better results than using any of the individual tasks alone. as our focus was a breadth-based search of possible auxiliary tasks, we believe that further gains are possible through a deeper exploration of each task. using soft labels in asp for sentences that are near (but not directly adjacent to) the other sentence has been shown to provide improvements @xcite . (n!)-way classification with n sentence-pieces for sentence ordering is a more challenging task that could provide additional benefits. other similarity metrics, such as dot product or euclidean distance, may provide more useful for the fs or qt methods. beyond using a loss based on a similarity metric, it is possible that other unsupervised clustering algorithms could be beneficial. currently, each task has different loss ranges based on the nature, and not the inherent value, of the task. as some tasks may be more useful than others, it is likely that weighting each task based on some value metric could prove beneficial. groups with sufficient computational resources may also be interested in exploring how the ordering in the continual multi-task learning framework affects downstream tasks. lastly, we do not tune hyperparameters, using only the stated values from previous papers for all our experiments. we leave the above potential to future work. ## conclusion we investigate and support several reasons why next-sentence prediction is ill-suited for bert pretraining, we provide better inference-based alternatives, and we develop other novel auxiliary tasks based on word importance and soft clustering that provide substantial benefits to bert pre-training. we also demonstrate the benefit of multi-task learning in bert pre-training, and identify key factors on how to best combine tasks. we hope the insights provided here will help guide the development of better language models in the future.