Distilling Knowledge Learned in BERT for Text Generation (2020)

Abstract: Large-scale pre-trained language model such as BERT has achieved great success in language understanding tasks. However, it remains an open question how to utilize BERT for language generation. In this paper, we present a novel approach, Conditional Masked Language Modeling (C-MLM), to enable the finetuning of BERT on target generation tasks. The finetuned BERT (teacher) is exploited as extra supervision to improve conventional Seq2Seq models (student) for better text generation performance. By leveraging BERT’s idiosyncratic bidirectional nature, distilling knowledge learned in BERT can encourage auto-regressive Seq2Seq models to plan ahead, imposing global sequence-level supervision for coherent text generation. Experiments show that the proposed approach significantly outperforms strong Transformer baselines on multiple language generation tasks such as machine translation and text summarization. Our proposed model also achieves new state of the art on IWSLT German-English and English-Vietnamese MT datasets.

## introduction large-scale pre-trained language model, such as elmo @xcite , gpt @xcite and bert @xcite , has become the de facto first encoding step for many natural language processing (nlp) tasks. for example, bert, pre-trained with deep bidirectional transformer @xcite via masked language modeling and next sentence prediction, has revolutionized the state of the art in many language understanding tasks, such as natural language inference @xcite ) and question answering @xcite . however, beyond common practice of finetuning bert for language understanding @xcite , applying bert to language generation still remains an open question. text generation aims to generate natural language sentences conditioned on certain input, with applications ranging from machine translation @xcite @xcite ), text summarization @xcite @xcite , to image captioning @xcite @xcite . in this work, we study how to use bert for better text generation, which is still a relatively unexplored territory. intuitively, as bert is learned with a generative objective via masked language modeling (mlm) during the pre-training stage, a natural assumption is that this training objective should have learned essential, bidirectional, contextual knowledge that can help enhance text generation. unfortunately, this mlm objective is not auto-regressive, which encumbers its direct application to auto-regressive text generation in practice. we tackle this challenge by proposing a novel and generalizable approach to distilling knowledge learned in bert for text generation tasks. we first propose a new conditional masked language modeling (c-mlm) task, inspired by mlm but requiring additional conditional input, which enables finetuning pre-trained bert on a target dataset. in order to extract knowledge from the finetuned bert and apply it to a text generation model, we leverage the finetuned bert as a teacher model that generates sequences of word probability logits for the training samples, and treat the text generation model as a student network, which can effectively learn from the teacher's outputs for imitation. the proposed approach improves text generation by providing a good estimation on word probability distribution for each token in a sentence, consum-ing both the left and the right context, the exploitation of which encourages conventional text generation models to plan ahead. at inference time, the teacher model (bert) is not required thus the decoding speed is as fast as the underlying student model. text generation models are usually trained via maximum likelihood estimation (mle), or teacher forcing @xcite : at each time step, it maximizes the likelihood of the next word conditioned on its previous ground-truth words. this corresponds to optimizing one-step-ahead prediction. as there is no explicit signal towards global planning in the training objective, the generation model may incline to focusing on local structure rather than global coherence. with our proposed approach, bert's looking into the future ability can act as an effective regularization method, capturing subtle long-term dependencies that ensure global coherence and in consequence boost model performance on text generation. an alternative way to leverage bert for text generation is to initialize the parameters of the encoder or decoder of seq2seq with pretrained bert, and then finetuning on the target dataset. however, this approach requires the encoder/decoder to be identical to bert, inevitably making the final text generation model too large. our approach, on the other hand, is modular and compatible to any text-generation model, and has no restriction on model size or model architecture (e.g., lstm or transformer). the main contributions of this work are threefold: (i) we present a novel approach to utilizing bert for text generation. the proposed method induces sequence-level knowledge into the conventional one-step-ahead and teacher-forcing training paradigm, by introducing an effective regularization term to mle training loss. (ii) we conduct comprehensive evaluation on multiple text generation tasks, including machine translation and text summarization. experiments show that our proposed approach significantly outperforms strong transformer baselines and is generalizable to different tasks. (iii) the proposed model achieves new state of the art on both iwslt14 german-english and iwslt15 english-vietnamese datasets. in our study, we focus on bert due to its superior performance on multiple language understanding tasks. however, different from previous work exploiting bert for language understanding tasks, here we aim to apply bert to text generation. to the best of our knowledge, this is still a relatively unexplored space. the proposed approach is also model-agnostic and can be applied to other pretrained language models as well. bert for text generation there has been some recent attempt on applying bert to text generation. specifically, @xcite trained cross-lingual mlm and demonstrated promising results for cross-lingual natural language inference @xcite and unsupervised neural machine translation (nmt) @xcite . @xcite formulated bert as a markov random field lm and showed preliminary results on unsupervised text generation with improved diversity. @xcite al. (2019a) our approach is novel in the sense that we do not directly use the parameters of bert in the seq2seq model. instead, bert acts as an effective regularization to the mle training loss, by proactively injecting future information for predicting the present. ## approach in this section, we present our proposed approach to distilling the knowledge in bert for text generation in generic sequence-to-sequence (seq2seq) setting. we first review seq2seq learning in section 3.1, and then describe the proposed approach in section 3.2 and 3.3. ## experiments in this section, we describe our experiments on two well-studied text generation tasks: machine translation, and abstractive text summarization. ## conclusion in this work, we propose a novel and generic approach to utilizing pre-trained language models to improve text generation without explicit parameter sharing, feature extraction, or augmenting with auxiliary tasks. our proposed conditional mlm mechanism leverages unsupervised language models pre-trained on large corpus, and then adapts to supervised sequence-to-sequence tasks. our distillation approach indirectly influences the text generation model by providing soft-label distributions only, hence is model-agnostic. experiments show that our model improves over strong transformer baselines on multiple text generation tasks such as machine translation and abstractive summarization, and achieves new state-of-the-art on some of the translation tasks. for future work, we will explore the extension of conditional mlm to multimodal input such as image captioning.