Bridging Subword Gaps in Pretrain-Finetune Paradigm for Natural Language Generation (2021)

Abstract: A well-known limitation in pretrain-finetune paradigm lies in its inflexibility caused by the one-size-fits-all vocabulary. This potentially weakens the effect when applying pretrained models into natural language generation (NLG) tasks, especially for the subword distributions between upstream and downstream tasks with significant discrepancy. Towards approaching this problem, we extend the vanilla pretrain-finetune pipeline with an extra embedding transfer step. Specifically, a plug-and-play embedding generator is introduced to produce the representation of any input token, according to pre-trained embeddings of its morphologically similar ones. Thus, embeddings of mismatch tokens in downstream tasks can also be efficiently initialized. We conduct experiments on a variety of NLG tasks under the pretrain-finetune fashion. Experimental results and extensive analyses show that the proposed strategy offers us opportunities to feel free to transfer the vocabulary, leading to more efficient and better performed downstream NLG models.

## introduction pretrain-finetune paradigm has been highly successful on tackling challenging problems in natural language processing, e.g., domain adaptation @xcite , incremental learning @xcite , as well as knowledge transferring @xcite . the rise of large-scale pre-trained language models further attracts increasing attention towards this strategy @xcite . typically, these methods first pretrain a universal m-bert ce no zo ic pala eo hy dro dyn ami c out-of-domain cen ozo ic pal a e o hydro dynamic thesis cenozoic palaeohydrodynamic in this paper, we provide the first answer from the subword discrepancy aspect, namely, the subword vocabulary extracted according to the pretraining data distribution is insufficient to cope with the downstream nlg tasks. such inflexibility stems from the fact that downstream nlg models have to inherit the vocabulary from their pre-trained counterparts. in order to deal with the open-vocabulary problem, it is de-facto standard for pre-trained models to employ heuristic subword segmentation methods @xcite . however, the segmentation learns on the upstream corpus other than the finetuned data and is likely to be sub-optimal @xcite . we argue that these lead to subword discrepancy and bring two defects. firstly, the pre-trained model usually learns a fine-grained subword segmentation to maintain the coverage of a large amount of diverse vocabulary. consequently, downstream nlg models may suffer from more serious exposure bias @xcite and expensive computational cost caused by the increased sequence lengths. as one example, m-bert exploits 100 thousand fine-grained subwords to encode hundreds of languages, while most of downstream nlg tasks, in fact, require only one language and its associate tokens. secondly, words that are rare in upstream task but frequent in downstream task may be segmented end up poorly understood @xcite . considering the english sequence "cenozoic palaeohydrodynamic" shown in to this end, we propose a simple yet generalized pretrain-finetune strategy, where an embedding transfer stage is inserted between pre-training and finetuning to eliminate their token granularity gaps. unlike the prior strategy using a fixed vocabulary, our vocabulary is changeable and its items including mismatched ones can be easily initialized by the pre-trained embeddings. concretely, we equip the pre-trained model with a plug-and-play embedding generator, which is able to produce the embedding of any token by feeding its subwords and hyperwords that appeared in pre-trained vocabulary. to train this generator, we randomly split or merge some tokens to replace their original embeddings with those produced by the generator. the parameters of the generator are optimized under the vanilla pre-training framework to minimize the divergence before and after replacing the embeddings. accordingly, we can use a task-specific vocabulary for the downstream task, where common tokens are immediately initialized with pre-trained embeddings while mismatched ones are initialized by our generator. we conduct experiments on various tasks under nlg context, in a range from domain adaptation to knowledge transferring, and from machine translation to answer-aware question generation. empirical results demonstrate the universal-effectiveness of the proposed strategy comparing with strong baselines and related approaches. quantitative and qualitative analyses verify that tackling subword discrepancy can exactly alleviate the problem of exposure bias, large computational cost, and the under-represented tokens in vanilla pretrainfinetune paradigm. to summarize, the contributions of our work are as follows: ## related work recent studies observe that pre-trained models suffer a bottleneck when they are applied to nlg tasks @xcite @xcite . this problem has been attributed to many reasons. for example, @xcite point out pretrain-finetune discrepancy caused by the absent masked frames in real data when adopting pretrained masked language models. @xcite investigate catastrophic forgetting in finetuning stage. it can be said that how to successfully employ pretrain-finetune to enhance nlg models remains a great challenge. we explore this problem from another direction, i.e., the unsuitable subword segmentation for downstream tasks. task-specific vocabulary a natural manner to address this issue is to adopt a task-specific vocabulary. @xcite first replace the embedding layer with an independent encoder, of which vocabulary and parameters are learned from the downstream corpus. along this line, @xcite exploit external monolingual data to construct a new embedding layer and achieve improvements in domain adaptation. this series of studies empirically confirm the necessity of the suitable vocabulary for the finetuning stage. however, these methods have to learn the task-specific embeddings separately before each adaptation, which brings in additional computational cost thus limiting their applicability. besides, they completely discard the pre-trained embeddings, which have been proved to be useful by @xcite . extra encoder or embedding layer may fail to be well optimized with insufficient downstream resources. accordingly, rothe et al. (2020) employ a task-specific vocabulary to retrain m-bert, which is then used to initialize neural machine translation (nmt) model. considering more robust approaches, @xcite and @xcite randomly sample segmentations for each sentence at the training time. unlike the above methods, our goal is to build a plug-andplay component, that involves neither retraining the pre-trained model nor learning task-specific embeddings separately. embedding generator our work is also related to studies with respect to generating embeddings for out-of-vocabulary (oov) words. in this context, researchers use embeddings of characters or subwords to predict those of unseen words @xcite @xcite . for example, @xcite train an embedding generator through reconstructing the original representation of each word from its bag of subwords. @xcite progressively improve the generator using attention mechanism. @xcite further leverage similar words to enhance this procedure. our work significantly differs from the above studies in two aspects. due to the vocabulary is fixed once predefined, the embedding reconstruction can be merely drawn on a few of selected words. by contrast, our generator is able to produce embeddings of any tokens, since these embeddings are directly embedded into the pre-trained model with an objective in terms of minimizing the divergence. moreover, previous studies mainly focus on handling the problem of oov, while our work, to our best of knowledge, is the first study that exploits embedding generator to transfer granularity over subwords for pretrain-finetune paradigm. ## methodology in this section, we introduce our proposed pretrainfinetune strategy in detail. ## experiments in this section, we examine the effectiveness of the proposed strategy in a variety of nlg tasks. we first run a set of experiments to compare the variants of our approach and the related methods on domain adaptation translation tasks. then, we assess the superiority of our approach on transferring the knowledge from m-bert @xcite and m-bart @xcite to two downstream nlg tasks: machine translation (mt) and answer-aware question generation (qg). ## analysis to better understand subword discrepancy and our method, we make in-depth analyses on wmt en⇒de task to investigate three problems: q1: how subword granularity affects nlg models? ( § 5.1) q2: how embedding transfer benefits to downstream models? ( § 5.2) q3: dose our strategy acquire large computational costs? ( § 5.3) q4: can our strategy exactly handle under-represented tokens? ( § 5.4) ## conclusion in this paper, we point out that the one-size-fits-all subword vocabulary, despite its all-encompassing superiority, is not the preferred solution for the popular pretrain-finetune paradigm. it causes the subword discrepancy among upstream and downstream models, which is given concrete form to the unsuitable granularity and under-represented words. consequently, we propose a novel embedding transfer strategy with a plug-and-play embedding generator. empirical results suggest that: 1) our approach is universally effective on overcoming subword discrepancy; 2) embedding transfer can bring benefits to computational efficiency; and 3) embedding generator can be achieved via either directly averaging the input embeddings or applying trainable components, the latter performs better but depends on few of training. as our approach is transparent to model architectures and tasks, we believe it can be widely applied and further raise the flexibility and applicability of pre-trained models. in the future, we plan to investigate its effectiveness on other generation tasks, such as code generation @xcite , summarization @xcite and so on.