Identifying the Limits of Cross-Domain Knowledge Transfer for Pretrained Models

1 INTRODUCTION . Fine-tuning pretrained language models has proven to be highly effective across a wide range of NLP tasks ; the leaderboards for standard benchmarks are currently dominated by models that adopt this general strategy ( Rajpurkar et al. , 2016 ; 2018 ; Wang et al. , 2018 ; Yang et al. , 2018 ; Wang et al. , 2019 ) . Recent work has extended these findings in even more surprising ways : Artetxe et al . ( 2020 ) , Karthikeyan et al . ( 2019 ) , and Tran ( 2020 ) find evidence of transfer between natural languages , and Papadimitriou & Jurafsky ( 2020 ) show that pretraining language models on non-linguistic data such as music and computer code can improve test performance on natural language . Why does pretraining help even across what appear to be fundamentally different domains , and what are the limits of such cross-domain transfer ? In this work , we seek to inform these questions via a systematic exploration of how much cross-domain transfer we see when the model is denied any information about word identity . In this setting , we can vary the pretraining and fine-tuning examples dramatically while holding other aspects of the task constant . This allows us to quantify the extent of transfer , and it can yield insights into the wide-ranging transfer results cited above . Figure 1 gives an overview of our core experimental paradigm : starting with two identical copies of a single pretrained model for English , we fine-tune one on English examples and the other on scrambled English sentences , using a scrambling function F ( section 3 ) , and then we evaluate the resulting models . We apply this paradigm to four classification tasks and two sequence modeling tasks , and we evaluate bag-of-words baselines , LSTMs with GloVe initialization and rich attention mechanisms , and BERT . Our central finding is that only BERT is able to achieve robust cross-domain transfer , and for classification tasks but not sequence labeling ones . To try to understand why such transfer is successful for some tasks but not others , we pursue a number of hypotheses . First , we consider whether using a scrambling function F that matches word frequencies is required for transfer , and we find that such matching plays a small role , but not enough to account for the observed performance ( section 7.1 ) . Second , we assess whether frequency matching might actually be inserting semantic consistency into the scrambling process by , for example , systematically creating substitution pairs like good/great and professor/teacher ( section 7.2 ) . However , we find no evidence of such semantic consistency . Third , we try to isolate the contribution of pretraining versus fine-tuning by fine-tuning randomly initialized models of different sizes ( section 7.3 ) and by freezing the BERT parameters , such that only task-specific parameters are updated ( section 7.4 ) . These variations lead to a substantial drop in transfer , suggesting that finetuning is vital , although our LSTM results show that the BERT pretrained starting point is also an essential component . Fourth , we ask whether the fine-tuning process is primarily learning to reassociate scrambled words with their sources , and we find that it is not ( section 7.5 ) . While these findings do not fully account for the transfer we observe , they offer a partial explanation which should help guide future studies of this issue and which can help with practical fine-tuning work . 2 RELATED WORK . 2.1 EVIDENCE FOR TRANSFER . Transferability across domains is often used to benchmark large pretrained models such as BERT ( Devlin et al. , 2019b ) , RoBERTa ( Liu et al. , 2019b ) , ELECTRA ( Clark et al. , 2019 ) , and XLNet ( Yang et al. , 2019 ) . To assess transferability , pretrained models are fine-tuned for diverse downstream tasks ( Wang et al. , 2018 ; 2019 ) . Recently , pretrained Transformer-based models ( Vaswani et al. , 2017 ) have even surpassed estimates of human performance on GLUE ( Wang et al. , 2018 ) and SuperGLUE ( Wang et al. , 2019 ) . While the benefits of pretraining are reduced when there is a large train set ( Hernandez et al. , 2021 ) , there is little doubt that this pretraining process helps in many scenarios . 2.2 STUDIES OF WHY TRANSFER HAPPENS . There are diverse efforts underway to more deeply understand why transfer occurs . Probing tests often involve fitting supervised models on internal representations in an effort to determine what they encode . Such work suggests that BERT representations encode non-trivial information about morphosyntax and semantics ( Tenney et al. , 2019 ; Liu et al. , 2019a ; Hewitt & Manning , 2019 ; Manning et al. , 2020 ) and perhaps weakly encode world knowledge such as relations between entities ( Da & Kasai , 2019 ; Petroni et al. , 2019 ) , but that they contain relatively little about pragmatics or rolebased event knowledge ( Ettinger , 2020 ) . Newer feature attribution methods ( Zeiler & Fergus , 2014 ; Springenberg et al. , 2015 ; Shrikumar et al. , 2017 ; Binder et al. , 2016 ; Sundararajan et al. , 2017 ) and intervention methods ( McCoy et al. , 2019 ; Vig et al. , 2020 ; Geiger et al. , 2020 ) are corroborating these findings while also yielding a picture of the internal causal dynamics of these models . Another set of strategies for understanding transfer involves modifying network inputs or internal representations and studying the effects of such changes on task performance . For instance , Tamkin et al . ( 2020 ) show that BERT ’ s performance on downstream GLUE tasks suffers only marginally even if some layers are reinitialized before fine-tuning , and Gauthier & Levy ( 2019 ) , Zanzotto et al . ( 2020 ) , Pham et al . ( 2020 ) , and Sinha et al . ( 2021 ) show that BERT-like models are largely insensitive to word order changes . 2.3 EXTREME CROSS-DOMAIN TRANSFER . Cross-domain transfer is not limited to monolingual cases ( Karthikeyan et al. , 2019 ) . With modifications to its tokenizer , English-pretrained BERT improves performance on downstream multilingual NLU tasks ( Artetxe et al. , 2020 ; Tran , 2020 ) . Papadimitriou & Jurafsky ( 2020 ) show that pretraining language models on structured non-linguistic data ( e.g. , MIDI music or Java code ) improves test performance on natural language . Our work complements and advances these efforts along two dimensions . First , we challenge models with extremely ambitious cross-domain settings and find that BERT shows a high degree of transfer , and we conduct a large set of follow-up experiments to help identify the sources and limitations of such transfer . 3 EXPERIMENTAL PARADIGM . We now describe the evaluation paradigm summarized in figure 1 ( section 3.1 ) , with special attention to the scrambling functions F that we consider ( sections 3.2–3.3 ) . 3.1 EVALUATION PIPELINE . Figure 1 shows our main evaluation paradigm for testing the transferability of a model without word identity information . On the left side , we show the classic fine-tuning pipeline ( i.e. , we fine-tune on the original English training set and evaluate on the original English test set ) . On the right side , we show our new evaluation pipeline : starting from a single model , we ( 1 ) fine-tune it with a corrupted training split where regular English word identities are removed and then ( 2 ) evaluate the model on a version of the evaluation set that is corrupted in the same manner . The paradigm applies equally to models without any pretraining and with varying degrees of pretraining for their model parameters . 3.2 SCRAMBLING WITH SIMILAR FREQUENCY . To remove word identities , we scrambled each sentence in each dataset by substituting each word w with a new word w′ in the vocabulary of the dataset . For Scrambling with Similar Frequency , we use the following rules : 1. w and w′ must have the same sub-token length according to the BERT tokenizer ; and 2. w and w′ must have similar frequency . The first rule is motivated by the concern that sub-token length may correlate with word frequency , given that rarer and longer words may be tokenized into longer sub-tokens . The second rule is the core of the procedure . The guiding idea is that word frequency is often reflected in learned embeddings ( Gong et al. , 2018 ) , so this scrambling procedure might preserve useful information and thus help to identify the source of transfer . Table 5 shows an example , and Appendix C provides details about the matching algorithm and additional examples of scrambled sentences . 3.3 RANDOM SCRAMBLING . To better understand the role of frequency in domain transfer , we also consider a word scrambling method that does not seek to match word frequencies . For this , we simply shuffle the vocabulary and match each word with another random word in the vocabulary without replacement . We include the distributions of the difference in frequency for every matched word pair in Appendix C to make sure a word is paired with a new word with drastically different frequency in the dataset . We also tried to pair words by the reverse order of frequencies , which yielded similar results , so we report only random scrambling results here . 4 MODELS . In this section , we describe the models we evaluated within our paradigm . Appendix B provides additional details about how the models were designed . BERT For our BERT model ( Devlin et al. , 2019a ) , we import weights from the pretrained BERTbase model through the HuggingFace transformers library ( Wolf et al. , 2020 ) . For sequence classification tasks , we append a classification head after the [ CLS ] token embedding in the last layer of the BERT model . If an input example contains a pair of sentences , we concatenate them using a [ SEP ] token in between . For sequence labeling tasks , we append a shared classification head to each token embedding in the last layer of the BERT model . LSTM We contextualize our results against a strong LSTM-based model ( Hochreiter & Schmidhuber , 1997 ) . We lower-case each input sentence and tokenize it by separating on spaces and punctuation . We then use 300-dimensional GloVe embeddings ( Pennington et al. , 2014 ) 1 as inputs to a single-layer recurrent neural network with LSTM cells , with a hidden size of 64 . We use dot-product attention ( Luong et al. , 2015 ) to formulate a context vector for each sentence . Finally , we pass the context vector through a multilayer perceptron ( MLP ) layer to get the final prediction . For an input example with a pair of sentences , we concatenate two sentences together before feeding them into our LSTM encoder . For sequence labeling tasks , we directly feed the hidden state at each position to the MLP layer to get the final prediction . Bag-of-Words ( BoW ) Model We compare against a BoW classifier , which serves as a proxy of model performance when only given word co-occurrence information . For each sentence in a dataset , we first formulate a BoW vector that uses unigram representations of an input sentence . Then , we feed the BoW vector through a softmax classifier . For examples with a pair of sentences , we create two BoW vectors for each sentence , and concatenate them together before feeding them into the linear layer for predicting labels . For sequence labeling tasks , we use Conditional Random Fields models ( CRFs ; Lafferty et al. , 2001 ) with character-level unigram BoW features . Dummy Model We include a random classifier that generates predictions randomly proportional to the class distribution of the training set . We use this model to further contextualize our results . 1We use the Common Crawl cased version : http : //nlp.stanford.edu/data/glove.840B.300d.zip

Summary: This paper proposes an evaluation pipeline for pre-trained models by testing their transferability without word identity information. Specifically, they take an English pre-trained BERT off-the-shelf and fine-tune it with a corrupted English dataset. Those corrupted texts are designed to remove word identity information while preserving the word frequency information. They conduct experiments on 6 tasks.