Identifying the Limits of Cross-Domain Knowledge Transfer for Pretrained Models

1 INTRODUCTION . Fine-tuning pretrained language models has proven to be highly effective across a wide range of NLP tasks ; the leaderboards for standard benchmarks are currently dominated by models that adopt this general strategy ( Rajpurkar et al. , 2016 ; 2018 ; Wang et al. , 2018 ; Yang et al. , 2018 ; Wang et al. , 2019 ) . Recent work has extended these findings in even more surprising ways : Artetxe et al . ( 2020 ) , Karthikeyan et al . ( 2019 ) , and Tran ( 2020 ) find evidence of transfer between natural languages , and Papadimitriou & Jurafsky ( 2020 ) show that pretraining language models on non-linguistic data such as music and computer code can improve test performance on natural language . Why does pretraining help even across what appear to be fundamentally different domains , and what are the limits of such cross-domain transfer ? In this work , we seek to inform these questions via a systematic exploration of how much cross-domain transfer we see when the model is denied any information about word identity . In this setting , we can vary the pretraining and fine-tuning examples dramatically while holding other aspects of the task constant . This allows us to quantify the extent of transfer , and it can yield insights into the wide-ranging transfer results cited above . Figure 1 gives an overview of our core experimental paradigm : starting with two identical copies of a single pretrained model for English , we fine-tune one on English examples and the other on scrambled English sentences , using a scrambling function F ( section 3 ) , and then we evaluate the resulting models . We apply this paradigm to four classification tasks and two sequence modeling tasks , and we evaluate bag-of-words baselines , LSTMs with GloVe initialization and rich attention mechanisms , and BERT . Our central finding is that only BERT is able to achieve robust cross-domain transfer , and for classification tasks but not sequence labeling ones . To try to understand why such transfer is successful for some tasks but not others , we pursue a number of hypotheses . First , we consider whether using a scrambling function F that matches word frequencies is required for transfer , and we find that such matching plays a small role , but not enough to account for the observed performance ( section 7.1 ) . Second , we assess whether frequency matching might actually be inserting semantic consistency into the scrambling process by , for example , systematically creating substitution pairs like good/great and professor/teacher ( section 7.2 ) . However , we find no evidence of such semantic consistency . Third , we try to isolate the contribution of pretraining versus fine-tuning by fine-tuning randomly initialized models of different sizes ( section 7.3 ) and by freezing the BERT parameters , such that only task-specific parameters are updated ( section 7.4 ) . These variations lead to a substantial drop in transfer , suggesting that finetuning is vital , although our LSTM results show that the BERT pretrained starting point is also an essential component . Fourth , we ask whether the fine-tuning process is primarily learning to reassociate scrambled words with their sources , and we find that it is not ( section 7.5 ) . While these findings do not fully account for the transfer we observe , they offer a partial explanation which should help guide future studies of this issue and which can help with practical fine-tuning work . 2 RELATED WORK . 2.1 EVIDENCE FOR TRANSFER . Transferability across domains is often used to benchmark large pretrained models such as BERT ( Devlin et al. , 2019b ) , RoBERTa ( Liu et al. , 2019b ) , ELECTRA ( Clark et al. , 2019 ) , and XLNet ( Yang et al. , 2019 ) . To assess transferability , pretrained models are fine-tuned for diverse downstream tasks ( Wang et al. , 2018 ; 2019 ) . Recently , pretrained Transformer-based models ( Vaswani et al. , 2017 ) have even surpassed estimates of human performance on GLUE ( Wang et al. , 2018 ) and SuperGLUE ( Wang et al. , 2019 ) . While the benefits of pretraining are reduced when there is a large train set ( Hernandez et al. , 2021 ) , there is little doubt that this pretraining process helps in many scenarios . 2.2 STUDIES OF WHY TRANSFER HAPPENS . There are diverse efforts underway to more deeply understand why transfer occurs . Probing tests often involve fitting supervised models on internal representations in an effort to determine what they encode . Such work suggests that BERT representations encode non-trivial information about morphosyntax and semantics ( Tenney et al. , 2019 ; Liu et al. , 2019a ; Hewitt & Manning , 2019 ; Manning et al. , 2020 ) and perhaps weakly encode world knowledge such as relations between entities ( Da & Kasai , 2019 ; Petroni et al. , 2019 ) , but that they contain relatively little about pragmatics or rolebased event knowledge ( Ettinger , 2020 ) . Newer feature attribution methods ( Zeiler & Fergus , 2014 ; Springenberg et al. , 2015 ; Shrikumar et al. , 2017 ; Binder et al. , 2016 ; Sundararajan et al. , 2017 ) and intervention methods ( McCoy et al. , 2019 ; Vig et al. , 2020 ; Geiger et al. , 2020 ) are corroborating these findings while also yielding a picture of the internal causal dynamics of these models . Another set of strategies for understanding transfer involves modifying network inputs or internal representations and studying the effects of such changes on task performance . For instance , Tamkin et al . ( 2020 ) show that BERT ’ s performance on downstream GLUE tasks suffers only marginally even if some layers are reinitialized before fine-tuning , and Gauthier & Levy ( 2019 ) , Zanzotto et al . ( 2020 ) , Pham et al . ( 2020 ) , and Sinha et al . ( 2021 ) show that BERT-like models are largely insensitive to word order changes . 2.3 EXTREME CROSS-DOMAIN TRANSFER . Cross-domain transfer is not limited to monolingual cases ( Karthikeyan et al. , 2019 ) . With modifications to its tokenizer , English-pretrained BERT improves performance on downstream multilingual NLU tasks ( Artetxe et al. , 2020 ; Tran , 2020 ) . Papadimitriou & Jurafsky ( 2020 ) show that pretraining language models on structured non-linguistic data ( e.g. , MIDI music or Java code ) improves test performance on natural language . Our work complements and advances these efforts along two dimensions . First , we challenge models with extremely ambitious cross-domain settings and find that BERT shows a high degree of transfer , and we conduct a large set of follow-up experiments to help identify the sources and limitations of such transfer . 3 EXPERIMENTAL PARADIGM . We now describe the evaluation paradigm summarized in figure 1 ( section 3.1 ) , with special attention to the scrambling functions F that we consider ( sections 3.2–3.3 ) . 3.1 EVALUATION PIPELINE . Figure 1 shows our main evaluation paradigm for testing the transferability of a model without word identity information . On the left side , we show the classic fine-tuning pipeline ( i.e. , we fine-tune on the original English training set and evaluate on the original English test set ) . On the right side , we show our new evaluation pipeline : starting from a single model , we ( 1 ) fine-tune it with a corrupted training split where regular English word identities are removed and then ( 2 ) evaluate the model on a version of the evaluation set that is corrupted in the same manner . The paradigm applies equally to models without any pretraining and with varying degrees of pretraining for their model parameters . 3.2 SCRAMBLING WITH SIMILAR FREQUENCY . To remove word identities , we scrambled each sentence in each dataset by substituting each word w with a new word w′ in the vocabulary of the dataset . For Scrambling with Similar Frequency , we use the following rules : 1. w and w′ must have the same sub-token length according to the BERT tokenizer ; and 2. w and w′ must have similar frequency . The first rule is motivated by the concern that sub-token length may correlate with word frequency , given that rarer and longer words may be tokenized into longer sub-tokens . The second rule is the core of the procedure . The guiding idea is that word frequency is often reflected in learned embeddings ( Gong et al. , 2018 ) , so this scrambling procedure might preserve useful information and thus help to identify the source of transfer . Table 5 shows an example , and Appendix C provides details about the matching algorithm and additional examples of scrambled sentences . 3.3 RANDOM SCRAMBLING . To better understand the role of frequency in domain transfer , we also consider a word scrambling method that does not seek to match word frequencies . For this , we simply shuffle the vocabulary and match each word with another random word in the vocabulary without replacement . We include the distributions of the difference in frequency for every matched word pair in Appendix C to make sure a word is paired with a new word with drastically different frequency in the dataset . We also tried to pair words by the reverse order of frequencies , which yielded similar results , so we report only random scrambling results here . 4 MODELS . In this section , we describe the models we evaluated within our paradigm . Appendix B provides additional details about how the models were designed . BERT For our BERT model ( Devlin et al. , 2019a ) , we import weights from the pretrained BERTbase model through the HuggingFace transformers library ( Wolf et al. , 2020 ) . For sequence classification tasks , we append a classification head after the [ CLS ] token embedding in the last layer of the BERT model . If an input example contains a pair of sentences , we concatenate them using a [ SEP ] token in between . For sequence labeling tasks , we append a shared classification head to each token embedding in the last layer of the BERT model . LSTM We contextualize our results against a strong LSTM-based model ( Hochreiter & Schmidhuber , 1997 ) . We lower-case each input sentence and tokenize it by separating on spaces and punctuation . We then use 300-dimensional GloVe embeddings ( Pennington et al. , 2014 ) 1 as inputs to a single-layer recurrent neural network with LSTM cells , with a hidden size of 64 . We use dot-product attention ( Luong et al. , 2015 ) to formulate a context vector for each sentence . Finally , we pass the context vector through a multilayer perceptron ( MLP ) layer to get the final prediction . For an input example with a pair of sentences , we concatenate two sentences together before feeding them into our LSTM encoder . For sequence labeling tasks , we directly feed the hidden state at each position to the MLP layer to get the final prediction . Bag-of-Words ( BoW ) Model We compare against a BoW classifier , which serves as a proxy of model performance when only given word co-occurrence information . For each sentence in a dataset , we first formulate a BoW vector that uses unigram representations of an input sentence . Then , we feed the BoW vector through a softmax classifier . For examples with a pair of sentences , we create two BoW vectors for each sentence , and concatenate them together before feeding them into the linear layer for predicting labels . For sequence labeling tasks , we use Conditional Random Fields models ( CRFs ; Lafferty et al. , 2001 ) with character-level unigram BoW features . Dummy Model We include a random classifier that generates predictions randomly proportional to the class distribution of the training set . We use this model to further contextualize our results . 1We use the Common Crawl cased version : http : //nlp.stanford.edu/data/glove.840B.300d.zip

Summary: This work aims to identify the source of transfer learning on neural models.  To this end they set up a series of experiments where models are trained and tested with english data, and, with scrambled (or randomly replaced) data, keeping token-wise sentence length. Scrambled sentences also use replacement tokens keeping token frequency with the original token. Models are tested on standard benchmarks on text classification and sequence modelling, showing a drop in performance when the input is scrambled, and a similar drop with randomized input. Further analysis are meant to test of the scrambling maintains sentence semantics (it doesn't'), if BERT is just retraining (there is some transfer but inconclusive), keeping BERT frozen (it depends on the layer) and finally identifying word identity reassociation (it seems there is not reassociation).