Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: This work aims to identify the source of transfer learning on neural models.  To this end they set up a series of experiments where models are trained and tested with english data, and, with scrambled (or randomly replaced) data, keeping token-wise sentence length.
Scrambled sentences also use replacement tokens keeping token frequency with the original token.
Models are tested on standard benchmarks on text classification and sequence modelling, showing a drop in performance when the input is scrambled, and a similar drop with randomized input.
Further analysis are meant to test of the scrambling maintains sentence semantics (it doesn't'), if BERT is just retraining (there is some transfer but inconclusive), keeping BERT frozen (it depends on the layer) and finally identifying word identity reassociation (it seems there is not reassociation).

main_review: 
The overall presentation is regular. I think authors fail to properly state all the possible explanations to support the existence of transfer learning with scrambled or random input, 
If transfer learning were the explanation of why scrambled input keeps showing transfer learning capabilities an experiment training with the original data while tested on scrambled or vice versa is needed. Why didn't the authors try that configuration?
The analysis section can be largely improved. Three out of four experiments show inconclusive results. If results cannot be interpreted, authors should find other experiments to support (or refute) their claims.
Despite the interesting results and large number of experiments, the inconclusive results make unclear the scientific contribution of this research piece.

Some details:

_Paper's title is somehow misleading "Cross-Domain Knowledge Transfer for Pretrained Models". Authors use just BERT as pretrained model and a LSTM with GloVe embeddings as part of the experimentation.


_ The introduction and the Conclusions are inconsistent:
"we evaluate LSTMs using GloVe embeddings, BERT, and baseline models" vs "we take an English pretrained BERT off-the-shelf and fine-tune it with a scrambled English dataset"


_table 4 is shown on page 6, but referenced on page 8. Please bring the table closer to its reference.

_ "Our leading hypothesis here is that the LSTMs may actually relearn all weights without taking advantage of pretraining". 

summary_of_the_review: Finding are unclear
The paper is difficult to follow
Writing can be improved
Experiments are inconclusive.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The paper studies the generalization of several architectures (BOW, GloVe, LSTM, BERT) under scrambling of vocabularies - with and without frequency constraints. 
main_review: The paper is well-written and does a reasonable job evaluating the robustness of architectures under an artificial form of vocabulary shift. However, the authors fail to convince me why studying this distributional shift would bring new and relevant insights to the table. I think for two reasons: 

(a) The shift is not interesting in the sense that it is related to cross-domain or cross-lingual or cross-time shifts, or any other shifts that we observe in the wild. 
(b) The shift does not seem to add value over other synthetic shifts that have already been studied, e.g., word-level shuffling, character-level shuffling, etc. 

You could have studied many other scrambling functions, so why this one? That said, I liked §7.3, which nicely baselines the scrambled model performance. §7.5 is also nice, except that it’s hard to see why any network would learn to reassociate words in the context of learning a particular task. (Learning to reassociate words is not necessary to solve most tasks. Maybe in machine translation?)

Writing: The paper is overall well-written, but I found the use of the word ‘scrambling’ for a left-to-right in-order replacement of words, misleading. In related work, scrambling refers to scrambling of word order. I would also have liked to see a stronger motivation for focusing on frequency. 

Other weaknesses: 

(i) BERT is old now. We have already seen a lot of studies of BERT (in the so-called ‘Bertology’-literature), but isn’t it time to move on? I’m a little worried about community-wide overfitting of our intuitions about language models. There’s the popular alternatives (RoBERTa, GPT-2, t5, etc.), but also a lot of faster, fairer, more interpretable alternatives. 

(ii) The two sequential labelling tasks are very similar. POS is a give-away of NER. How about finegrained sentiment, coreference resolution, semantic tagging, grammatical error detection, etc.?

(iii) As shown in related work, the GLUE tasks are relatively easy (e.g., solvable without word order information). Maybe also consider harder sequence classification problems?

(iv) Using only a small subset of GLUE tasks looks like cherry-picking. 

(v) It is unclear how sensitive the results for each architecture are to hyper-parameter changes. 
summary_of_the_review: The paper is well-written, but poorly motivated. I simply do not see what we learn from the paper (in the intersection of 'previously unknown' and 'relevant'). In addition, while the experiments are well-designed, the datasets seem cherry-picked, and the models out-dated.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper elaborates on transfer learning and domain adaptation in language models. The
authors argue that there are limits on how much information could be transferred when a
model is scrambled or somehow randomized. They used different strategies to randomize
training data including frequency matching and completely random replacement. They suggest
that only Bert shows high rates of transfer into their scrambled domains in classification tasks.
Their experimental results exhibit importance of pretraining on sequence labeling tasks where
randomness occurs. For example, they showed that identity of words are important and when
they are swapped with completely different frequencies (random) the performance drops
significantly. They also supported the view of Sinha et al. (2021) and Ethayarajh (2019) that BERT
model may preserves frequency better and this is the reason behind its superiority.
main_review: Overall, I think the paper is well-written and debates interesting points in transferability.
However, I want to see some measures of intrinsic evaluation related to the scrambled datasets
as well (e.g., perplexity). My concern is that these scrambling techniques might push your
dataset outside of real word adversarial attacks. I would suggest comparing them all together
to support validity of the scrambling methods. Furthermore, could you every tried replacing
word using ontology or a collection of semantically related words? Please add similar
experiment to show that scrambling with synonyms and antonym have different effects (or
positive vs negative replacements).
summary_of_the_review: I believe regardless  of how good some of transfer learning models perform, we need to elaborate on situations those models collapse. Therefore I think having similar papers truly exhibit weakness and strengths of these models is necessary.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper proposes an evaluation pipeline for pre-trained models by testing their transferability
without word identity information. Specifically, they take an English pre-trained BERT off-the-shelf
and fine-tune it with a corrupted English dataset. Those corrupted texts are designed to remove word identity information while preserving the word frequency information. They conduct experiments on 6 tasks.
main_review: Strength:
1. the paper is mostly well-written and easy to follow.
2. the experiments are well-designed to support the claims

Weakness:
2. I'm not convinced by the results about BERT in S7.3. The confounder factor -- pre-training, is not considered. For instance, for the pre-trained BERT, the word frequencies information is already captured in the parameters (along with many other features that also hold for scrambled text). Such that when fine-tuned on scrambled data (word frequency preserved), it's not surprising that pre-trained BERT can clearly perform better. And the reason why LSTM is more robust is that the model size is very small such that it can relearn all weights. We can observe the same trend for 1-layer-BERT on classification tasks, the gaps are generally smaller than other model variants. Some claims in the paper can be further substantiated if the author can experiment with a BERT that is pre-trained on scrambled English.

3. I also have some doubts about S6.2, in which the author claim that the bad performance is because sequence labeling tasks are more likely to rely on word identities. However, as explored in Hewitt & Liang, those labeling supervisions (although in POS tagging) can be easily learned by fine-tuning. Can authors find a better explanation for the poor performance on labeling tasks?

1. The method proposed in this paper is widely used in many other studies, e.g., in adversarial attacking. But it is still somewhat novel since it is applied to new domains. Also, the observations made are marginally novel or significant, there are not many new results compared to prior observations. The title is about pre-trained models, but only BERT is discussed throughout the paper.
summary_of_the_review: This paper presents some interesting observations, but the methods or findings are only marginally significant or novel.
review_title: null

Meta-review: This work performs an analysis of the generalization ability of pre-trained models under the condition of vocabulary scrambling. The paper is well written and easy to understand. However, a full story and investigation into the cause of the observed transfer under word scrambling is lacking. For example, do more powerful models transfer less because pre-training is more effective?

While the effect described in the paper is interesting, it lacks a solid connection to important areas such as adversarial attacks, cross-lingual domain shifts and doesn't seem to have any effect on the application of fine-tuning to pre-trained models. The experimental section could also be improved with comparisons to other more recent models such as RoBERTa and GPT-2. Even though these more recent models are still Transformer-based models it can help answer the question if more powerful models transfer less under word scrambling as raised by reviewer 6a8W. The results on LSTMs seem to imply this case. We thank the authors for including additional results on DeBERTa but this was insufficient to change the authors' opinion of the value of the paper.
Decision: Reject