Before revision:  We find that a BERT model is strikingly successful at the systematic generalization task we pose using this dataset , and our active manipulations of model-internal vectors help us understand why: despite the densely interconnected nature of the BERT architecture, the learned model embeds modular, general theories of lexical entailment relations .
After revision:  We find that a BERT model is strikingly successful at the systematic generalization task we pose using this dataset , and our intervention experiments bolster this, showing that the causal dynamics of the BERT architecture, the learned model embeds modular, general theories of lexical entailment relations .