When Does Syntax Mediate Neural Language Model Performance? Evidence from Dropout Probes (2022)

Abstract: Recent causal probing literature reveals when language models and syntactic probes use similar representations. Such techniques may yield “false negative” causality results: models may use representations of syntax, but probes may have learned to use redundant encodings of the same syntactic information. We demonstrate that models do encode syntactic information redundantly and introduce a new probe design that guides probes to consider all syntactic information present in embeddings. Using these probes, we find evidence for the use of syntax in models where prior methods did not, allowing us to boost model performance by injecting syntactic information into representations.

## introduction recent large neural models like bert and gpt-3 exhibit impressive performance on a large variety of linguistic tasks, from sentiment analysis to question-answering @xcite . given the models' impressive performance, but also their complexity, researchers have developed tools to understand what patterns models have learned. in probing literature, researchers develop "probes:" models designed to extract information from the representations of trained models @xcite @xcite . for example, @xcite demonstrated that one can train accurate linear classifiers to predict syntactic structure from bert or elmo embeddings. these probes reveal what information is present in model embeddings but not how or if models use that information @xcite . to address this gap, new research in causal analysis seeks to understand how aspects of models' representations affect their behavior @xcite @xcite @xcite . typically, these techniques create counterfactual representations that differ from the original according to some property (e.g., syntactic interpretation of the sentence). researchers then compare outputs when using original and counterfactual embeddings to assess whether a property encoded in the representation is causally related to model behavior. unfortunately, negative results -wherein researchers report that models do not appear to use a property causally -are difficult to interpret. such failures can be attributed to a model truly not using the property (true negatives), or to a failure of the technique (false negatives). for example, as depicted in figure 1 , if a language model encodes syntactic information redundantly (here illustrated in two-dimensions), the model and probe may differentiate among parses along orthogonal dimensions. when creating counterfactual representations with such probes, researchers could incorrectly conclude that the model does not use syntactic information. in this work, we present new evidence for the causal use of syntactic representations on task performance in bert, using newly-designed probes that take into account the potential redundancy in a model's internal representation. first, we find evidence for representational redundancy in bertbased models. based on these findings, we propose a new probe design that encourages the probe to use all relevant representations of syntax in model embeddings. these probes are then used to assess if language models use representations of syntax causally, and, unlike prior art, we find that some fine-tuned models do exhibit signatures of causal use of syntactic information. lastly, having found that these models causally use representations of syntax, we used our probes to boost a questionanswering model's performance by "injecting" syntactic information at test time. foot_0 2 related work ## experiments here, we report the results from three experiments establishing the benefits of dropout probes. first, we found evidence supporting our hypothesis of redundantly-encoded syntactic information by calculating the mutual information between various activations in trained networks. second, we compared dropout probes to standard probes in a set of syntactically-ambiguous test domains. we found that our method revealed evidence supporting the causal use of syntax in models where other methods did not @xcite . lastly, given our findings that models used syntax causally, we demonstrated how one could "inject" syntactic information into models to improve performance in syntactically-challenging tasks. experiments were conducted on four models, all based on huggingface's bert-base-uncased @xcite . the mask model was the original model, trained on a masked language modeling task and next-sentence prediction @xcite . the qa model was fine-tuned on the stanford question answering dataset 2.0 @xcite . 2 lastly, we trained two models, dubbed nli and nli-hans, that were finetuned on the multi-genre natural language inference dataset or that dataset augmented with the heuristic analysis for nli systems (hans) dataset, respectively @xcite . the mask model was used to compare our method to @xcite , who found that such models used syntactic information causally. the qa model was used to study a finetuned model; prior art did not find evidence of causal use. lastly, the nli models were used because natural language inference is recognized as a difficult linguistic task that models appear to "cheat" on by leveraging spurious correlations in datasets ( mccoy 2019; @xcite . ## contributions and conclusion in this work, we designed and evaluated "dropout probes," a new neural probing architecture for generating useful causal analysis of trained language models. our technical contribution -adding a dropout layer before probes -was inspired by a theory of redundant syntactic encodings in models. our results fit within three categories: we showed that 1) models encoded syntactic information redundantly, 2) dropout probes, unlike standard probes, revealed that qa models used syntactic representations causally, and 3) by injecting syntactic information at test time in syntacticallychallenging domains, we could increase model performance without retraining. despite our step towards better understanding of pretrained models, future work remains. natural extensions include studying pretrained models beyond those considered in this work, further research into redundancy in embeddings, more investigation into inserting symbolic knowledge into neural representations, and new methods for training models to respond appropriately to interventions.