Probing for targeted syntactic knowledge through grammatical error detection (2022)

Abstract: Targeted studies testing knowledge of subject-verb agreement (SVA) indicate that pre-trained language models encode syntactic information. We assert that if models robustly encode subject-verb agreement, they should be able to identify when agreement is correct and when it is incorrect. To that end, we propose grammatical error detection as a diagnostic probe to evaluate token-level contextual representations for their knowledge of SVA. We evaluate contextual representations at each layer from five pre-trained English language models: BERT, XLNet, GPT-2, RoBERTa and ELECTRA. We leverage public annotated training data from both English second language learners and Wikipedia edits, and report results on manually crafted stimuli for subject-verb agreement. We find that masked language models linearly encode information relevant to the detection of SVA errors, while the autoregressive models perform on par with our baseline. However, we also observe a divergence in performance when probes are trained on different training sets, and when they are evaluated on different syntactic constructions, suggesting the information pertaining to SVA error detection is not robustly encoded.

## introduction recent work investigates whether linguistic information is encoded in pre-trained transformer-based language models @xcite . research using diagnostic methods @xcite @xcite @xcite indicates models encode syntax via experiments targeting, for example, part-of-speech and dependency labelling @xcite @xcite , while targeted syntactic evaluation studies show models encode a large amount of hierarchical syntactic information in tests for subject-verb agreement @xcite @xcite . although previous research has covered a large number of probing tasks @xcite , no one has yet fully explored grammatical error detection (ged) as a probe. we assert that the ability to detect ungrammatical tokens serves as a complementary evaluation to assess linguistic competence. ged is a natural and complex nlp task that assesses a model's ability to detect which tokens in a sentence are grammatically incorrect. ungrammatical tokens may be categorised within a taxonomy foot_0 comprising three operational categories (replacement, unnecessary, and missing) and twenty-five categories based on parts-of-speech. for example: to do well in the task, a model must encode and make use of a wide array of linguistic information. for example, detecting subject-verb agreement errors in english tests a model's capacity to identify i) verbs, ii) the subjects of the verbs, iii) the grammatical number (singular/plural) of both, and iv) whether their number agrees. the above makes the task a very interesting testbed for evaluating a model's syntactic knowledge. we operationalise the ged task and train probes to detect replacement subject-verb agreement errors (as in example 1) using contextual representations from different hidden layers from five pre-trained english language models -bert @xcite , @xcite , gpt-2 @xcite , roberta @xcite , and electra @xcite . to ensure a robust and thorough evaluation, we leverage existing publicly annotated data from two domains for training: essays by learners of english as a second language from both the cambridge english write & improve + locness (w&i) corpus @xcite and the first certificate in english corpus (fce) @xcite , along with a corpus of automatically extracted edited sentences from native english wikipedia edit histories @xcite . for evaluation, we reframe the minimal-pair dataset from @xcite to create targeted evaluation sets annotated for ged. in doing so, we demonstrate how existing minimal-pair datasets can be leveraged to create challenging and interpretable test sets for ged models foot_2 @xcite . we find that electra, bert, and roberta linearly encode information for the detection of sva errors in the contextual representations of verbs, however, we observe a gap in performance when probes are trained on data from different domains, implying the information is not encoded consistently or robustly. the results show consistent patterns across layers: both bert and electra encode information related to sva errors in the middle-to-late layers, while roberta seems to encode information earlier in the model. probes trained on representations from gpt-2 and xlnet (with uni-and bi-directional decoding) perform poorly on the evaluation set, indicating a fundamental difference from either the training objective or pre-training data. finally, we show that ged probes can complement existing tools for syntactic evaluation: our results suggest that although neural language models perform well on targeted syntactic evaluation tasks, their encoding of sva does not robustly extend to the detection of sva errors. foot_3 ## token-level grammaticality we motivate the use of ged-probes by first reviewing previous literature involving grammaticality judgements and tests for subject-verb agreement, then discuss the advantages in tests for ged. boolean acceptability judgements have long been used as a primary behavioural measure to observe humans' grammatical knowledge @xcite , and have recently been employed in computational linguistics to evaluate grammatical knowledge in neural models. for example, @xcite train classifiers to predict sentence-level boolean acceptability judgements on example sentences from the linguistics literature. as each sentence is designed to demonstrate a particular grammatical construction, performance on the task is interpreted as a reflection of the implicit knowledge of the classifier. an alternative approach frames acceptability as a choice between minimal pairs of sentences -one grammatical and another ungrammatical, where the difference between the two is typically one or two tokens. @xcite evaluate linguistic knowledge by testing whether a language model assigns higher probability to a grammatical sentence relative to its minimally different ungrammatical counterpart. similar to @xcite , fine-grained grammatical knowledge is evaluated by controlling the evaluation stimuli, with the hypothesis that models must have implicit knowledge of the underlying grammatical concept to succeed. rather than evaluating sentence-level scores, @xcite compare predicted probabilities assigned to target verbs in minimal pair sentences, where each sentence in a pair uses a different form of the verb. @xcite extends this to masked language models where he replaces a target verb with the [mask] token and feeds the entire sentence to a bert model. a model is considered successful, and thereby has knowledge related to sva, if it assigns higher probability to the correct form of the verb. our work differs from the above in three important ways. first, we don't assume to know where the incorrect token is -the probe is trained to detect errors for all tokens in a sentence, given each token's contextual representation. this is a more fine-grained evaluation compared to sentence-level judgements and tests whether probes know where the error is located. second, instead of targeting information in the masked token, we investigate whether the model implicitly encodes sva information in a token's contextual representation. third, we test for knowledge of sva without comparing to the counterpart token or sentence. we argue that if a model has knowledge of sva, it should ## experiment 1: per layer probes we first investigate whether models encode svaerrors by examining probing performance at each layer; we want to test whether models encode this information in the final layer, the token representation, but also how this information develops across the layers. we then break-down performance by syntactic construction to better understand how the sva-error encoding generalizes. finally, we carry out a follow-up experiment to investigate the impact of training data size and verb frequency. ## experiment 2: generalisation to unseen verbs in our first experiment, we observe that although the mlms encode more information for sva error detection compared to the alms, the information does not always generalize across domains or syntactic constructions. we carry out a follow-up experiment to investigate whether the information generalizes across verbs using probes trained on bert representations and wiked data. we focus on layers 6 to 12 as these were the layers where performance was above the baseline. there are 13 target verbs in the m&l stimuli, of which "to be" is the most frequent with 946 occurrences in wiked-s. the remaining verbs appear very infrequentlyfor example, eight verbs have frequencies less than 30. to test generalization across verbs we remove all sentences from the training and development sets which contain a verb from the m&l stimuli except for "to be". we then re-sample sentences from the full wiked data to maintain 1936 sentences as in the first experiment. due to the infrequency of many verbs and to ensure a more thorough evaluation, we also increase the training set size by 4-and 8-times to yield training sets with 7744 and 15488 sentences, respectively. we refer to the three sizes as small, medium, and large. this results in paired training sets: for each training set size, there is one set "with m&l verbs" and a set "without m&l verbs". we sample each training set five times and report the mean and standard deviation over the samples. for example, we sample five training sets with 7744 sentences "with m&l verbs", and another five "without m&l verbs". since we are only interested in the performance of 12 verbs, we modify the evaluation stimuli to remove a) sentences containing only "to be" verbs, and for sentences with multiple verbs we remove the "to be" token from evaluation. for example, in the sentence "the movie the security guards like is good", we remove "is" from evaluation. figure 3 illustrates the f 1 scores: the left and centre plots show results for probes trained on data with and without m&l verbs, respectively. the plot on the right presents the mean and standard deviation for the pairwise differences between probes trained on datasets of the same size. the right-hand plot shows that for the smallest training set size we observe a slight benefit when including the m&l verbs, but this is limited to ~0.05 f 1 and restricted to layers 8-10. we generally observe no difference in performance for the medium and large training sets, indicating that sva-error information does generalize across verbs. furthermore, we observe no difference when comparing results across training set sizes in the left and central plots, except for probes trained on the small training set without m&l verbs. these results indicate that sva-error information is linearly accessible and generalizable across across the verbs we test, even when probes are trained with limited data. future work may expand the investigation to cover more verbs, though we expect performance to deteriorate as verbs become infrequent in the pre-training data @xcite . ## discussion our experiments test whether information for sva errors is implicitly encoded in the contextual representations of verbs, but they don't provide any indication as to how the information is encoded: is grammaticality encoded atomically or compositionally? furthermore, we note that selecting the "erroneous token" can be an ambiguous choice between the noun and the verb, for example in "the authors laughs". yet, the probes we evaluate never tag the nouns. this could indicate that a) the probes learn to only tag verbs, and/or b) that sva-grammaticality is disparately encoded between nouns and verbs. a compositional account of grammatical encoding is a plausible explanation given the results provided in @xcite that nouns and verbs have different encodings for number. we plan to investigate how grammaticality is encoded in future work, both in pre-trained language models as well as models trained specifically for ged. ## conclusion we analyse whether pre-trained transformer-based language models implicitly encode knowledge of sva errors using ged probes. we carry out a thorough evaluation on five models, using two public training sets from different domains, and evaluate on a manually constructed evaluation set. this enables us to get a more complete and reliable picture of a models' performance. grammatical error detection is a challenging and linguistically aligned task to assess the knowledge of neural language models; we show that gedprobes can be used as a complementary analysis tool to evaluate a models' linguistic capabilities. our results show that electra, bert, and roberta encode information for sva-error detection, but gpt-2 and xlnet do not. for bert and roberta, we find that the sva-error encoding is not robust across all syntactic constructions or training set domains, though we do find some evidence that the encoding generalizes across verbs for bert. furthermore, a layer-wise analysis reveals the final layers in electra and bert are the most salient for sva-error detection.