Larger Probes Tell a Different Story: Extending Psycholinguistic Datasets Via In-Context Learning (2023)

Abstract: Language model probing is often used to test specific capabilities of models. However, conclusions from such studies may be limited when the probing benchmarks are small and lack statistical power. In this work, we introduce new, larger datasets for negation (NEG-1500-SIMP) and role reversal (ROLE-1500) inspired by psycholinguistic studies. We dramatically extend existing NEG-136 and ROLE-88 benchmarks using GPT3, increasing their size from 18 and 44 sentence pairs to 750 each. We also create another version of extended negation dataset (NEG-1500-SIMP-TEMP), created using template-based generation. It consists of 770 sentence pairs. We evaluate 22 models on the extended datasets, seeing model performance dip 20-57% compared to the original smaller benchmarks. We observe high levels of negation sensitivity in models like BERT and ALBERT demonstrating that previous findings might have been skewed due to smaller test sets. Finally, we observe that while GPT3 has generated all the examples in ROLE-1500 is only able to solve 24.6% of them during probing. The datasets and code are available on Github.

## introduction understanding the limitations of large language models (llms) becomes ever more important with their accelerated adoption and application to reallife tasks. after the original discovery that large lms could perform simple nlp tasks without additional training @xcite , the use of these models has rapidly grown, as have their capabilities @xcite @xcite . as the community invests considerable effort into creating, training, and deploying these models @xcite , it is important to understand the types of data and tasks they might not be well-suited. the field of analysis of pre-trained models has grown rapidly in recent years @xcite @xcite @xcite . methods such as attention pattern analysis @xcite , linear probing @xcite , and zero-shot probing @xcite @xcite allow us to evaluate specific capabilities of pre-trained models. zero-shot methods give us arguably the most clear picture, as they directly probe what the model learned through the upstream task and allow the researcher to target very specific skills such as understanding of negation or role. however, even though these methods do not require training data, producing a good dataset for zero-shot evaluation of these language models is not an easy task. we want these datasets to be clean, diverse and to have enough statistical power to be useful for model comparison @xcite . many existing probing datasets struggle with at least one of these requirements. psycholinguistic datasets used in a study by @xcite have been particularly interesting in that they enabled a comparison between model behavior and human response, including both n400 effects and well-reasoned cloze judgments by human speakers. despite being used in multiple studies since @xcite @xcite , these datasets are quite small, ranging in size from 18 sentence pairs in negation to a maximum of 44 sentence pairs in the role-reversal dataset . in our work, the neg-136-simp and role-88 datasets are dramatically extended. for each of them, we follow the original dataset collection method @xcite , with the exception of n400 amplitude (requires electroencephalography) and cloze probability studies on humans @xcite . to explore different approaches to data ex-tension, we extended negation dataset using two methods 1) a template-based and 2) using gpt3 2 with human-in-the-loop. for role, we only used gpt3 to extend the dataset, as no additional role categories were available in original source paper for this data @xcite . to understand how well language models perform on these extended datasets, we evaluated 22 models, including gpt3, following the methodology from @xcite . compared to the original test sets, we see a significant drop (up to 57%) in accuracy for both role and negation tasks. at the same time most models show higher sensitivity to negation compared to the original dataset. finally, while gpt3 has generated all of the examples in role-1500, it is only able to predict the correct answer in 24.6% cases (top-5 accuracy). albert-v2-xxlarge surpasses gpt3 by 4.6%. ## related work recent research has shown that llms are capable of generating labeled data @xcite @xcite @xcite @xcite . most previous studies used gpt or gpt2 for dataset extension @xcite @xcite . @xcite attempted to generate labels for the unlabelled dataset using gpt3. in our work, we generate the entire dataset, rather than just the labels. @xcite used gpt3 to generate questions, sampling from 100k documents to create the prompts. @xcite and @xcite prompted gpt3 to generate synthetic translation and nli datasets, respectively. lialin et al. (2022) and ettinger (2019) evaluated language models on smaller datasets for negation and role reversal. we extend these datasets to around 1500 data points and evaluate 22 models, including gpt3. to our knowledge, our work is the first to extend psycholinguistic datasets and use it to evaluate an extended set of models. ## evaluation on extended datasets following methodology from lialin et al. (2022), we evaluated 22 models on the newly created negation and role reversal data (figure 1 ). both negation and role reversal tasks were first converted into a masked language modeling task, where the category (the final verb) was replaced with a mask token. for gpt2 and gpt3, the task was converted into a causal language modeling task, where the target was removed, and the model had to predict the target word. the responses of the models were evaluated against the true target. for gpt3, we also performed zero-shot evaluation with two prompts: with and without instructions pre-pended to the input sentence (see appendix a). we used the gpt3-text-davinci-002 model via openai api with the default parameters with max_tokens as 1. gpt3 response was evaluated against gold label. our evaluation metric is the top-5 prediction accuracy, following @xcite . it is the percentage of the model responses where the gold label is among the top five predictions from the model. to assess the reliability of the new datasets, we employed the methodology proposed by @xcite to evaluate the statistical power of the datasets. specifically, we used mcnemar's test to assess the top two models on each dataset. for the role dataset, our primary criterion was accuracy, while for the neg dataset, we used the accuracy on affirmative examples (as in @xcite ). additionally, we conducted a permutation test to check the statistical significance of the differences for extended datasets. the results are discussed in section 5. we validated our new datasets through human evaluation. for this, we randomly selected 100 samples from each of the extended datasets. each of these sentences was presented to two annotators, who were asked to complete the sentences with a one-word response. the completion was compared against target labels. this is a method analogous to the cloze test used traditionally to gauge human language comprehension. we used cohen's kappa to compute inter-annotator agreement. dropped by 40-50% on negation (affirmative-only) and by 20-30% on role reversal for all models except gpt2. even top-20 prediction accuracy couldn't reach the performance shown for the original datasets. for neg-1500-simp-temp, bert 6 , roberta, distilbert and t5 accuracy decreased 30% each, while albert-xxlarge-v2 had the biggest drop of 57%. compared to neg-1500-simp-temp, neg-1500-simp-gen shows 5-10% more (absolute) drop in the performance. in role-1500, the performance of bert, dis-tilbert and t5 decreased by about 5-15% each. roberta and albert showed the highest drop of 23-29% on this task. increasing dataset size had a larger effect on model accuracy for negation than for the role-reversal task. the models that performed best on the small datasets did not remain in the lead on the extended. for example, in role, roberta-large was the best model on the original dataset, but albert-v2-xxlarge surpassed it on the extended one. for negation, bert and al-bert were best originally, but gpt3 surpassed them when the datasets became larger. gpt3 cannot solve, but can generate such examples. while gpt3 is unable to reliably solve scores 40% and model b scores 20%, model b is worse by 20 percentage point and not by 50%. 6 when we mention a model without specifying its size, we refer to the average percentage change for different sizes within that model family used in our experiments. for example, bert performance will include bert-base and bertlarge accuracies. ## conclusion we provide a new resource that dramatically extends existing probing data for negation and role reversal. the psycholinguistic datasets neg-136-simp and role-88 are extended to more reliable and large datasets, each with 1500 data points. we evaluate 22 models using the new data, observing that the absolute accuracy drops for most models on both tasks. however, most models do show an increased sensitivity to negation. strikingly, as seen in the role reversal task, we show that gpt3 may be capable of generating the data for a task while failing to show top performance on it.