Larger Probes Tell a Different Story: Extending Psycholinguistic Datasets Via In-Context Learning
Authors: Namrata Shivagunde
Vladislav Lialin
Anna Rumshisky
Venue: EMNLP 2023 Main (2023)
Keywords: Psycholinguistic datasets
Negation
Role Reversal
Larger Dataset
ICL
TL;DR: New and large psycholinguistic datasets for negation and role reversal task.

Abstract: Language model probing is often used to test specific capabilities of models. However, conclusions from such studies may be limited when the probing benchmarks are small and lack statistical power. In this work, we introduce new, larger datasets for negation (NEG-1500-SIMP) and role reversal (ROLE-1500) inspired by psycholinguistic studies. We dramatically extend existing NEG-136 and ROLE-88 benchmarks using GPT3, increasing their size from 18 and 44 sentence pairs to 750 each. We also create another version of extended negation dataset (NEG-1500-SIMP-TEMP), created using template-based generation. It consists of 770 sentence pairs. We evaluate 22 models on the extended datasets, seeing model performance dip 20-57% compared to the original smaller benchmarks. We observe high levels of negation sensitivity in models like BERT and ALBERT demonstrating that previous findings might have been skewed due to smaller test sets. Finally, we observe that while GPT3 has generated all the examples in ROLE-1500 is only able to solve 24.6% of them during probing. The datasets and code are available on Github.

Larger Probes Tell a Different Story:
Extending Psycholinguistic Datasets Via In-Context Learning
Namrata Shivagunde, Vladislav Lialin, and Anna Rumshisky
Department of Computer Science
University of Massachusetts Lowell (nshivagu, vlialin, arum]@cs. uml edu
Abstract
The field of analysis of pre-trained models has grown rapidly in recent years (Zagoury et al., 2021; Liu et al., 2021; Lialin et al., 2022; bench authors, 2023; Rogers et al., 2020). Methods such as at- tention pattern analysis (Kovaleva et al., 2019; Kobayashi et al., 2020), linear probing (Tenney et al. 2019), and zero-shot probing (Belinkov et al.. 2020; Talmor al., 2019; Ettinger, 2019; Lialín et al., 2022) allow us to evaluate specific capabili- ties of pre-trained models. Zero-shot methods give us arguably the most clear picture, as they directly probe what the model learned through the upstream task and allow the researcher to target very specific skills such as understanding of negation or role.
Language model probing is often used to test specific capabilities of models. However, con- clusions from such studies may be limited when the probing benchmarks are small and lack statistical power. In this work, we introduce new, larger datasets for negation NEG-1500- SIMP) and role reversal (ROLE-IS) 1500) inspired by psycholinguistic studies. We dramatically extend existing NEG-136 and ROLE-88 bench- marks using GPT3, increasing their size from 18 and 44 sentence pairs to 750 each. We also create another version of extended negation dataset (NEG-1500 SIMP-TEMP), created us- ing template-based generation. It consists of 770 sentence pairs. We evaluate 22 models on the extended datasets, seeing model perfor- mance dip 20-57% compared the original smaller benchmarks. We observe high levels of negation sensitivity in models like BERT and ALBERT demonstrating that previous findings might have been skewed due to smaller test sets. Finally, we observe that while GPT3 has gen- erated all the examples in ROLE-1500 is only able to solve 24.6% of them during probing. The datasets and code are available on Github'.
However, even though these methods do not re- quire training data, producing a good dataset for zero-shot evaluation of these language models is not an casy task. We want these datasets to be clean, diverse and to have enough statistical power to be useful for model comparison (Card al., 2020). Many existing probing datasets struggle with at least one of these requirements.
Psycholinguistic datasets used in study by Et- tinger (2019) have been particularly interesting in that they enabled comparison between model be- havior and human response, including both N400 effects and well-reasoned cloze judgments by hu- man speakers. Despite being used in multiple stud- ies since (Lialin et al., 2022; Rogers et al., 2020; Zhang et al.. 2020), these datasets are quite small, ranging in size from 18 sentence pairs in negation (NEG-136-SIMP)) to maximum of 44 sentence pairs in the role-reversal dataset (ROLF-88).
1 Introduction
Understanding the limitations of large language models (LLMs) becomes ever more important with their accelerated adoption and application to real- life tasks. After the original discovery that large LMS could perform simple NLP tasks without ad- ditional training (Radford et al., 2019), the use of these models has rapidly grown, as have their ca- pabilities (Brown et al., 2020; Sanh al., 2021; Chowdhery et al., 2022). As the community in- vests considerable effort into creating, training, and deploying these models (Zhang et al., 2022; Black et al., 2021), it is important to understand the types of data and tasks they might not be well-suited.
In our work, the NEG-136-SIMP and ROLE- 88 datasets are dramatically extended. For each of them, we follow the original dataset collec- tion method (Fischler et al., 1983; Chow al., 2016), with the exception of N400 amplitude (re- quires electroencephaloggaphy and cloze proba- bility studies on humans (Federmeier and Kutas, 1999). To explore different approaches to data ex-
'htrps//github.c. wwtps//ctc/ott//ceeeeceee machine- lahfcxtending aablcctending__syycllinggg pwycholigguuisticccccccccooocc datasct

---

tension, we extended negation dataset using two methods 1) a template-based and 2) using GPT3² with human-in-the-loop. For ROLE, we only used GPT3 to extend the dataset, as no additional role categories were available in original source paper for this data (Chow et al., 2016).
sentence e.g. the sentences "A robin is bird" (af- firmative) and "A robin is not tree" (negated) form a pair. We extend this dataset using two methods: template- based and generation based. By employing both methods, we could gauge the po- tential strengths and limitations inherent to cach method, which we discuss in Section 5. We create 770 sentence pairs with the template- based method and generate 750 pairs using GPT3. We refer to these datasets as NEG-1500-SIMP-GEN) respectively.
To understand how well language models per- form on these extended datasets, we evaluated 22 models, including GPT3, following the methodol- ogy from Lialin al. (2022). Compared to the original test sets, we see a significant drop (up to 57%) in accuracy for both Role and Negation tasks. At the same time most models show higher sensi- tivity to negation compared to the original dataset. Finally, while GPT3 has generated all of the ex- amples in ROLE-1500, it is only able to predict the correct answer in 34.6% cases (top-5 accuracy). ALBlRT--2-xxlarge surpasses GPT3 by 4.6%
and
NEG-1500-SIMP-TEMM Each pair in the orig- inal dataset follows a template. For affirmative sentences, the template is "A/An (subject] is alan [object)". Its negated version is "A/An [subject] is not a/an fanother object/". Battig and Montague (1969) and Fischler et al. (1983) provide template and a list of objects (e.g., robin, pine) and the cor- responding categories (e.g.. bird, tree) which are used to generate samples 3. For instance, the cate- gory, "bird" in the example, "A robin a bird" is replaced with another category ("tree") to generate the negated example "A robin is not tree".
2
Related Work
Recent research has shown that LLMs are capa- ble of generating labeled data (Anaby-Tavor et al.. 2020; Papanikolaou and Pierlconi, 2020; Kumar et al., 2020; Meng et al., 2022; Mohapatra et al., 2020; Yang et al., 2020; Li et al., 2022; Wu et al., 2022). Most previous studies used GPT or GPT2 for dataset extension (Schick and Schütze, 2021; Gao et al., 2022; Whitfield, 2021; Anaby-Tavor et al., 2020). Smith et al. (2022) attempted to gen- erate labels for the unlabelled dataset using GPT3. In our work, we generate the entire dataset, rather than just the labels. Bonifacio et al. (2022) used GPT3 to generate questions, sampling from 100k documents to create the prompts. Han et al. (2021) and Liu al. (2022) prompted GPT3 to generate synthetic translation and NLI datasets, respectively. Lialin et al. (2022) and Ettinger (2019) evalu- ated language models on smaller datasets for nega- tion and role reversal. We extend these datasets to around 1500 data points and evaluate 22 models, including GPT3. To our knowledge, our work is the first to extend psycholinguistic datasets and use it to evaluate an extended set of models.
We experimented with using WordNet to gener- ate from this template, but found that it required careful curation of hypernym hierarchies. Since our goal was to reduce human effort and cost, we decided against it. We used a template from Fis- chler et al. (1983) along with the categories and subcategories listed in Battig and Montague (1969) to create 770 sentence pairs. Similar to Ettinger (2019), we did not use multi-word category names.
We use human-in-the loop and few-shot prompted -eT3-text-davinei. 002 with default parameters, to generate 750 pairs (1500 sentences). Each GPT3 prompt consists of an instruction and four randomly selected in-context examples. The cost of generating the dataset is around $30. An example of the prompt is shown in the appendix A. After generating samples from GPT3, the dataset was manually cleaned to filter out poorly generated examples. Details about man- ual filtering in mentioned in Appendix B.I.
NEG-1500-SIMP-GEN
3 Data generation
We analyzed the word distribution for the ex- tended datasets and found that for NEG- SIMP-GEN, few categorics are more frequent than others, for example, the highest frequent category "animal" has three times more examples than the
3.1 Negation Dataset: NEG-1500-SIMP
The existing negation dataset BEG--13--SIMG from Ettinger (2019) consists of 18 sentence pairs. Each pair made of one affirmative and one negated
FFischler et al. (1983) refers to these as "subjects" and "objects", respecticlly...
We use text-davinci 002 version of GPT3
davinci--e22

---

lowest frequent category "tree". This difference is 1.5 times in NEG-150-SIMP-TEMMP Appendix D show the top 20 categories for datasets.
For GPT3, we also performed zero-shot evalua- tion with two prompts: with and without instruc- tions pre-pended to the input sentence (see Ap- pendix A). We used the GPT33text-davinci-o0 model via OpenAl API with the default parame- ters with max tokens as 1. GPT3 response was evaluated against gold label. Our evaluation metric is the top-5 prediction curacy, following Etünger (2019). It is the percent- age of the model responses where the gold label is among the top five predictions from the model. Table shows the results for the original and ex- tended datascts. We also included top 1. 10, and 20 accuracies in Appendix F. For the negation dataset, used model sensitivity Since we didn't have N400 amplitude and cloze probability for the extended datasets, we defined sensitivity as the percentage of sentence pairs for which the top-1 prediction changed, when "not" was added. Table shows the sensitivity result. Re- sults for all models are shown in Appendix E. We could not compute ROLE-1500 sensitivity, as in some sentence pairs the target word was the same. To assess the reliability of the new datasets, employed the methodology proposed by Card et al. (2020) evaluate the statistical power of the datasets. Specifically, we used McNemar's test to assess the top two models on each dataset. For the ROLE dataset, our primary criterion was accuracy, while for the NEG dataset, we used the accuracy on affirmative examples (as in Ettinger (2019)). Additionally, we conducted permutation test check the statistical significance of the differences for extended datasets. The results are discussed in Section
3.2 Role-reversal Dataset: ROLE-1500
The original ROLE-88 dataset (Ettinger, 2019; Chow et al., 2016) is a role reversal dataset consist- ing 88 sentences (44 sentence pairs). Here is an example of a sentence pair: "The librarian docu- mented which journalist the celebrities had avoided." "The librarian documented which celebrities the journalist had interviewed". The first sentence has two words representing roles (here: journal- ist, celebrities), which are reversed in the second sentence. The target verb (the last verb in the sen- tence) is always preceded by the auxiliary "had"" This dataset was created to observe the effect of role reversal on the target verb. We notice that all of the role words represent human roles (chef, worker, etc.) animals (whale, dog, etc.). There were some pairs where the role reversal didn't 't change the target word, but they were semantically correct.
additional metric.
To extend ROLE dataset, we used the same and generated 1500 sentences with temperature set to 0.64 4, We call this new dataset ROLE-1500 and cost of generating is $25. After generating samples from GPT3, the dataset was manually cleaned. An ex- ample of the prompt, details on data filtering and word distribution for ROLE- 1500 is provided the Appendix A, B.2, and D respectively. Word dis- tribution analysis in COLE-1500 revealed three- fold difference between the highest and lowest fre- quency categories, similar to the findings in NEG- 1500-SIMP-GEN
method as BLGG-1500-IMPP
We validated our new datasets through human evaluation. For this, we randomly selected 100 samples from each of the extended datasets. Each of these sentences who were asked complete the sentences with one-word response. The completion was compared against target labels. This is method analogous to the cloze test used traditionally to gauge human language comprehension. We used Cohen's kappa to compute inter-annotator agreement.
4 Evaluation on Extended Datasets
Following methodology from Lialin et al. (2022), we evaluated 22 models on the newly created nega- tion and role reversal data (Figure I). Both nega- tion and role reversal tasks were first converted into a masked language modeling task, where the category (the final verb) was replaced with a mask token. For GPT2 and GPT3, the task was converted into a causal language modeling task, where the target was removed, and the model had to predict the target word. The responses of the models were evaluated against the true target.
presented to two annotators,
Results
Models performance dropped substantially extended datasets. Model performance
"We set Hyperparameters by manually ovaluating the out- put and selecting values that produced samples close to the specifications mentioned in the original dataset papers.
Differennc in the model performance is mentioned as the difference in percentage points. For example, if model A

---

NEG-1500 SIMP TEMP(A(D)
NEG-1500 SIMP GEN(A()
NEG-1500 SIMP-TEMP
NEG-1500
NEG-136 SIMP(A())
NEG- 136 Sensitivity
ROLE- 1500
ROLE-88
SIMP-GEN
Sensitivity
Sensitivity
BERT
27.3
100.0
16.7
20.3
58.4
55.3
53.5
35,9
BERT large
37.5
100.0
33.3
21.5
65.1
53.8
53.5
40.3
RoBERTabas
46.6
94.4
66.7
23.0
62.1
44.0
64.4
71.5
RoBKR'I Falagee
55.7
94.4
50.0
26.1
63.4
53.7
64.5
69.5
DistilBERT base
28.4
94.4
27.8
19.3
57.3
$2.8
44.7
41.5
AIBERTv24 v2
26.1
33.3
38.9
15.3
10.0
11.7
37.3
35,9
AIBERTv2.aag
37.5
94.4
27.8
21.1
40.4
47.8
52.7
51.1
AIBERT 2exhaage
50.0
100.0
44.4
29.0
42.9
45.3
55.7
61.5
TSaagee
36.4
94.4
50.0
18.2
60.4
49.8
44.0
0.0
T5a1
44.3
83.3
66.7
21.5
60.9
60.9
68.2
70.0
GPT2
0.0
66.7
11.2
48.3
37.7
60.4
56.4
GPT22
0.0
63.6
52.8
59.0
GPT3
44.4
65.9
63.3
100.0
100.0
GPT3 preomps
38.6
72.2
100.0
55.9
55.2
100.0
100.0
Table Zero-shot top-5 word prediction accuracy and sensitivity (top-5 over the whole vocabulary). ROLE-88 and NEG-136 from Lialin et al. (2022). ROLE-1500 and NEG-1500 are the new datasets. The best result for each task is in bold. "SIMP" stands for simple, "prompt" stands for prompting(addigg instruction). The negation task is evaluated in the affirmative form (Aff). Sensitivity the percentage of sentence pairs for which the top-1 prediction changed. See Appendix E for accuracy and sensitivity of all models extended datasets.
dropped by 40-50% on negation (affirmative-only) and by 20-30% on role reversal for all models except GPT2. Even top-20 prediction accuracy couldn't reach the performance shown for the original datasets. For NEGG-1500-SMMP-TEEMEP BERTS, RoBERTa, DistilBERT and T5 accuracy decreased 30% each, while ALBERT-xxlagge- had the biggest drop of 57%. Compared NEG- shows
these tasks, perfectly capable of picking up the pattern and generating valid examples. ROLE- 1500 was generated by GPT3, and yet, RoBERTa- large, and ALBERT-xxlarge perform better on it than GPT3. There is a gap of around 4.5% be- tween the best model and GPT3. Adding instruc- tions to the prompt, didn't change the performance on ROLE- 1500, but decreased the accuracy by 8- 10% on negation. Sensitivity to negation remains the same for GPT3 with and without instruction prompts.
1500-SIMP-TEMP, NEG-1500SSIMP 5-10% more (absolute) drop in the performance. In ROLE-1500, the performance of BERT, Dis- niBERT and T5 decreased by about 5-15% each. RoBERTa and ALBERT showed the highest drop of 23-29% this task. Increasing dataset size had larger effect on model accuracy for negation than for the role-reversal lask. The models that per- formed best on the small datasets did not remain in the lead on the extended. For example, in ROLE, RoBeRTa-large was the best model on the original dataset, but ALBERT-v2-xxlaree surpassed it on the extended one. For negation, BERT and AL- BERT were best originally, but GPT3 surpassed them when the datasets became larger.
GPT3 responses contain variety of mistakes. Approximately 2/3 of the generated ROLE data was manually filtered out. 14.3% of responses were sin- gle sentences without pair. 7.4% responses were semantically incorrect. Similar kind of errors were filtered in the negation data as well. GPT3 also generated duplicate entries. 62.5% of negation data generated with GPT3 were duplicate samples, compared to 32.5% for the role reversal dataset.
GPT3 generates more diverse examples than the template-base method. While GPT-3 provided way to generate diverse samples, the template- based method ensured consistency and alignment with the original dataset. Our result shows that GPT-3 generated samples offered variety with 63 unique target labels whereas the template-based method provided controlled variability based on predefined structures with 23 unique target labels. of these target labels are common between both
GPT3 cannot solve, but can generate such ex- amples. While GPT3 is unable to reliably solve
scores 40% and model scores 20%, model worse by 20 percentage point not by 50%.
"When we mention a model without specifying its size, we refer to the average percentage change for different sizes within that model family used in our experiments. For exam- ple, BERT performance will include BERT-base and BERT- large accuracics.

---

the datasets. The models exhibit similar perfor- mance when evaluated on datasets produced by both template as well as model based methods.
in distinguishing top models. However, for the cemplate-generatee NEG-1500-SIMP, the power remained low: 0.04 for the top two models and 0.23 for the top 1 versus top 3. From these re- sults, it is evident that ROLF-1500 and NEG- 1500-SIMP-GEN are dramatically more reliable datasets. Specifically, they can distinguish between small effects of approximately 0.03 for ROLE and about 0.15 for NEG- NEG-1500-ShD 500-SSMPP-ENN (minimum detectable effect, MDE).
Models are more sensitive to negation for ex- tended dataset. All models except ALBERT- v2-base, T5-large, and GPT2-base show higher sensitivity to negation on both extended nega- tion datasets compared to the original one. For NEG-I500-SIMP-TEMPP the sensitivity for BERT and ALBERT-xl increased by 36.8% and 21-25% respectively. For RoBERTa-base, T5-large and GPT2, sensitivity to negation dropped by 2.3- 6%. The sensitivity to negation on NEG-1500- SIMP-GEN either increased remained the same compared to original data for all models, except ALBERT-v2-basee T5-large and GPT2-base. For BERT and RoBERTa the sensitivity increased by al- most 20%. Results demonstrate high levels of nega- tion sensitivity in models like BERT and ALBERT, suggesting that previous findings might have been skewed due to smaller test sets.
The permutation test shows a statistically signifi- cant difference between the accuracies of the top- 2 models for ROLE-1500 and NEG-1500-SIMP. GEN a 0.05 significance level. P-values are 0.01 and 0,0002 respectively. In contrast, template gen- erated dataset show statistically significant accuracies and fails the test with a p-value of 0.22.
does not
Human performance surpasses all model perfor- mance. From human cloze test on ROLE-1500 and NEG- NEG--1500SIMPPTEEN
and GEN), we ob- served that human performance consistently ex- ceeded model performance across all datasets. Specifically, for evaluation resulted in top-1 accuracy of 58.5%, whereas the top-performing model, T5-xl, achieved about 40%. In the case of JEGG-550-SIMPP
Change in model performance on extended datasets depends on its architecture and size. Encoder-only models had the highest performance drop of 15% for ROLF-1500 and 40% for NEG- 1500 (average of template-basse based and generated negation dataset). In comparison, the seq-to-seq models accuracy decreased by 14% in ROLE-1500 and 29% in VEG-1500, while decoder-only mod- els show a gain of 5% in ROLE-1500 and 27% in NEG- Encoder-only models also demon- strate the highest increase in negative sensitivity, with gain of 13%, while seq-to-seq and decoder- only models gains only 1.2% and 7.4% respectively. When analyzed across model size, we found mod- els of size around 60M show the highest drop of around 35% in NEG-1500, while models of size close to 300M experience the highest decrease of 14.6% in the ROLE dataset. The 300M size mod- els exhibit the highest gain of 44% in negativity sensitivity (see tables in Appendix G).
human
the human evaluation yielded an accuracy of 45.5%, 5,, 5.8% higher than the leading model, T5-xl. For ROLE, human performance (10%) is better than all of the model performances except ALBERT- xxlarge (vl and v2). Human evaluation results are included in Tables 3, 4 and 5. Inter-annotator agree- ment for ROLE-1500 is 0.07, whereas for NEG- 1500-SIMP-TEMP and NEG-1500-SIMP-GEN, it is 0.50 and 0.45 respectively.
6 Conclusion
We provide a new resource that dramatically ex- tends existing probing data for negation and role reversal. The psycholinguistic datasets NEG- SIMP and ROLE-88 are extended to more reliable and large datasets, each with 1500 data points. We evaluate 22 models using the new data, observing that the absolute accuracy drops for most models on both tasks. However, most models do show an increased sensitivity to negation. Strikingly, as seen in the role reversal task, we show that GPT3 may be capable of generating the data for a task while failing to show top performance on it.
New extended datasets are more reliable than original small datasets. The power analysis us- ing McNemar's 's test reveals very low power for original datasets. ROLE-88 88 has power of 0.26 whereas NEG-136's power is 0.01. For the ex- tended dataset ROLE-1500, the power to differen- tiate between the top two accuracy models signifi- cantly improved to 0.72, and for GPT3-generated NEG-1500-SIMP SIMP, the power reached a full 1.00

---

Limitations
replication and extension of the connecticut category norms. Journal of Experimental Psychology, 80:1- 46.
The current work has several limitations. The study did not include N400 amplitude. Another limitation is the smaller size of the original datasets. As we had a limited number of in-context samples that were drawn from original datasets, there was a limitation on the number of new data points we could generate. Additionally, the cost of generating samples with GPT3 another limiting factor. Lastly, the frequency distribution of the categories in the new datasets is not balanced, which may introduce some bias in model evaluation. This imbalance exists in the original datasets too.
Yonatan Belinkov, Sebastian Gehrmann, and Ellic Pavlick. 2020. Interpretability and analysis in neural NLP. In Proceedings of the 58th Annual Meeting of the Asvociation for Computational Linguistics: Tu- torial Abstracts, pages 1-5, Online, Association for Computational Linguistics.
BIG bench authors. 2023. Beyond the imitation game: Quantifying and extrapolating the capabilities of lan- guage models. Transactions on Machine Learning Research.
Sid Black, Leo Gao, Phil Wang, Connor Leahy, and Stella Rose Biderman. 2021. Gpt-neo: Large scale autoregressive language modeling with mesh- tensorflow.
Ethics Statement
Using large closed source language models to gen- erate evaluation data, while highly effective, should be approached with caution. First, neural networks of hundreds of billions of parameters are compu- tationally expensive to infer and potential carbon dioxide emissions should be taken into consider- ation. Second, while API access to such models allows to avoid many technical challenges, espe- cially having enough resources to inference large models, it comes with limitations. For example, GPT3 can only return the logprops of at most five top tokens over the vocabulary. Also, the API pro- vides no control over the random seed used for sampling reducing the potential reprehensibility of the research. Finally, without the direct access to the model weights, there is no guarantee that this version of the model will be always accessible to researchers in the future and won't be restricted, modified, deleted, or lost.
Luiz Bonifacio, Hugo Abonizio, Marzieh Fadace, and Rodrigo Nogueira. 2022. Inpars: Data augmentation information retrieval using large language models.
Tom Brown, Benjamin Mann, Nick Ryder, and et. al Subbiah. 2020. Language models are few-shot learn- ers. In Advances in Neural Information Processing Systems, volume 33, pages 1877-1901. Curran Asso- ciates, Inc.
Dallas Card, Peter Henderson, Urvashi Khandelwal. Robin Jia, Kyle Mahowald, and Dan Jurafsky. 2020. With little power comes great responsibility. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 9263-9274, Online. Association for Computa- tional Linguistics.
Wing- Yee Chow, Cybelle Smith, Ellen Lau, and Colin Phillips. 2016. A "bag-of--rgumment"" mechanism for initial verb predictions. Language, Cognition and Neuroscience, 31(5):577-55)
Aakanksha Chowdhery, Sharan Narang, and Jacob De- vlin et. al. 2022. Palm: Scaling language modeling with pathways. ArXiv, abs/2204.02311.
Acknowledgment
This project is funded in part by an NSF CAREER award to Anna Rumshisky (1IS-1652742) We would like to express our gratitude to Vijeta Desh- pande and Sherin Muckatira for participating in human evaluation.
Allyson Ettinger. 2019. What bert not: Lessons from a new suite of psycholinguistic diagnostics for lan- guage models. Transactions of the Association for Computational Linguistics, 8:34-48.
Kara D. Federmeier and Marta Kutas. 1999. A rose by any other name: Long-term memory structure and sentence processing. Journal of Memory and Language, 41 11:469-995
References
Ateret Anaby-Tavor, Boaz Carmeli, Esther Goldbraich, Amir Kantor, George Kour, Segev Shlomov, Naama Tepper, and Naama Zwerdling, 2020. Do not have enough data? deep learning the rescue! Proceed- ings the the 34:7383-7390.
Ira Fischler, Paul A Bloom, Donald G Childers, Salim E Roucos, and Nathan W Perry Jr. 1983. Brain poten- tials related to stages of sentence verification. Psy- chophysiology, 20(4):400
AAAI Conference on Artificial Intelligence.
Jiahui Gao, Renjie Pi, Yong Lin, Hang Xu, Jiacheng Ye, Zhiyong Wu, Xiaodan Liang, Zhenguo Li, and Lingpeng Kong. 2022. Zerogen* Self-guided high- quality data generation in efficient zero-shot learning.
William Battig and William Edward Montague. 1969. Category norms of verbal items in 56 categories a

---

Jesse Michael Han, Igor Babuschkin, Harrison Edwards, Arvind Neelakantan, Tao Xu, Stanislas Polu, Alex Ray, Pranav Shyam, Aditya Ramesh, Alec Radford, and Ilya Sutskever. 2021. Unsupervised neural ma- chine translation with generative language models only. ArXiv, abs/al 10.05448.
Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and llya Sutskever. 2019. Language models are unsupervised multitask learners.
Anna Rogers, Olga Kovaleva, and Anna Rumshisky. 2020. A primer in bertology: What we know about how bert works. Transactions of the Association for Computational Linguistics, 8:842- 866.
Goro Kobayashi, Tatsuii Kuribayashi, Sho Yokoi, and Kentaro Inui. 2020. Attention is not only a weight: Analyzing transformers with vector norms. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). pages 7057-7075.
Victor Sanh, Albert Webson, and Colin Raffel et. al. 2021. Multitask prompted training enables zero-shot task generalization. ArXiv, abs/21 10.08207.
Timo Schick and Hinrich Schütze. 2021. Generating datasets with pretrained language models.
Olga Kovaleva, Alexey Romanov, Anna Rogers, and Anna Rumshisky. 2019. Revealing the dark secrets BERT. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference Natu- ral Language Processing TEMNLP-IJCNLA pages 4365 4374, Hong Kong, China. Association for Com- putational Linguisties.
Ryan Smith, Jason A. Fries, Braden Hancock, and Stephen Bach. 2022. Language models in the loop: Incorporating prompting into weak supervi- sion.
Alon Talmor, Yanai Elazar, Yoav Goldberg, and Jonathan Berant. 2019. olmpics-on what language model pre-training captures. Transactions of the As- sociation for Computational Linguistics, 8:743-758.
Varun Kumar, Ashutosh Choudhary, and Eunah Cho. 2020. Data augmentation using pre-trained trans- former models. CoRR, abs/2003.02245.
Ian Tenney, Dipanjan Das, and Ellie Pavlick. 2019. Bert rediscovers the classical nlp pipeline. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4593-4601.
Siyan Li, Riley Carlson, and Christopher Potts. 2022. Systematicity gpt-3's interpretation of novel en- glish noun compounds.
Dewayne Whitfield. 2021. Using gpt-2 to create syn- thetic data to improve the prediction performance of nlp machine learning classification models.
Vladislav Lialin, Kevin Zhao, Namrata Shivagunde, and Anna Rumshisky. 2022. Life after BERT: What do other muppets understand about language? In Pro- ceedings of the 60th Annual Meeting of the Associa- tion for Computational Linguistics (Volume 1: Long Papers), pages 3180 3180-3193, Dublin, Ireland. Associa- tion for Computational Linguistics.
Yuxiang Wu, Matt Gardner, Pontus Stenetorp, and Pradeep Dasigi. 2022. Generating data mitigate spurious correlations in natural language inference datasets.
Yiben Yang. Chaitanya Malaviya. Jared Fernandez. Swabha Swayamdipta, Ronan Bras, Ji-Ping Wang. Chandra Bhagavatula, Yejin Choi, and Doug Downey, 2020. G-DAUG: generative data augmentation for commonsense reasoning. CORR, abs/2004.11544
Alisa Liu, Swabha Swayamdipta, Noah A. Smith, and Yejin Choi. 2022. Wanli: Worker and ai collaboration for natural language inference dataset creation.
Zeyu Liu, Yizhong Wang, Jungo Kasai, Hannaneh Ha- jishirzi, and Noah A. Smith. 2021. Probing across time: What does RoBERTa know and when? In Findings of the Asvociation for Computational Lin- guistics: EMNLP 2021, pages 920-842, Punta Cana, Dominican Republic. Association for Computational Linguistics.
Avishai Zagoury, Einat Minkov, Idan Szpektor, and William W. Cohen, 2021. What's the best place for Why com-
ai conference, vancouver or pleting comparative questions is difficult. In AAAL.
Susan Zhang. Stephen Roller, and Naman Goyal et. al. 2022. Opt: Open pre-trained transformer language models. ArXiv, abs/2205.01068.
Yu Meng. Jiaxin Huang, Yu Zhang, and Jiawei Han. 2022. Generating training data with language mod- els: Towards zero-shot language understanding. In Advances in Neural Information Processing Systems.
Yian Zhang. Alex Warstadt, Haau-Sing Li, and Samuel R. Bowman. 2020. When do you need billions of words of pretraining data? CoRR. Abs/2011.099..
Biswesh Mohapatra, Gaurav Pandey, Danish Contractor, and Sachindra Joshi. 2020. Simulated chats for task- oriented dialog: Learning to generate conversations from instructions. CoRR, abx/2010.10216.
Yannis Papanikolaou and Andrea Pierleoni. 2020. DARE: data augmented relation extraction with GPT-
CoRR, 0D./2044.1884,

---

A
GPT3 prompt
Instruction is same for ROLE-1500 and NEG- 1500-SIMP-GEN, Only the in-context example changes.
A.1 ROLE-1500 generation prompt
An example prompt for generating ROLE-1500 dataset is given below. The roles are colored orange and blue, whereas the target word is highlighted in green:
B Manual cleaning of the datasets
B.1 NEG-1500-SIMPP
The task is reverse the role in the sentences. Generate more sentence like this:
For NEG-1500-SIMP- to get 1500 cleaned samples, we generated 12000 sentences. 87.5% of the responses were rejected as they were a) dupli- cates (62.5%) b) empty lines (15%) c) only negated sentences (8.75%) d) others (1.25%). Others in- clude sentences like "A toy is toy" and "A toy is an object", as well objects with multiple words and semantically wrong sentences (c.g.. "A flower a rose", "A cloud a weather").. The cost of generating SLGGSIIP--1500-C was $35.
The journalist investigated which athlete the team had recruited, The journalist investigated which team the athlete had joined,
The detective interviewed which witness the doe- tor had suspected, The detective interviewed which doctor the witness had seen.
The teacher lectured which student the class had ignored, The teacher lectured which class the stu- dent had left,
The police reported which criminal the witness had described, The police reported which witness the criminal had robbed,
B.2 ROLE-1500
After generating GPT3 responses to these prompts, the samples were manually cleaned to get 1500 sentences (750 sentence pairs). We generated 4700 sentences (including empty, partial and no-pair sen- tences), of which 68% were removed, a) 32.5% were duplicates, b) 14.3% missed on get- ting a pair or were simple sentences, c) 8.5% were empty lines, d) 7.4% were semantically wrong sen- tences or had wrong role reversal (e.g., "The camper reported which book the girl had eaten"), e) 5.3% were repeating instructions or partial sen- tences. In some instances, generated examples in- cluded some repetition of the role nouns present in the in-context examples.
The doctor treated which patient the nurse had examined, The doctor treated which nurse the patient had injured,
A.2 NEG-1500-SIMP-GEN generation prompt
An example prompt for generating NEG-1500- SIMP-GEN dataset given below.
"The task is to generate affirmative sentences and its negation. The object of the sentence should be hypernym of the subject of the senience. Gen- erate more sentence pairs like these:
C Rules to create NEG-1500-SIMP-GEN
A robin is a bird, robin not a tree,
An affirmative sentence and a negation sentence make pair. Affirmative sentences follow the for- mat of "subject is object" whereas its negation form is "subject not an object". Affirmative sen- tences can be, for example "Robin is a bird" "Robin is an animal", the subject word is "Robin" the object word is "bird" and its immediate super- ordinate category name is "animal". The negation for any of these two affirmative sentences can be "Robin is not a plant" or "Robin is not tree". The target completion (e.g. "plant" "tree") has to be an object superordinate word from an- other category. Figure I in the Appendix 1 shows two categories (Animal and Plant) together with its respective subject and object words (Battig and Montague, 1969).
An oak is a tree, An oak is not a flower, A carrot is a vegetable, A carrot is not bird,
A hammer is tool, A hammer is not tree,"
A.3 ROLE-1500 evaluation prompts with and without instructions
Prompt with instruction for ROLE-1500:
"The goal is to complete the given sentence with one English word. Avoid punctuation or a new line or space. The librarian documented which journalist the celebrities had".
The prompt without instruction: "The librarian documented which journalist the celebrities had". The model predicts the next word in sentence.

---

Sparno
Pine
Rose
Daiy
Figure I: Sentence subject word (Robin). Affirmative sentence completion are in green object word(Bird) and super ordinate category name(Animal). Negative sentence completion are in red object word(Tree) and super ordinate category name(Plant). (Battig and Montague, 1969)
Top 20 target words for all datasets
Figure 2 and 6 show top 20 object words for original datasets NEG-136-SIMM and ROLE-88. Figure 3, and depict top 20 object words for original new extended datasets. axis represents the number of times the target words. NEG-136 SIMP has less than 20 categories, therefore all of them has been shown in the figure 2.
E Result for all models
Table shows the top five accuracy for all the models.
F
Top 1, 10 and 20 prediction accuracy for all models
Table and show top 10 and 20 accuracy for new negation datasets. Table 5 depict top 1, 10 and accuracy for new role reversal dataset.
Results across model type and model size
Table 6 and show the change in the model ac- curacy for extended dataset as compared to the original dataset across model type and model size respectively. NEG-1500 represents the average of NEG-1500-SIMP-TEBS
and BEG-1500-SIMG GEN. We didn't t mention IB and 175B model sen- sitivity in Table 7 as they were 100% in original and extended datasets, therefore the difference is zero.

---

10
tool
flower
bird
tree
vegetable
vehicle
building
fish
Insect
Figure Top categories for NEG-136-SIM 136-SPPP
80
50
30
20
fish
profession animal
weapon
bird
sport
country
furniture.
state
city
tree
toy
color
vehicle
metal
flower-
fruit
crime
Figure Top 20 categories for NEG-1500-SIMP-TTI

---

80
70
60
40
30
20
10
animal
toy
plant
bird
tool
person
fruit
food
flower
building
vegetable
reptile
fish
mineral
device
vehicle
tree
drink
Figure 4: Top 20 categories for NEG-I5000SSIPPGEN
50
40
30
10
seen
hired
chosen
liked
killed
reported
attacked
interviewed
helped
belonged
robbed
identified
bitten
insulted
impressed
thanked
beaten-
feared
eaten
Figure 5: Top 20 categories for ROLB-1500

---

4
3
2
1
0
attacked
seen
killed
eaten
liked
hired
chosen
beaten
avoided
thanked
served
for
photogra progrrapeed
identifled
robbed
impressed
feared
slain
interviewed
haunted
Figure 6: Top 20 target words for ROLE-88
NEG-1500
NEG-1500 SIMP GEN(AI)
NEG- SIMP-TEMP
NEG-1500
NEG-136 SIMP(Aff)
NEG-136 Sensitivity
ROLE- 1500
ROLE-88
SIMP TEMP(A())
SIMP-GEN
Sensitivity
Sensitivity
BERT base
27.3
100.0
58.4
$5.3
53.5
35.9
BERT large
37.5
100.0
65.1
$3.8
53.5
RoBERTan Abaso
46.6
62.1
64.4
RoBEITatare MMarge
55.7
63.4
DistilBERT base
28.4
57.3
44.7
AIBERTV base
17.1
72.2
22.2
37.1
36.4
40.0
35.6
AIBERTvl II large
26.1
22.2
17.4
48.4
42.4
38.6
32.9
AIBERTV xtarge
34.1
55.6
19.5
22.3
26.0
77.0
62.5
AIBERT Laclarge
53.4
72.2
55.6
28.5
38.7
39.3
59.9
61.7
AIBERT Vasee
26.1
33.3
38.9
15.3
10.0
11.7
37.3
35,9
AIBERT Viuarge
29.5
83.3
22.2
18.8
36.6
36.2
31.7
33.1
AIBERT Vetange
37.5
94.4
27.8
21.1
40.4
47.8
52.7
51.1
AIBERT AIBERTyvestaa Vasshagge
50
100.0
44.4
29.0
42.9
45.3
55.7
61.5
T5.mall
9.1
44.4
0.0
5.8
33.6
25.3
16.8
19.6
T'5base
27.3
88.9
27.8
14.8
58.2
45.1
46.5
33.5
T5targe
36.4
94.4
50.0
18.2
60.4
49.8
44.0
0.0
T55xa
44.3
83.3
66.7
21.5
60.9
60.9
68.2
70.0
GPT2
0.0
0.0
66.7
11.2
48.3
37.7
60.4
56.4
GPT2 m'mdimm
0.0
0.0
50.0
16.4
61.4
46.9
53.1
60.7
GPT2ngge
0.0
0.0
38.9
16.8
61.7
51.7
$7.9
69,2
CPT2.1
0.0
0.0
44.4
18.8
63.6
52,8
59.0
71.9
GPT3
44.4
94.4
100.0
24,6
65.9
63.3
100.0
100.0
CPT3promps
38.6
72.2
100.0
24.4
55.9
55.2
100.0
100.0
Table Zero-shot top-5 word prediction accuracy and sensitivity (top-5 the whole vocabulary). ROLE-88 and
NEG-136 are from Lialin et al. (2022). ROLE-1500 and NEG-1500 are the new datasets. The best result for each task is in bold. "SIMP" stands for simple, "prompt" stands for prompting. The negation task evaluated in the affirmative form (Af'). Sensitivity is the percentage of sentence pairs for which the top-1 prediction changed.

---

NEG-1500-SIMP-1I
Top
Top
Top
BERT base
28.5
64.7
71.4
BERT
34.2
71.9
76.5
RoBERTabae Tabose
22.2
68.7
73.0
RoBERTaberg
30
69.2
74.4
DistilBERT base
28.6
67.8
75.6
AIBERTV base
43.9
49.5
AIBERTY EEAARTT'Iaass large
13.5
56.9
63.3
AIBERTy Lalarge
10.3
28.0
32.6
AIBERTy Lexiharg
48.6
$7.3
AIBERT Vusse
2.0
18.8
29.2
AIBERTY Vtarge
46.8
55.6
AIBERT 2elarge
24.7
45.2
50.0
AIBERTy2 2exhaage
53.4
62.8
T5mmlll
43.6
51.6
TSbase
21.4
65.6
71.3
Thaage
22.7
68.3
73.8
T5
39.7
66.2
69.9
GTT2
0,0
S7.1
63.6
meedium
0.0
69.5
73.2
CTT2iagge
0,0
70.0
75.7
GPT2
0,0
70.1
75.6
CPT3
27.5
N/A
N/A
21.8
N/A
N/A
Human claze completion
58.5
Table 3: Zero-shot top-1,10 and 20 word prediction accuracy NEG-1500 500--00M--TEMEE Top-5 is selected over the whole model vocabulary. The best result on each task is highlighted in bold. SIMP stands for simple, TEMP stands for template. Negation tasks are evaluated in the affirmative form (aff). GPT3 only produces top 5 predictions. Human cloze completion accuracy is the average accuracy of two annotators.
NEG-1500-SIMP-GRB
Top
Top 10
Top 20
BERT base
66.0
73.3
Barge
22.8
64.3
72.0
RoBERTupae
15.4
$1.6
60.1
RoBERTaaars Fajarge
24.4
61.6
71.7
DistilBEITT base
22.5
61.9
70.5
AIBERT 11
11.2
46.0
53.6
AIBERTV V Liarge
12.9
50.5
59.6
AIBERTy V sebsrge
7.2
34.4
41.6
AIBERT V sxclarge
12.4
50.7
60.7
AIBERTY V Vhnse
2.5
22.2
32.3
AIBERTVA VVR Vaarge
10.7
44.8
54.7
AIBERTv2.cm 22.large
24.0
56.5
64.5
AIBERTVA aexlarge
15.3
55.3
54.1
T5small
2.9
36.3
45.6
T5kmm
15.1
65.5
65.3
T5tagge
15.1
60.5
68.3
T5x1
39,7
66.2
69.9
GP
16.4
48.0
60.0
GPT2 GPT2medium
26.1
56.1
66.9
GPT2a
29.3
60.7
69.6
GPT221
30.4
60.1
70.4
GPT3
26.5
N/A
N/A
GPT3ppompp
24.3
N/A
N/A
Human cloze completion
45.5
Table Zero-shot 10 and 20 word prediction accuracy 1EGG-1500SSIMP-CEE Top-5 is selected the whole model vocabulary. The best result on each task is highlighted in bold. SIMP stands for simple, negation tasks evaluated the affirmative form. GPT3 allow only till top 5 predictions.

---

ROLE-1500
Top
Top 10
Top 20
BERT have
28.1
38.9
BERT large
9.1
30.9
40.7
RoBER Tabaae Tabase
33.9
47.3
RoBERT Fagare
38.6
52
DistilBEITI base
6.5
27.3
36,9
AIBERT base
2.8
16.2
25.3
AIBERT Ltarge
4.8
24.2
32,7
AIBERTY v Leharge
7.2
27.3
35.5
AIBERT Lexhhrge
12.2
37.2
45.1
AIBERTY SABERTY2nee Vhnse
6,2
22.7
29.9
AIBERTVE Viaagg
5,9
27.0
36.1
AIBERTY LIBERTY2earrg e2llagge
8.8
29.6
37.9
AIBERTV2
11.6
37.1
45.1
T5.mall
1.3
12.1
18.3
Taaase
4.7
21.9
28.5
Totage
6.2
25.3
31.3
T5x1
7,9
29.0
36.5
CPT2mme
0
18.9
27.1
GPT2
25.2
36.5
GPrre
27.0
37.7
GPT2
29.9
38.8
GPT3
7.7
N/A
N/A
Gproprrmps
5.7
N/A
N/A
Human cloze completion
10.0
Table 5: Zero-shot 1,1-1.10 and 20 word prediction accuracy ROLB-155 15500. Top-5 is selected over the whole model vocabulary. The best result on each task is highlighted in bold. GPT3 produces only top 5 predictions.
Model type
ROLE-1500
NEG-1500
NEG-1500 sentivity
Encoder-Only
-15.32
-39.52
13.07
Seq-to-Seq Decoder-Only 4.87
-14.20
-28.48
1.20
27.60
7.38
Table Change in the model accuracy for extended dataset as compared to the original dataset across model type. Negative sign shows drop the model performance whereas positive number shows the gain in model performance when dataset was extended.
Model type
ROLE-1500
NEG-1500
NEG-1500 sentivity
<60M
-10.04
-34.83
-13.83
300M
-14.60
-24.77
44.52
IB
13.3
175B
-17.00
-23.23
Table 7: Change in the model accuracy for extended dataset compared to the original dataset across model size. Negative sign shows drop model performance whereas positive number shows the gain model performance when dataset was extended.


Reviews:
[{"review_id": "UQTwYERK80", "reviewer": "Reviewer_yjSX", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**R1. “No error ranges are shown and no statistical tests are performed on findings about bakeoff winners”**\n\nWe have performed power analysis using McNemar's test on the accuracies of the top2 models for each dataset. Our analyses revealed the following:\n- We saw very low power for ROLE-88 (0.255) and NEG-136 (0.014)\n- For ROLE-1500, the power to differentiate between the top two accuracy models significantly improved to 0.721\n- For GPT3-generated NEG-1500-SIMP, the power reaches a full 1.000 in distinguishing top models\n- However, for the template-generated NEG-1500-SIMP, the power remained low: 0.043 for the top two models and 0.230 for the top 1 vs. top 3.\n\nFrom these results, it is evident that ROLE-1500 and NEG-1500 (GEN) are dramatically more reliable datasets. Specifically, they can distinguish between small effects of approximately 0.03 for ROLE and about 0.15 for NEG.  Note that 0.15 is the smallest difference we could find in the real data, with a simulated power for a 0.03 difference in ROLE being 0.933.\n\nWe also conducted a permutation test to check the statistical significance of the differences for extended datasets. The p-values for the top 2 models (referenced in Table 1) are as follows:\n\n- ROLE-1500: 0.0124\n- NEG-1500-SIMP-TEMP: 0.2206\n- NEG-1500-SIMP-GEN: 0.0002\n\nAs you can see, there's a statistically significant difference for the models on ROLE-1500 and NEG-1500-GEN at the 0.05 significance level. In contrast, the template-generated dataset does not distinguish between the top 2 models. Thanks again for your feedback – we will include the above results in the final version of the paper.\n\n**R1. “The rankings shown in Table 1 don't change much”**\n\nThe rankings indeed mostly (but not always) stay similar as we extend the dataset. But it's not just about rank, it's also about performance drop. For example, in the ROLE task, the top model doesn’t just move from first to second place. Its accuracy falls drastically from 55.7% to 26.1%. In other words, if we’re trying to answer a larger question of how well these models are able to do the ROLE task, the extended datasets show a significant change in model capability.\n\nWith these additional details in mind, we'd appreciate if you could revisit your score.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The paper corrects two severely underpowered datasets which were used to support conclusions that may not have been statistically sound.", "reasons_to_reject": "Related to soundness: No error ranges are shown and no statistical tests are performed on findings about bakeoff winners.  This would not be difficult: For example, the evaluation could use a per-item permutation test on the first and second place models of the original and extended datasets.  I'm not sure how widely these results are used in model selection, but if these rankings are important (and they are mentioned in the introduction), they should be tested for significance.  I imagine differences of a few percentage points given only a few dozen datapoints in the original experiments would have failed significance, had it been tried, and alerted the original experimenters to the insufficient power of the data.\n\nRelated to excitement: The rankings shown in Table 1 don't change much.  If I am reading this right, in all cases, the first place model as ranked by the extended dataset places no worse than second in the original ranking.  Again, if these rankings are important, the best models were not grossly misrepresented in the original experiments.", "questions_to_authors": "", "paper_topic_and_main_contributions": "Work described in this paper extends  two severely underpowered datasets which were used to support conclusions that may not have been statistically sound.  Results show rankings of language models evaluated on these extended datasets differ from rankings on the original datasets.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "6Nv7tBLGjv", "reviewer": "Reviewer_qAnt", "summary": "", "questions": "The reasons to reject section mentions rectifiable concerns that are essemtially questions about the reasoning behind the design of the experiment. ", "limitations": "", "rating": 5, "confidence": 5, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**R2. Include Statistical tests of the results like Card style power estimate or bootstraps resampling method.**\n\nThank you for your suggestion. We used the methodology proposed by Card et al. to evaluate statistical power. Specifically, we used McNemar's test to assess the top two models on each dataset.  For the ROLE dataset, our primary criterion was accuracy, while for the NEG dataset, we used the accuracy on affirmative examples (as in Ettinger et al., 2020). Our analyses revealed the following:\n- Very low power for ROLE-88 (0.255) NEG-136 (0.014)\n- For ROLE-1500, the power to differentiate between the top two accuracy models significantly improved to 0.721\n- For GPT3-generated NEG-1500-SIMP, the power reaches a full 1.000 in distinguishing top models\n- However, for the template-generated NEG-1500-SIMP, the power remained low: 0.043 for the top two models and 0.230 for the top three.\n\nThe outcomes, especially from points 3 and 4, underscore the potential and advantages of deploying large language models in data generation. Thanks again for your feedback – we will incorporate this discussion into the final version of our paper.\n\n**R2. Why do we have two different ways of expanding the datasets, one template-based, one using GPT3 as a generator?**\n\n- Diverse Expansion Techniques: We aimed to explore diverse approaches to dataset expansion. By employing both methods, we could gauge the potential strengths and limitations inherent to each method. While GPT-3 provided a dynamic way to generate diverse samples, the template-based method ensured consistency and alignment with the original dataset.\n- Historical Continuity: As you correctly pointed out, the template-based expansion aligned with the method utilized for the original NEG-136-SIMP dataset. By using this approach, we maintained a degree of historical continuity and comparability between the original and expanded datasets.\n- Comparative Insights: Although direct comparisons between the two methods were not presented in the paper, our internal investigations suggested that both methods had their merits. GPT-3 generated samples offered variety and unpredictability, whereas the template-based method provided controlled variability based on predefined structures.\n- Generalizability: Both methods produced similar outcomes, which strengthens our confidence in the results.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "This is important work, cleanly executed. It demonstrates the value of expanding the probe sets.", "reasons_to_reject": "I think this is a strong paper already, but it could be made stronger if the authors expand on some aspect that they have left implicit.\n\nIn the abstract, the authors mention that they have \"demonstrated\" that the previous results may have been skewerd: a claim whivch they repeat later in the paper. Yet they do not spell out the logic that leads them to this conclusion, nor do they say exactly what they mean by \"skewed\". Presumably they are relying on arguments about statistical power similar to those in the (cited) work by Card et al. In essence, they are saying, I believe, that one or more of the potential issues with underpowered experiments identified in Card et al has actually happened. This point needs to be made explicit in the present paper. What is the evdience, and is it compelling?\n\nIt would be appropriate to do a Card-style power estimate for the present experiments. If this is feasible, the authors should do it. If not, it may be sensible to do bootstrapped replications of the present studies, using probe datasets of various different sizes.\n\nThe authors describe two different ways of expanding the datasets, one template-based, one using GPT3 as a generator. It is not obvious why they did this, and there are no direct comparisons between the two expansion methods. Since it appears that for the purposes of the paper both methods produce approximately the same results. it would be helpful to explain why two methods were used and what was learned from the presence of both alternatives.", "questions_to_authors": "The reasons to reject section mentions rectifiable concerns that are essemtially questions about the reasoning behind the design of the experiment. ", "paper_topic_and_main_contributions": "This paper describes two approaches to expanding two datasets from Ettinger (2019). The paper increases the size of the datasets substantially, and provides evidence that previous studies with smaller datasets may have used too few probes, and that their results should therefore be treated with caution.", "excitement": 5, "reproducibility": 5, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Missing_References": "Not aware of any.", "Typos_Grammar_Style_and_Presentation_Improvements": "Nothing substantial. The sentence from 79-81 is ungrammatical: I thnk it needs to be reformulated  in some way. The intent is clear."}, "extra_text": {}}, {"review_id": "d2hcQXLrra", "reviewer": "Reviewer_Vfwk", "summary": "", "questions": "I'm not sure the numbers reported in the main text check out against the table, but this may well be on me: just to clarify, when, for example, at line 250 you mention DistillBERT and T5 having a lower performance by 5-15% each, which values from Table 1 are you comparing? (a side note on this: sometimes you simply mention a model name without specifying where it is large or base, please double check that every time you refer to a model, you do so unambiguously).\n\n-l104: which specific unlabelled dataset?\n-what did the curation of hypernyms in WordNet entail that made you decide against it? it'd be useful to know for other researchers (an appendix would do fine for this)\n-could you provide an example of multi-category names you did not use?\n-you choose performance at 5 for the main paper but also report it at 1, 10, and 20: it's unclear to me why you also run these evaluations. What would we learn if a model performs poorly at 1 but exceptionally at 20? is there a correlation between performance at different thresholds? is it just a robustness check?\n-l250 what is an absolute percentage increase?\n-what are the units of measurement in tables 6 and 7? percentages, percentage points, something else?\n\n---------------------------------\nI think it'd be good to clarify these points in the final version.", "limitations": "", "rating": 4, "confidence": 5, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**R3. “I'm not convinced the resources are valid and reliable next to being bigger”**\n\nThank you for your insightful comments. In response to your concerns, we employed the methodology proposed by Card et al. to evaluate statistical power. Specifically, we used  McNemar's test to assess the top two models on each dataset.  For the ROLE dataset, our primary criterion was accuracy, while for the NEG dataset, we used the accuracy on affirmative examples (as in Ettinger et al., 2020). Our analyses revealed the following:\n- Very low power for ROLE-88 (0.255) NEG-136 (0.014)\n- For ROLE-1500, the power to differentiate between the top two accuracy models significantly improved to 0.721\n- For GPT3-generated NEG-1500-SIMP, the power reaches a full 1.000 in distinguishing top models\n- However, for the template-generated NEG-1500-SIMP, the power remained low: 0.043 for the top two models and 0.230 for the top three.\n\nThe outcomes, especially from points 3 and 4, underscore the potential and advantages of deploying large language models in data generation. Thanks again for your feedback – we recognize the significance of this discussion and will incorporate it into the final version of our paper.\n\n**R3: “The novel datasets exacerbate an imbalance between the most represented and least represented categories, which may introduce biases in model evaluation”**\n\nWe agree with the need to ensure that the dataset's expansion does not inadvertently favor certain categories over others. However, it is essential to note that all of the models evaluated in our study are trained on extensive representative corpora. These corpora are designed to reflect a broad spectrum of language use, mitigating potential biases in terms of category representation. Consequently, the models' intrinsic ability to generalize and adapt to diverse language patterns implies that the likelihood of certain models having a pronounced advantage in predicting specific categories over others is minimal.\n\n**R3. What is human performance on these tasks?**\n\nWe recognize the importance of understanding human performance in evaluating models, especially when aiming to assess the extent to which models mirror human linguistic abilities.\n\nIn response to your concerns, we undertook a cloze test evaluation on the NEG datasets. We randomly selected 100 samples from each of the extended datasets. Each of these sentences was presented to an annotator, who was then asked to complete the sentences with a one-word response. This is a method analogous to the cloze test used traditionally to gauge human language comprehension. Upon comparison of these human responses to the target words, it was observed that human performance consistently exceeded model performance across datasets, which is what we would expect. Specifically:\n\n- For NEG-1500-SIMP-TEMP, human evaluation resulted in a top-1 accuracy of 56%, whereas the top-performing model, T5-xl, achieved about 40%.\n- In the case of NEG-1500-SIMP-GEN, the human evaluation yielded an accuracy of 44%, a significant 13.2% higher than the leading model, GPT2-xl.\n\nWe appreciate the additional external validation offered by N400 human performance. It's worth noting that the authors of the original datasets relied on data from established psycholinguistic studies rather than replicating these complex and resource-intensive experiments. Given that both cloze and N400 human results are available for the original data, it's reasonable to infer a level of generalizability to the extended datasets. The underlying linguistic phenomena and structures being probed remain consistent, thus lending credence to the notion that human performance metrics on the original data would, in all likelihood, be comparable if reproduced on the new data.\n\nWe believe our cloze test results offer some evidence for the external validity of our extended datasets in capturing non-expert linguistic behavior. We will conduct similar human evaluations for the ROLE datasets in the finalized version of our paper. We hope this allays your concerns about the external validation of our datasets.\n\n**R3. “A further problem concerns the reliability of the differences…”**\n\nFollowing our power analysis in response to your earlier comments, we’d like to highlight several key takeaways that we believe represent robust systematic effects:\n- **Performance Decline in Extended Datasets**: One of the major and consistent findings from our research was the decline in model performance on extended datasets. Across the board, model performance decreased between 20% to 57%. This isn't a marginal drop, and it is consistent across various datasets and metrics. This suggests that there is a systematic effect of dataset size on model performance, regardless of other factors.\n- **Rank Swapping among Top Models**: While the rank of models might swap, this isn't an arbitrary change. For instance, in the ROLE task using the ROLE-88 dataset, the RoBERTa-large led with a 5.7% advantage over the ALBERTv2-xxlarge. Yet, when evaluated on the more extensive ROLE-1500 dataset, ALBERTv2-xxlarge took the lead by 2.9%. This suggests that while some models might perform better on certain tasks and datasets, their relative performance can change when evaluated on more extensive and possibly more complex datasets. This observation was consistent, as also seen in the NEG-1500-SIMP-TEMP task.\n- **Model Architecture and Size Impact**: Our analysis indicates that specific architectures and sizes are more affected by dataset size. Encoder-only models and those smaller than 300M showed the most significant drop in performance when evaluated on larger datasets. This finding suggests a trend that could be explored further in future research and provides insight into which model types might be more susceptible to performance degradation on extended datasets.\n\n**R3. Question 1 : Clarify which model version is referenced in the discussion of results.**\n\nThank you for pointing out the potential ambiguity in our model references.  When we mention a model without specifying its size, we refer to an aggregate assessment of all sizes of that particular model used in our experiments. Specifically, we calculate the average percentage change for different sizes within that model family. For instance, our reference to T5's performance drop of 5-15% encompasses all the T5 model sizes we utilized: T5-base, T5-small, T5-large, and T5-xl. The complete list of these models can be found in Table 2. For clarity, the computed average drop across these various sizes is 14.2%, which falls within the mentioned range of 5-15%. We appreciate your feedback and will clarify the methodology behind these aggregations in the final version of the paper.\n\n**R3. Question 2 :**\n\n **Which labeled datasets?** - YouTube, SMS and Spouse\n\n **what did the curation of hypernyms in WordNet entail that made you decide against it?**  - The decision to not curate hypernyms in WordNet was influenced by the inconsistency of their hierarchy levels. A coherent pattern for selecting hypernyms using WordNet could not be established. For instance, in some cases, shifting the hypernym up by just one level would yield a proper sentence, while in others, a higher-level hypernym was needed for sentence correctness. We will provide additional details on this decisionin the paper's appendix in the final version.\n\n **Example of multicategory names we didn’t select.**:  Some of the examples of multiword categories we didn’t select are :  “Unit of time”, “Weather phenomenon”, “Type of dance”\n\n**Why choose a different threshold?**: By assessing performance at 1, 5, 10, and 20, we wanted to see if there's a clear trend in how models behave across different thresholds. We find that model performance increase as we set higher threshold. For example, in ROLE-1500, for threshold 1, BERT-base accuracy is 6.7% which increases to 28.1% and 38.9% for threshold value of 10 and 20 respectively This analysis helps us understand the stability and consistency of model performance.\n\n**what are the units of measurement in tables 6 and 7?**  percentage points\n\n**What is absolute percentage increase?**  It is the total percentage point increase.\n\nWith these additional details in mind, we'd appreciate if you could revisit your score.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "It provides resources that may offer better tools to probe LLMs along two specific linguistic tasks, negation and role reversal. In doing so, it analysis the statistical power the old and new datasets offer.", "reasons_to_reject": "I'm not convinced the resources are valid and reliable next to being bigger. Moreover, the abstract mentions lack of statistical power but no specific analysis is performed to determine the size of the datasets agains specific effects of interest, leaving the possibility that the new larger datasets also lack statistical power very much open. More concerning, the novel datasets exacerbate an imbalance between the most represented and least represented categories, which may introduce biases in model evaluation: a model that does well on the most frequent categories will have an edge, but that does not per se indicate the model is better unless one was to show that the most represented categories in the dataset also cover a larger share of sentences from some representative corpus of how people use language - I'm assuming here the goal is to probe LLMs for human-like language processing abilities. \n\nThis is another point that, to me at least, makes the paper less interesting and impactful than it could be: what is human performance on these tasks? We are using these probes to assess the extent to which these models mirror human language abilities, but except for the role of the researchers in prompting the LLMs, choosing the templates and the examples, and filtering out bad generated sentences, we have no idea whether these datasets actually capture human language abilities exhibited by non-experts. In other words, I think these datasets currently lack external validation.\n\nA further problem concerns the reliability of the differences. This is arguably a finer issue especially in the context of NLP research where the amount of comparisons is typically large and there is little interest in the reliability of the differences. Nonetheless, with 22 models, 3 datasets, 4 evaluation metrics (@1, 5, 10, 20), the sheer amount of numbers being compared is huge and some differences are bound to arise without them necessarily meaning the models are _truly_ different. So, given such a large number of comparisons, how can I trust that models coming out on top are _actually_ better than the others? and more importantly, what does this mean? There is a first attempt at relating performance with model type and size, but it is not very systematic (how about, for example, fitting a statistical model predicting performance as a function of model's features of interest and actually checking whether they play a role?).\n\nThe limitations section is not particularly insightful either: it touches upon relevant points but does not critically connect them with the issues they may pose for the datasets being introduced.\n\nIn conclusion, bigger is sometimes better unless it keeps repeating bad biases present in the data. If I derive a bad sample (not representative of the population, biased, ...) and then replicate it, I will have a bigger dataset (and higher statistical power) but I won't be closer to an answer to my question. Given the current version of the paper, I'm not convinced the larger datasets are qualitatively better than the small ones next to being quantitatively larger. Money is of course a concern, and having good large datasets that are relatively inexpensive to generate is a great thing, but we should not forget to validate them properly and carefully before. I think this step is missing and would make a difference in making these new datasets more trustworthy and usable.\n\nMost of the issues raised above have been thoroughly addressed by the authors, in my opinion strengthening the paper and the conclusions substantially. I still have some reservations about the category imbalance being a negligible factor (after all, you use a specific LLM to generate examples, and that model, even if trained on large representative corpora like the others, may have incorporated specific biases that differ from that of other models. I think i would be good to mention this in the limitations.", "questions_to_authors": "I'm not sure the numbers reported in the main text check out against the table, but this may well be on me: just to clarify, when, for example, at line 250 you mention DistillBERT and T5 having a lower performance by 5-15% each, which values from Table 1 are you comparing? (a side note on this: sometimes you simply mention a model name without specifying where it is large or base, please double check that every time you refer to a model, you do so unambiguously).\n\n-l104: which specific unlabelled dataset?\n-what did the curation of hypernyms in WordNet entail that made you decide against it? it'd be useful to know for other researchers (an appendix would do fine for this)\n-could you provide an example of multi-category names you did not use?\n-you choose performance at 5 for the main paper but also report it at 1, 10, and 20: it's unclear to me why you also run these evaluations. What would we learn if a model performs poorly at 1 but exceptionally at 20? is there a correlation between performance at different thresholds? is it just a robustness check?\n-l250 what is an absolute percentage increase?\n-what are the units of measurement in tables 6 and 7? percentages, percentage points, something else?\n\n---------------------------------\nI think it'd be good to clarify these points in the final version.", "paper_topic_and_main_contributions": "The paper introduces three new datasets to probe LLMs on negation and role reversal tasks, scaling up smaller datasets previously used in the literature. Two extended version of the negation dataset are introduced, one obtained by generating new sentence pairs using templates and one obtained by having GPT3 generating new pairs given instructions and example pairs. Moreover, one extended version of the role reversal dataset was created by having GPT3 generate novel sentence pairs again given instructions and examples.\n\n22 LLMs of different size and type (encoder only, sequence to sequence, decoder only) were evaluated against the original small-scale datasets and against the new larger datasets, reporting a generalised decrease in performance, but also - in some cases - a higher sensitivity to negation, measured as the proportion of sentences where the model prediction changes between the affirmative and negative sentence in a pair.\n\nThe goal of contributing the field with larger resources to probe LLMs is certainly meritorious, and the comparison involves a vast amount of models. After the rebuttal, I think the paper is ripe enough for acceptance.", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Missing_References": "-"}, "extra_text": {"Typos_Grammar_Style_and_Presentation_Improvements": "The clarity of the manuscript can be vastly improved:\n- please get rid of commas between subject and main verb (e.g., line 039)\n- avoid contractions\n- lines 207 and 208: the is duplicated, and or should read of\n- you often talk about differences between performance as percentages but judging from the tables it seems you talk about percentage points (to clarify, if model A scores 40% and model B scores 20%, model B is worse by 50% (performance is half) or 20 percentage points)\n- Table 1 is very large and combines different tasks and metrics: I think it can be organised better (by task, rather than by size, for example) or, if possible given the page limit, split in three (I'm thinking ROLE, NEG, sensitivity - but there may be better ideas than this)\n- a general copy-editing of the prose would be warranted, some sentence convey the message but do not read very nicely and could be improved."}}]

Author rebuttal:

Meta-review: The work presents two valuable datasets on negation and role reversal and evaluate the performance of 22 models, showing that previous findings might have been skewed due to smaller test sets. 

The three reviewers agree on their soundness and excitement scores, all on the upper bound, from good to strong/transformative, and the vast majority of comments and concerns are addressed on the rebuttal. These should be included in the final version of the manuscript.
Decision: Accept-Main