SlowGuess/NAACL_2022_OCR huggingface dataset:
We investigate the reasons behind the models' low performance on RNPC, specifically whether their failure is due to the lack of in-domain training data or an intrinsic deficiency in their architecture. Namely, we attempt to answer question (b): Is the target knowledge learnable with appropriate data?
We adopt the challenge set analysis technique from Liu et al. (2019a), which exposes a model to a small amount of challenge data and assesses how well it can adapt. Specifically, we split each RNPC task dataset into a training set of 200 examples and a new test set containing the rest, ensuring that they have different modifiers in the same position. For example, if a modifier appears as the  $M_{1}$  of an NP in the training set, it cannot appear in the same position of any NP in the test set. Then, we finetune
Figure 3: Learning curves of the best models on each RNPC task with an increasing number of finetuning examples.
each model from Figure 2 on an increasing number of examples (10 to 200). The learning curves of the best-performing models (RoBERTa-large (MNLI), RoBERTa-base (MPE), and RoBERTA-large (ADEPT)) are plotted in Figure 3.12
On SPTE, the accuracy rapidly climbs from 61.1 to 75.8 with only 10 examples, and saturates around 92 with 100 examples, approaching human performance (94.1). The learning curve on MPTE has more fluctuations, with a peak at 71.1 (150 examples) and a final score of 67.8. On EPC, starting around chance (39.5), the accuracy progressively increases up to 64.4 with 200 examples. These results indicate that the target knowledge is learnable with appropriate training data. Furthermore, SPTE may be the easiest task, since it only requires local knowledge about the meaning of the modifiers and the noun. By contrast, MPTE and EPC involve world knowledge (e.g., basketball players are generally tall among the population), as well as global reasoning between components in a sentence (e.g., the relationship between the event and the modifiers), which may explain the remaining large gap between model and human performance ( $>90$ ).
7 What can LMs learn from RNPC?
Given that the target knowledge is learnable, we now address question (c): What linguistic features have the models learned from RNPC? We probe for two features extensively studied in the relevant literature (cf. §2), using different techniques.
Modifier semantic category. We first investigate if models have learned the semantic category of modifiers using the "edge probing technique" (Tenney et al., 2019). Namely, each modifier is categorized as intersective, subsective, or privative (McCrae et al., 2014). The entailment pattern of
Figure 4: Probing accuracy for the "modifier semantic category" feature, before (left) and after (right) finetuning on each RNPC task.
individual modifiers is an important factor in determining the meaning of the entire NP.
Given a finetuned model, we take the contextualized representation of each modifier in the last hidden layer. Then, we attach a linear head on top of the token representation as an “auxiliary classifier”. We choose linear classifiers because more expressive ones like Multi-Layer Perceptron are more likely to capture the target feature themselves (Hewitt and Liang, 2019). The token representations are then frozen, while the linear head is trained to predict the semantic category of the modifiers.[13]
We probe the models finetuned on RNPC from Section 6, as well as the models finetuned on existing benchmarks for comparison. The results are shown in Figure 4. For all tasks, the probing accuracy is higher for models finetuned on RNPC than on existing benchmarks. The increase is small for SPTE (3.4) and MPTE (2.8), but more obvious for EPC (7.1). This is somewhat counter-intuitive since modifier category is defined in terms of entailment patterns, but models learn it better from EPC than from TE tasks. Nonetheless, the overall trend shows that models can learn the semantic category of modifiers to some extent after being finetuned on our datasets. Since the absolute increase is limited, we plan to explore ways to quantify the actual amount of learned knowledge in future work.
Modifier scope. We also probe for the scope of the first modifier  $(\mathbf{M}_1)$  in recursive NPs (Det  $\mathbf{M}_1$ $\mathbf{M}_2$  N). Specifically, we focus on privative  $\mathbf{M}_1$  's, since they can have different scopes when interacting with different  $\mathbf{M}_2$  's and  $\mathbf{N}$  's. For instance, in the NP a former American diplomat, former negates diplomat (N), but the person is still American; while in a former beginner drummer, it negates beginner  $(\mathbf{M}_2)$ , but the person may still be
12See Appendix E.2 for model and hyperparameter details.
<sup>13</sup>See Appendix E.3 for an illustration of the method.
Figure 5: A case study of modifier scope. Each subfigure shows the frequency distribution of the attention ratio  $r$  ( $0 < r < 1$ ) for an  $\mathbf{M}_1$ , divided into two sides at 0.5. The  $\mathbf{M}_2$  side contains NPs where  $\mathbf{M}_1$  attends more to  $\mathbf{M}_2$  than to  $\mathbf{N}$ ; vice versa for the  $\mathbf{N}$  side.
a drummer. $^{14}$  This difference cannot be captured by the semantic category of former.
As a proxy for the scope of  $\mathbf{M}_1$ , we use attention visualization, a widely adopted technique to study token correlations (Vig, 2019). We choose BERT-base finetuned on 200 MPTE examples from Section 6 as the model to be probed for a case study.
Let us denote any token in a given NP as  $\mathbf{x}$ . We define  $\mathbf{A}_{\mathbf{x}}$ , the average of the weights of all attention heads from  $\mathbf{M_1}$  to  $\mathbf{x}$  in the final layer, representing how much  $\mathbf{M_1}$  attends to token  $\mathbf{x}$ . We then calculate the ratio  $r = \mathbf{A_N} / (\mathbf{A_N} + \mathbf{A_{M_2}})$ $(0 < r < 1)$ . If  $r < 0.5$ , then  $\mathbf{M_1}$  attends more to  $\mathbf{M_2}$ ; else,  $\mathbf{M_1}$  attends more to  $\mathbf{N}$ . For each primitive modifier, we take all NPs containing it in the  $M_1$  position in our dataset and plot the distribution of  $r$ . Figure 5 shows three examples (alleged, counterfeit, or fraudulent) representing different patterns.
As shown in the first sub-figure, alleged attends more to either  $\mathbf{M_2}$  and  $\mathbf{N}$  depending on the NP. For example, it attends more to  $\mathbf{M_2}$  in an alleged antique bowl (0.454), since the NP describes a bowl
that may not be antique. Inversely, an alleged male criminal is on the N side (0.517), since they are most likely male but may not be a criminal.
The second sub-figure indicates that counterfeit mainly attends to  $\mathbf{M}_2$ . For instance, a counterfeit Hollywood movie (0.382) is still a movie, but is probably not made in Hollywood. This is similar to the cases of luxury bag, medical drugs, foreign cigarettes, etc. On the contrary, fraudulent mainly attends to  $\mathbf{N}$ , as shown in the third sub-figure. The fraudulent medical claims (0.559) are not valid claims but still on medical grounds. The same holds for electoral victory, medical excuse, etc.
Additionally, we notice that there are some boundary cases close to the  $\mathbf{r} = \mathbf{0.5}$  division line, like ruthless criminal and former thief in the alleged sub-figure. A plausible explanation is that  $\mathbf{M_1}$  is questioning both  $\mathbf{M_2}$  and  $\mathbf{N}$  in these cases (e.g., an alleged ruthless criminal is not necessarily ruthless or a criminal). Overall, the above results indicate that models finetuned on our tasks can capture modifier scope in recursive NPs.
8 Is RNPC useful for downstream tasks?
We finally address question (d): How can such knowledge benefit downstream tasks? We choose the task of Harm Detection (Banko et al., 2020) for extrinsic evaluation. Concretely, we consider the scenario where a user interacts with a task-oriented agent like Siri or Alexa, and the agent needs to determine whether the involved activity in the user query is potentially harmful. The definition of "harm" can be user-dependent. Here, we consider an activity to be harmful if it may cause pain, physical injury, or be illegal for minors. We choose this task because many false positives come from recursive NPs. For example, how to make a homemade bomb is obviously harmful while how to make a homemade bath bomb is harmless.
We collect a small test set from wikiHow, a website of how-to articles. Each article title is considered a query (e.g., how to make a cake). Then, we compile a list of 74 keywords about harmful entities (e.g., bomb, fire, drugs), only 12 of which occur in RNPC. We then select wikiHow queries containing at least an NP with one of the 74 keywords as the head noun, and sample a small subset for manual annotation. Each query is labeled as harmful or harmless, depending on whether it involves a harmful activity as defined above. After data cleaning and re-balancing, we obtain 170 queries, with a 1:1 positive/negative ratio.
14 Admittedly, there can be alternative interpretations: say, one can also imagine that a former beginner drummer describes a person who is no longer a drummer at all. However, in that case, it is enough to say a former drummer instead, considering the Gricean maxim of quantity. Therefore, here we still focus on the first interpretation, which is more straightforward.
15There have been recent debates on the faithfulness of this method (Jain and Wallace, 2019;Wiegrefe and Pinter, 2019). Therefore, we do not use attention weights to make claims about how our models work, but only what they capture, with attention weights.
Model	Acc.	P	R	F1
Always harmful	50.0	50.0	100.0	66.7
GPT3-ada	49.4	49.7	98.8	66.1
GPT3-curie	59.4	60.5	54.1	57.1
GPT3-davinci	51.3	50.6	100.0	67.2
RoBERTa-large (SPTE) (ours)	58.2	54.5	100.0	70.5
RoBERTa-large (EPC) (ours)	72.9	66.4	92.9	77.5
Table 5: Zero-shot performance of models trained on RNPC on the Harm Detection task. Baselines include a model that always predicts harmful and GPT3.
We design two zero-shot harm classifiers using models finetuned on our entire SPTE and EPC dataset. They share a few pre-processing steps: first, all NPs are extracted from the input query; then, NPs containing a keyword from our list in the head noun position are retained. For each retained NP (e.g., a water gun), we check if it is indeed a harmful entity using either the SPTE or the EPC model. The input to the SPTE model is a premise of the form "This is {NP}" (e.g., This is a water gun) and a hypothesis of the form "This is (a/an) {N}" (e.g., This is a gun). If the output label is entailment, we classify the query as harmful, otherwise harmless. Likewise, using the EPC model, we form two events given the retained NP: "(A/An) {N} is harmful" and " $\{\mathrm{NP}\}$  is harmful". If the second event is predicted as more or equally plausible compared to the first, the query is considered harmful.
We compare our two classifiers to a simple baseline that always predicts harmful as well as to three GPT3 models.16 Both classifiers meaningfully exceed the simple baseline, and the EPC-based classifier outperforms all the other methods by  $10+$  in terms of accuracy and  $F_{1}$ . This shows that the understanding of recursive NPs is beneficial for downstream tasks without any training data. To understand why EPC is more suitable than SPTE for this task, we further examine the errors they make. One major error type concerns polysemous keywords such as shot. For instance, the SPTE model mistakenly predicts how to have a good basketball shot to be harmful because a good basketball shot is still a shot (shot can mean both "shooting a gun" and "shooting a ball"). There are also some queries out of the scope of the EPC model, e.g., how to make a sake bomb. Since sake bomb is a cocktail, the gold label is harmful as our target users are minors. The EPC model correctly predicts that a sake bomb is less harmful than a bomb, but fails to capture that it may still be
harmful (for minors).
We introduce RNPC, a challenge set targeting the understanding of recursive NPs, a fundamental aspect of human common sense. Pretrained LMs with SOTA performance on Natural Language Understanding benchmarks have poor mastery of this knowledge, but can still learn it when exposed to small amounts of data from RNPC. Using different probing techniques, we show that models can learn relevant linguistic features, including modifier category and scope, from RNPC. They also achieve strong zero-shot performance on an extrinsic Harm Detection task, indicating the transferability of this knowledge. For future work, we hope to investigate other linguistic phenomena as a step towards comprehensively characterizing LMs' limitations and capabilities in language understanding.
This research is based upon work supported in part by the DARPA KAIROS Program (contract FA8750-19-2-1004), the DARPA LwLL Program (contract FA8750-19-2-0201), and the IARPA BETTER Program (contract 2019-19051600004). Approved for Public Release, Distribution Unlimited. The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies, either expressed or implied, of DARPA, IARPA, or the U.S. Government.
Special thanks go to our annotators, students in CIS 421/521 and MCIT 521 at the University of Pennsylvania. We also thank Artemis Panagopoulou for providing the extrinsic evaluation data. Meanwhile, we appreciate the support from OpenAI on finetuning GPT-3. Finally, we thank Haochen Zhang, Pengyuan Lu, Daniel Deutsch, Daphne Ippolito, Lara Martin, YoungMin Cho, Yi Zhang, Helen Jin, Siyi Liu, Eleni Miltsakaki, Jordan Kodner, Mingming Liu, Peng Zhou, Christopher Cieri, James J. Fiumara, Ellie Pavlick, Charles Yang, Yejin Choi, Alexander Koller, Chris Potts, and Mitch Marcus for their valuable feedback.
Nabil Abdullah and Richard A. Frost. 2005. Adjectives: A uniform semantic approach. In Advances in
16Used in a zero-shot setting; see Appendix E.4 for details.
Artificial Intelligence, pages 330-341, Berlin, Heidelberg. Springer Berlin Heidelberg.
Marianna Apidianaki and Aina Gari Soler. 2021. ALL dolphins are intelligent and SOME are friendly: Probing BERT for nouns' semantic properties and their prototypicality. In Proceedings of the Fourth BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP, pages 79-94, Punta Cana, Dominican Republic. Association for Computational Linguistics.
Michele Banko, Brendon MacKeen, and Laurie Ray. 2020. A unified taxonomy of harmful content. In Proceedings of the Fourth Workshop on Online Abuse and Harms, pages 125-137, Online. Association for Computational Linguistics.
Marco Baroni and Roberto Zamparelli. 2010. Nouns are vectors, adjectives are matrices: Representing adjective-noun constructions in semantic space. In Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing, pages 1183-1193, Cambridge, MA. Association for Computational Linguistics.
Yonatan Belinkov, Sebastian Gehrmann, and Ellie Pavlick. 2020. Interpretability and analysis in neural NLP. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: Tutorial Abstracts, pages 1-5, Online. Association for Computational Linguistics.
Gemma Boleda, Marco Baroni, The Nghia Pham, and Louise McNally. 2013. Intensionality was only alleged: On adjective-noun composition in distributional semantics. In Proceedings of the 10th International Conference on Computational Semantics (IWCS 2013) - Long Papers, pages 35-46, Potsdam, Germany. Association for Computational Linguistics.
Gemma Boleda, Eva Maria Vecchi, Miquel Cornudella, and Louise McNally. 2012. First order vs. higher order modification in distributional semantics. In Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, pages 1223-1233, Jeju Island, Korea. Association for Computational Linguistics.
Pierrette Bouillon and Evelyne Viegas. 1999. The description of adjectives for natural language processing: Theoretical and applied perspectives. In Proceedings of Description des Adjectifs pour les Traitements Informatiques. Traitément Automatique des Langues Naturelles, pages 20-30.
Samuel R. Bowman, Gabor Angeli, Christopher Potts, and Christopher D. Manning. 2015. A large annotated corpus for learning natural language inference. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 632-642, Lisbon, Portugal. Association for Computational Linguistics.
Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020. Language models are few-shot learners. In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual.
Richard Campbell. 2002. Computation of modifier scope in NP by a language-neutral method. In COLING 2002: The 19th International Conference on Computational Linguistics.
Gennaro Chierchia and Sally McConnell-Ginet. 2000. Meaning and grammar: An introduction to semantics. MIT press.
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171-4186, Minneapolis, Minnesota. Association for Computational Linguistics.
Ali Emami, Ian Porada, Alexandra Olteanu, Kaheer Suleman, Adam Trischler, and Jackie Chi Kit Cheung. 2021. ADEPT: An adjective-dependent plausibility task. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 7117-7128, Online. Association for Computational Linguistics.
Henry Hamburger and Stephen Crain. 1984. Acquisition of cognitive compiling. Cognition, 17(2):85-136.
Iris Hendrickx, Zornitsa Kozareva, Preslav Nakov, Diarmuid O Seaghdha, Stan Szpakowicz, and Tony Veale. 2013. SemEval-2013 task 4: Free paraphrases of noun compounds. In Second Joint Conference on Lexical and Computational Semantics (*SEM), Volume 2: Proceedings of the Seventh International Workshop on Semantic Evaluation (SemEval 2013), pages 138-143, Atlanta, Georgia, USA. Association for Computational Linguistics.
John Hewitt and Percy Liang. 2019. Designing and interpreting probes with control tasks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 2733-2743, Hong
Kong, China. Association for Computational Linguistics.
Sarthak Jain and Byron C. Wallace. 2019. Attention is not Explanation. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 3543-3556, Minneapolis, Minnesota. Association for Computational Linguistics.
Hans Kamp and Barbara Partee. 1995. Prototype theory and compositionality. Cognition, 57(2):129-191.
Ronald Kemker, Marc McClure, Angelina Abitino, Tyler L. Hayes, and Christopher Kanan. 2018. Measuring catastrophic forgetting in neural networks. In Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, (AAAI-18), the 30th innovative Applications of Artificial Intelligence (IAAI-18), and the 8th AAAI Symposium on Educational Advances in Artificial Intelligence (EAAI-18), New Orleans, Louisiana, USA, February 2-7, 2018, pages 3390-3398. AAAI Press.
Alice Lai, Yonatan Bisk, and Julia Hockenmaier. 2017. Natural language inference from multiple premises. In Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 100-109, Taipei, Taiwan. Asian Federation of Natural Language Processing.
Matthias Lalisse. 2015. Distinguishing Intersective and Non-Intersective Adjectives in Compositional Distributional Semantics. Master's thesis, University of Oxford.
Mark Lauer. 1995. Corpus statistics meet the noun compound: Some empirical results. In 33rd Annual Meeting of the Association for Computational Linguistics, pages 47-54, Cambridge, Massachusetts, USA. Association for Computational Linguistics.
Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. 2020. BART: Denoising sequence-to-sequence pretraining for natural language generation, translation, and comprehension. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7871-7880, Online. Association for Computational Linguistics.
Nelson F. Liu, Roy Schwartz, and Noah A. Smith. 2019a. Inoculation by fine-tuning: A method for analyzing challenge datasets. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 2171-2179, Minneapolis, Minnesota. Association for Computational Linguistics.
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019b. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692.
Mercedes Marcilese, LMS Correa, and Marina RA Augusto. 2013. Recursive pre-nominal modifiers interpretation in language acquisition. Advances in Language Acquisition, pages 138-146.
Mitchell P. Marcus, Beatrice Santorini, and Mary Ann Marcinkiewicz. 1993. Building a large annotated corpus of English: The Penn Treebank. Computational Linguistics, 19(2):313-330.
Edward H Matthei. 1982. The acquisition of prenominal modifier sequences. Cognition, 11(3):301-332.
John P. McCrae, Francesca Quattro, Christina Unger, and Philipp Cimiano. 2014. Modelling the semantics of adjectives in the ontology-lexicon interface. In Proceedings of the 4th Workshop on Cognitive Aspects of the Lexicon (CogALex), pages 198-209, Dublin, Ireland. Association for Computational Linguistics and Dublin City University.
Marcin Morzycki. 2016. Modification. Cambridge University Press.
James Mullenbach, Jonathan Gordon, Nanyun Peng, and Jonathan May. 2019. Do nuclear submarines have nuclear captains? a challenge dataset for commonsense reasoning over adjectives and objects. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 6052-6058, Hong Kong, China. Association for Computational Linguistics.
Preslav Nakov and Marti Hearst. 2005. Search engine statistics beyond the n-gram: Application to noun compound bracketing. In Proceedings of the Ninth Conference on Computational Natural Language Learning (CoNLL-2005), pages 17-24, Ann Arbor, Michigan. Association for Computational Linguistics.
Courtney Naples, Matthew Gormley, and Benjamin Van Durme. 2012. Annotated Gigaword. In Proceedings of the Joint Workshop on Automatic Knowledge Base Construction and Web-scale Knowledge Extraction (AKBC-WEKEX), pages 95–100, Montreal, Canada. Association for Computational Linguistics.
Neha Nayak, Mark Kowarsky, Gabor Angeli, and Christopher D Manning. 2014. A dictionary of non-subjective adjectives. Technical report.
Barbara H Partee. 2010. Privative adjectives: subsec- tive plus coercion. Presuppositions and discourse: Essays offered to Hans Kamp, 21:273-285.