COPEN : Probing Conceptual Knowledge in Pre-trained Language Models (2022)

Abstract: Conceptual knowledge is fundamental to human cognition and knowledge bases. However, existing knowledge probing works only focus on evaluating factual knowledge of pre-trained language models (PLMs) and ignore conceptual knowledge. Since conceptual knowledge often appears as implicit commonsense behind texts, designing probes for conceptual knowledge is hard. Inspired by knowledge representation schemata, we comprehensively evaluate conceptual knowledge of PLMs by designing three tasks to probe whether PLMs organize entities by conceptual similarities, learn conceptual properties, and conceptualize entities in contexts, respectively. For the tasks, we collect and annotate 24k data instances covering 393 concepts, which is COPEN, a COnceptual knowledge Probing bENchmark. Extensive experiments on different sizes and types of PLMs show that existing PLMs systematically lack conceptual knowledge and suffer from various spurious correlations. We believe this is a critical bottleneck for realizing human-like cognition in PLMs. COPEN and our codes are publicly released at https://github.com/THU-KEG/COPEN.

## introduction pre-trained language models (plms) have achieved superior performance on most nlp tasks requiring substantial world knowledge @xcite . it is interesting and meaningful to probe the extent and scope of world knowledge within plms. existing knowledge probing works have evaluated plms' knowledge about entities @xcite and their relations @xcite @xcite , i.e., factual knowledge, but ignore conceptual knowledge. conceptual knowledge, especially the abstraction ability, is fundamental to all kinds of human cognition @xcite including language processing @xcite . just as the quote of psychologist gregory murphy, concepts are the glue that holds our mental world together @xcite . moreover, knowledge bases @xcite @xcite organize massive entities via concept taxonomies as illustrated in figure 1 , which enable broad applications @xcite . therefore, probing whether plms have human-like conceptual knowledge is necessary in knowledge probing. inspired by the conceptual schema in knowledge representations @xcite @xcite , we comprehensively evaluate the conceptual knowledge of plms by asking three questions: do plms organize entities by conceptual similarities? do plms know the properties of concepts? can plms correctly conceptualize entities in contexts? in this paper, we design three probing tasks for these questions: (1) the conceptual similarity judgment (csj) task studies whether plms organize entities by conceptual similarities, which is the basis of understanding concepts. given a query entity, csj requires plms to choose the most conceptually similar entity among candidate entities. for example, in figure 1 , given dolly as the query entity, although uk has a direct relation and more co-occurrences with it, plms should choose grumpy cat. (2) the conceptual property judgment (cpj) task probes whether plms have the knowledge of conceptual properties, which are the generic abstractions of factual knowledge. given a statement about a specific property, such as "have feathers", cpj requires plms to judge whether it is true for a specific concept and also a concept chain, which evaluates whether plms understand the property transitivity among a chain of hierarchical concepts. (3) the conceptualization in contexts (cic) task evaluates the abilities of plms to correctly conceptualize entities within contexts. given an entity mentioned in a specific context, plms are required to choose the most appropriate concept in a concept taxonomy according to its context. cic requires not only disambiguating entity mentions, but also distinguishing superordinate and subordinate concepts. for instance, given the context "dolly is running on the grassland", plms should conceptualize dolly as an animal since there is no enough evidence for mammal. based on the above considerations, we construct a conceptual knowledge probing benchmark, copen, which contains a concept taxonomy with 446 concepts and high-quality data of 24k instances for the three probing tasks. the concept taxonomy is curated by experts based on dbpedia @xcite and wikidata @xcite to form a well-defined hierarchy and cover broad entities. the data instances for three tasks are collected by aligning entities in wikidata and sentences in genericskb @xcite , wikipedia foot_0 , and simple wikipedia foot_1 into the concept taxonomy and then manually annotated by crowd-sourcing annotators. we conduct extensive experiments on copen to evaluate various widely-used language mod-els (lms), which include three types: masked lms @xcite , autoregressive lms @xcite , and sequence-to-sequence lms @xcite . we conduct the experiments in three settings: (1) zero-shot probing, which reformulates the probing tasks into pretraining objectives and lets plms score answers without any training @xcite ; (2) linear probing, which only tunes additional linear classification heads and uses them to handle probing tasks with the frozen representations produced by plms; to summarize, our contributions are three-fold: (1) we propose to probe plms for conceptual knowledge, which has long been ignored, and design three probing tasks inspired by the knowledge representation works. (2) we construct copen, a probing benchmark containing high-quality concept taxonomy and probes. (3) we empirically show that existing plms systematically lack conceptual knowledge and analyze the reasons. we hope our benchmark and findings could facilitate further research on concept-aware plms and human-like language understandings. ## copen benchmark in this session, we introduce our copen benchmark, including the construction of the concept taxonomy ( § 2.1) and the datasets for three probing tasks ( § § 2.2 to 2.4). more construction and annotation details are shown in appendix d. ## evaluation setup we introduce the various widely-used plms investigated in our experiments ( § 3.1) and the three adopted probing methods ( § 3.2). ## experiment and analysis we first introduce the overall results in § 4.1 and conduct detailed analyses on the three probing tasks ( § § 4.2 to 4.4), respectively. we then analyze the performance at different model scales ( § 4.5). more observations and discussions on experimental results are placed in appendix b. ## conclusion and future work in this paper, we systematically analyze the conceptual knowledge in existing plms by constructing a high-quality conceptual knowledge probing benchmark (copen). extensive experiments show that existing plms have a certain conceptual knowledge, but are significantly worse than humans, even with billions of parameters. we further find that plms fail in distinguishing fine-grained concepts and understanding concept hierarchy, and suffer from conceptual hallucination caused by word occurrence and out-of-context bias. in the future, inspired by works infusing factual knowledge, we will try to develop conceptual knowledgeable plms by exploring concept-aware pre-training objectives and knowledge-enhanced architectures.