COMPS : Conceptual Minimal Pair Sentences for testing Robust Property Knowledge and its Inheritance in Pre-trained Language Models (2023)

Abstract: A characteristic feature of human semantic cognition is its ability to not only store and retrieve the properties of concepts observed through experience, but to also facilitate the inheritance of properties (can breathe) from superordinate concepts (animal) to their subordinates (dog)—i.e. demonstrate property inheritance. In this paper, we present COMPS, a collection of minimal pair sentences that jointly tests pre-trained language models (PLMs) on their ability to attribute properties to concepts and their ability to demonstrate property inheritance behavior. Analyses of 22 different PLMs on COMPS reveal that they can easily distinguish between concepts on the basis of a property when they are trivially different, but find it relatively difficult when concepts are related on the basis of nuanced knowledge representations. Furthermore, we find that PLMs can show behaviors suggesting successful property inheritance in simple contexts, but fail in the presence of distracting information, which decreases the performance of many models sometimes even below chance. This lack of robustness in demonstrating simple reasoning raises important questions about PLMs’ capacity to make correct inferences even when they appear to possess the prerequisite knowledge.

## introduction the ability to learn, update and deploy one's knowledge about concepts (robin, chair) and their properties (can fly, can be sat on), observed during everyday experience is fundamental to human semantic cognition @xcite @xcite . knowledge of a concept's properties, combined with the ability to infer the isa relation @xcite leads to an important behavior known as property inheritance @xcite @xcite , where subordinates of a concept inherit its properties. for instance, one is likely to infer that an entity called luna can meow, has a tail, is a mammal, etc., even if all they know is that it is a cat. the close connection between a word's meaning and its conceptual representation makes these abilities crucial to language understanding @xcite , making it critical for computational models of language processing to also exhibit behavior consistent with these capacities. indeed, modern pre-trained language models (plms; @xcite have made impressive empirical strides in eliciting general knowledge about real world concepts and entities @xcite , as well as in demonstrating isomorphism with real world abstractions like direction and color @xcite , often times without even having been explicitly trained to do so. at the same time, their ability to robustly demonstrate such capacities has recently been called to question, owing to failures due to reporting bias @xcite , lack of consistency @xcite , and sensitivity to lexical cues @xcite @xcite . in this work, we cast further light on plms' ability to robustly demonstrate knowledge about concepts and their properties. to this end, we introduce conceptual minimal pair sentences (comps), a collection of english minimal pair sentences, where each pair attributes a property (can fly) to two noun concepts: one which actually possesses the property (robin), and one which does not (penguin). following standard practice in the minimal pairs evaluation paradigm @xcite , we test whether plms prefer sentence stimuli expressing correct property knowledge over those expressing incorrect ones. comps can be decomposed into three subsets, each containing stimuli that progressively isolate deeper understanding of the task of attributing properties to concepts, by adding controls for more superficial heuristics. our first subset-comps-base-measures the extent to which plms attribute properties to the right concepts, while varying the similarity of the positive (robin) and the negative concepts @xcite vs. table @xcite ). this controls for the possibility that models are relying on coarse-grained concept distinctions. for instance, in this setup a model should prefer (1a) over both versions of (1b). next, drawing on the phenomenon of property inheritance, the comps-wugs set introduces a novel concept, wug, expressed as the subordinate of the positive and negative concepts from a subset of the comps-base set, and tests the extent to which plms successfully attribute it the given property when it is associated with the positive concept. this increases the complexity of the reasoning task, as well as the distance between the associated concept (robin) and property (can fly). these manipulations help to control for memorization of the literal phrases being tested, forcing models to judge properties for a novel concept that inherits the property from a known concept. in this task, given that a model successfully prefers (1a) over (1b), it should also prefer (2a) over ( 2b ): the final subset-comps-wugs-dist, combines the aforementioned controls by using negative concepts as distracting content and inserting them into the comps-wugs stimuli. specifically, we transform the stimuli of comps-wugs by creating two subordinates for every minimal pair; one for the positive concept (robin, subordinate: wug) and the other for the negative concept (penguin, subordinate: dax), which acts as a distractor. this way, we control for the possibility that models may be relying on simple word associations between content words-of which there are only two in the prior tests-by introducing additional, irrelevant but contentful words into the context. here, we consider models to be correct if they prefer (3a) over (3b), given that they prefer (1a) over (1b): together, the three sets of stimuli tease apart more superficial predictive behaviors, such as contextual word associations, from more robust reasoning behaviors based on understanding of concept properties. while we can expect superficial predictive strategies to be brittle in the face of shallow perturbations and irrelevant distractions, robust property knowledge and reasoning behaviors should not. we use comps to analyze robust property knowledge and its inheritance in 22 different plms, ranging from small masked language models to billion-parameter autoregressive language models. in our experiments with comps-base, we find plms to demonstrate strong performance in attributing properties to the correct concepts in our minimal pairs. however, we observe this strong performance largely when the concepts in the minimal pairs are trivially different (e.g., lion and tea for the property is a mammal). when the concept pairs are similar (on the basis of different knowledge representations), we find models' performance to degrade substantially, by as much as 25 points. we observe a similar trend in our analyses on comps-wugs-models first appear to show desirable behavior, potentially indicating proficiency in the more complex property inheritance reasoning. however, their overall performance declines drastically when investigated in the presence of distractors (i.e., on comps-wugs-dist). this failure is particularly pronounced in larger autoregressive plms, whose performance in fact drops below chance in cases where distracting information is proximal to the queried property, indicating the presence of a proximity effect. together, our findings highlight brittleness of plms with conceptual knowledge and reasoning, as evidenced by failures in the face of simple controls. we make our code and data available at: https://github.com/kanishkamisra/comps . ## general discussion and conclusion the overall goal of comps is to shed light on the extent to which plms can robustly (1) attribute to real world concepts (e.g., horse, whale) their correct properties (e.g., is a mammal); and (2) demonstrate behavior consistent with property inheritance: a reasoning process in which concepts are endowed with the properties of their superordinates @xcite @xcite . testing plms for these abilities allows us to ask key questions about how they encode and transfer knowledge. to target these capabilities more precisely, and mitigate potential inflation of performance by superficial heuristics such as coarsegrained similarity and word association, we propose incrementally increasing levels of controls in constructing our minimal pair stimuli, progressively making the task of attributing properties to concepts more challenging. findings from our initial experiment on comps-base established that the basic capacity of models to attribute properties to everyday concepts is largely coarse grained. plms were more successful in making correct property attributions when the candidate concepts were radically different, and struggled when the concepts shared semantic relations or had high co-occurrence. on testing for 'property inheritance' behavior (via comps-wugs), plms initially appeared to demonstrate reasonable success, but they also showed spurious behavior in achieving correct performance on a non-trivial number of instances for which they did not succeed in the prerequisite base condition. furthermore, this performance declined substantially in the presence of distracting information (comps-wugs-dist), providing further evidence that what property knowledge and reasoning we appear to see in these plms is more reliant on superficial heuristics than on ideal reasoning behavior. of particular note is our finding of catastrophic distraction in large autoregressive plms, whose sensitivity to proximity effects brings their overall performance well under chance, especially when scaled up to billions of parameters. contemporary work has highlighted the promise of plms on high-level tasks requiring-among other things-access to proper relational knowledge between concepts (see @xcite @xcite . by drawing on the concept of property inheritance, our experiments target reasoning ability based on perhaps the most well-established of relations-the taxonomic or the isa relation @xcite . recent work has also alluded to the proficiency of plms in capturing taxonomic information about everyday objects and entities @xcite @xcite ). findings from our controlled experiments suggest that plms' approximation of the consequences of the taxonomic relation is at best noisy, in light of clear failures especially in presence of similarity-governed competition. we conclude from our analyses that instead of robustly extracting relational information and reasoning about properties of concepts, it is likely that the plms tested here are optimized to prefer superficial cues in making word predictions, leading to mistakes and inaccuracies in presence of irrelevant and distracting information. since robust natural language understanding will be critically reliant on understanding of property knowledge and implications of property transfer, we hope that these findings will motivate adoption of rigorous assessment methods as well as work toward more robust property knowledge and reasoning in plms.