Testing Pre-trained Language Models’ Understanding of Distributivity via Causal Mediation Analysis (2022)

Abstract: To what extent do pre-trained language models grasp semantic knowledge regarding the phenomenon of distributivity? In this paper, we introduce DistNLI, a new diagnostic dataset for natural language inference that targets the semantic difference arising from distributivity, and employ the causal mediation analysis framework to quantify the model behavior and explore the underlying mechanism in this semantically-related task. We find that the extent of models’ understanding is associated with model size and vocabulary size. We also provide insights into how models encode such high-level semantic knowledge.

## introduction the ability to understand and utilize semantic knowledge (consciously or unconsciously) is essential to human reasoning process. although significant progress has been made by large-scale pre-trained language models on many reasoningrequired tasks, it is still unclear whether these models have reached a considerable level of competence in discerning and processing semantic knowledge. to break into the black box, recent studies employ various analysis methods and bring evidence that semantic knowledge @xcite @xcite is encoded by pre-trained models. however, some issues still remain. first, due to the difficulty in being analyzed and probed for, many semantic phenomena are not touched on by nlp researchers, even though they have been studied by linguists for decades. second, current analysis methods are not flawless. for example, @xcite reviews some limitations of the probing classifier paradigm such as the spurious correlation between the probing classifier and the original model. ## related work distributivity a sentence with the verb predication applying to a group, e.g. sumon and frank built a boat., can be interpreted into at least two readings: the distributive and the collective ones @xcite . in the distributive reading, the predication applies to each individual in the group (e.g. ## conclusion in this paper, we propose distnli, a diagnostic nli dataset to examine to what extent pre-trained language models can discern the phenomenon of distributivity. by extending the cma framework, we show that models including deberta and roberta have some understanding of distributivity, which provides further evidence that models have ability to encode high-level semantic knowledge, and reveal some interesting patterns related to the underlying mechanism of these models. one direction for future improvement would be increasing the diversity of predicates and subjects in distnli. at present, we only look at subjects that are two coordinated dps, but the phenomenon of distributivity applies to all noun phrases which denote groups and even without the utilization of conjuncts. it is possible that pre-trained language models can also differentiate more complicated combinations, as they are trained on large-scale text data. another direction would be investigating how robust the cma framework is to the definition of metrics, such as an empirical comparison between alternative definitions.