Logical Inferences with Comparatives and Generalized Quantifiers (2020)

Abstract: Comparative constructions pose a challenge in Natural Language Inference (NLI), which is the task of determining whether a text entails a hypothesis. Comparatives are structurally complex in that they interact with other linguistic phenomena such as quantifiers, numerals, and lexical antonyms. In formal semantics, there is a rich body of work on comparatives and gradable expressions using the notion of degree. However, a logical inference system for comparatives has not been sufficiently developed for use in the NLI task. In this paper, we present a compositional semantics that maps various comparative constructions in English to semantic representations via Combinatory Categorial Grammar (CCG) parsers and combine it with an inference system based on automated theorem proving. We evaluate our system on three NLI datasets that contain complex logical inferences with comparatives, generalized quantifiers, and numerals. We show that the system outperforms previous logic-based systems as well as recent deep learning-based models.

## introduction natural language inference (nli), or recognizing textual entailment (rte), is the task of determining whether a text entails a hypothesis and has been actively studied as one of the crucial tasks in natural language understanding. in recent years, systems based on deep learning (dl) have been developed by crowdsourcing large datasets such as stanford natural language inference (snli) @xcite and multi-genre natural language inference (multinli) @xcite and have achieved high accuracy. nli datasets focusing on complex linguistic phenomena, such as negation, antonyms, and numerals, have also been developed @xcite . however, it has been pointed out that these datasets contain various biases that can be exploited by dl models @xcite , including easily classifying numerical expressions in inference @xcite and answering by only looking at a hypothesis @xcite . this suggests that the success of nli models to date has been overestimated and that tasks remain unresolved. to handle inferences involving various linguistic phenomena, there are also studies to probe the effects of additional training using artificially constructed data @xcite . however, in the case of structurally complex inferences involving comparisons and numerical expressions, there is a myriad of ways to combine possible inference patterns. for example, consider the following inference. ## system overview figure 1 shows the pipeline of the proposed system. first, the input sentences are a set of premises p 1 , . . . , p n and a hypothesis h. next, the ccg derivation trees are obtained using ccg parsers. 1 github repository with code and data: https:// github.com/izumi-h/ccgcomp derivation trees are modified to derive appropriate lfs based on a-not-a analysis. we use the semantic parsing system ccg2lambda @xcite based on λ-calculus to obtain lfs, which are then converted to the typed first-order form (tff) of the thousands of problems for theorem provers (tptp) format @xcite , that is, a formal expression in firstorder logic with equality and arithmetic operations. finally, together with the axiom system comp @xcite for comparatives and numerical expressions, a theorem prover checks whether p 1 ∧ • • • ∧ p @xmath0. the system output is yes (entailment), no (contradiction), or unknown (neutral). ## conclusion in this study, we presented an end-to-end logicbased inference system for handling complex inferences with comparatives, quantifiers, and numerals. the entire system is transparently composed of several modules and can solve complex inferences for the right reason. in future work, we will extend our analysis to cover the more complex constructions mentioned in section 3. we are also considering combining our system with an abduction mechanism that uses large knowledge bases @xcite for handling commonsense reasoning with external knowledge.