Logic-Driven Context Extension and Data Augmentation for Logical Reasoning of Text (2022)

Abstract: Logical reasoning of text requires identifying critical logical structures in the text and performing inference over them. Existing methods for logical reasoning mainly focus on contextual semantics of text while struggling to explicitly model the logical inference process. In this paper, we not only put forward a logic-driven context extension framework but also propose a logic-driven data augmentation algorithm. The former follows a three-step reasoning paradigm, and each step is respectively to extract logical expressions as elementary reasoning units, symbolically infer the implicit expressions following equivalence laws and extend the context to validate the options. The latter augments literally similar but logically different instances and incorporates contrastive learning to better capture logical information, especially logical negative and conditional relationships. We conduct experiments on two benchmark datasets, ReClor and LogiQA. The results show that our method achieves state-of-the-art performance on both datasets, and even surpasses human performance on the ReClor dataset.

## introduction recent years have witnessed a growing interest in logical reasoning of text, which learns to understand a given text in logical level and perform logical inference to deduce implications from asserted ones @xcite . as a significant component of human reading comprehension, it is essential in many application scenarios, such as negotiation and debate. and several datasets have been proposed as benchmarks for this task @xcite @xcite . an example of logical reasoning problems is shown in figure 1 , which takes a context descrip-logical symbols : extend the implicit logical expressions by laws: context: if you have no keyboarding skills at all, you will not be able to use a computer. and if you are not able to use a computer, you will not be able to write your essays using a word processing program. question： if above statements are true, which one of the following must be true? options： a. if you are not able to write your essays using a word processing program, you have no keyboarding skills. b. if you are able to write your essays using a word processing program, you have at least some keyboarding skills. c. if you are not able to write your essays using a word processing program, you are not able to use a computer. d. if you have some keyboarding skills, you will be able to write your essays using a word processing program. logical expressions : ## logic-driven context extension in this section, we present a logic-driven context extension framework for logical reasoning of text, which is illustrated in figure 2 . the framework is divided into three steps as follows. it first identifies the logical symbols and expressions explicitly mentioned in the context and options ( § 3.1). then it performs interpretable logical inference over them to extend the logical expressions implicit in the context ( § 3.2). finally, it verbalizes the extended logical expressions related to each option as an context: if you have no keyboarding skills at all, you will not be able to use a computer. and if you are not able to use a computer, you will not be able to write your essays using a word processing program. options: a. if you are not able to write your essays using a word processing program, you have no keyboarding skills. b. if you are able to write your essays using a word processing program, you have at least some keyboarding skills. c. if you are not able to write your essays using a word processing program, you are not able to use a computer. d. if you have some keyboarding skills, you will be able to write your essays using a word processing program. symbol α symbol β symbol γ implicit logical expressions: ## logic-driven data augmentation in order to make the pre-trained model put more focus on logical information in the context, especially logical negative and conditional relationships, we further introduce a logic-driven data augmentation algorithm. inspired by simclr @xcite , we augment challenging instances with literally similar but logically different contexts built by modifying logical expressions. it then adopts contrastive learning and encourages our model to distinguish logically correct context supporting the answer. we first introduce the background of sim-clr and then describe our logic-driven contrastive learning. simclr as a paradigm of self-supervised representation learning by comparing different samples, contrastive learning @xcite aims to make the representations of similar samples be mapped close together, while that of dissimilar samples be further away in the encoding space. the goal can be described as following. ## related work in recent years, there has been a surge in nlp research towards complex reasoning, such as reasoning for commonsense knowledge @xcite , numerical calculation @xcite or multi-hop aggregation @xcite . compare to these widely studied reasoning tasks, logical reasoning is also an essential and challenging capability but is relatively unexplored. natural language inference (nli) @xcite @xcite is a typical task requiring logical reasoning, which aims to determine whether a hypothesis can be reasonably entailed from a premise. however, these nli datasets mainly handle the task at sentence-level and are limited to only a few logical reasoning types, such as entailment, contradiction, and neutral. to promote a deeper passage-level logical reasoning ability, several qa datasets have been proposed. logiqa @xcite is collected from the national civil servants examination of china covering 5 logical reasoning types. @xcite propose reclor dataset from the gmat and lsat tests which examines 17 types of logical reasoning. in this paper, we take both reclor and logiqa as the testbed to investigate diverse and complicated logical reasoning skills. pre-trained language models @xcite @xcite have been widely adopted for various reasoning tasks and achieve promising performance. however, they directly encode the given texts to predict the output while failing to identify the symbolic logical structure and perform explicit logical inference for logical reasoning of text. semantic parsers @xcite are usually employed for converting texts to logical forms, and graph neural networks @xcite and neural module networks @xcite also have been attempted to partly imitate the human reasoning process. but these neural methods may not be easily generalized to our desired propositional logical schema without annotations and still perform an implicit inference. to circumvent these limitations and utilize the superior performance of neural models, we take inspiration from neuro-symbolic reasoning @xcite to integrate symbolic inference and neural representation. we design an explicit three-step logical reasoning paradigm and propose a logic-driven reasoning system to generically identify the logical structure and perform interpretable logical inference in a symbolic module while taking a pre-trained model as the backbone. ## conclusion and future work in this paper, we focus on the task of logical reasoning of text. following a three-step logical reasoning paradigm, we first propose a neuro-symbolic logicdriven context extension framework. it identifies logical expressions as elementary units of logical inference and symbolically deduces the implicitly mentioned expressions, and verbalizes them as an extended context into a pre-trained model to match the answer. we also introduce a logic-driven data augmentation algorithm, which augments literally similar but logically different instances and employs contrastive learning to help our model better capture logical information. experimental results confirm the general effectiveness of our lreasoner, and it even surpasses human performance on the reclor dataset. in the future, we will explore to model different logical reasoning types and directly incorporate symbolic logic into the model structure.