MTR : A Dataset Fusing Inductive, Deductive, and Defeasible Reasoning (2023)

Abstract: A long-standing difficulty in AI is the introduction of human-like reasoning in machine reading comprehension. Since algorithmic models can already perform as well as humans on simple quality assurance tasks thanks to the development of deep learning techniques, more difficult reasoning datasets have been presented. However, these datasets mainly focus on a single type of reasoning. There are still significant gaps in the studies when compared to the complex reasoning used in daily life. In this work, we introduce a brand-new dataset, named MTR. There are two parts to it: the first combines deductive and inductive reasoning, and the second does the same with inductive and defeasible reasoning. It consists of more than 30k QA instances, inferring relations between characters in short stories. Results show that state-of-the-art neural models do noticeably worse than expected. Our empirical results highlight the gap in the models’ ability to handle sophisticated inference.

## introduction natural language understanding (nlu) has long pursued the goal of working like a human that can perceive information and conducts logical reasoning over knowledge @xcite . deep neural networks (dnns) have achieved great success recently @xcite and have excelled at information perception tasks such as text classification and sentiment analysis @xcite @xcite . however, logical reasoning, which needs to confront a novel environment or complex task, exposes the ## conclusion in this paper, we propose mt r, a large-scale logical reasoning dataset including deductive, in-10085 ductive, and defeasible reasoning. it is a more complex relational inference dataset with a mixture of various inferences. in addition to testing the reasoning capacities of state-of-the-art neural models, our dataset helps to re-examine some deficiencies in the research of logical artificial intelligence in the era of deep learning nlp. the results demonstrate that even the most advanced machine readers lag well below human ability.