Machine Reasoning: Technology, Dilemma and Future (2020)

Abstract: Machine reasoning research aims to build interpretable AI systems that can solve problems or draw conclusions from what they are told (i.e. facts and observations) and already know (i.e. models, common sense and knowledge) under certain constraints. In this tutorial, we will (1) describe the motivation of this tutorial and give our definition on machine reasoning; (2) introduce typical machine reasoning frameworks, including symbolic reasoning, probabilistic reasoning, neural-symbolic reasoning and neural-evidence reasoning, and show their successful applications in real-world scenarios; (3) talk about the dilemma between black-box neural networks with state-of-the-art performance and machine reasoning approaches with better interpretability; (4) summarize the content of this tutorial and discuss possible future directions.

## introduction machine reasoning research aims to build interpretable ai systems that can solve problems or draw conclusions from what they are told (i.e. facts and observations) and already know (i.e. models, common sense and knowledge) under certain constraints. although its "formal" definitions vary in different publications @xcite @xcite @xcite , machine reasoning methods usually share some commonalities. first, such systems are based on different types of knowledge, such as logical rules, knowledge graphs, common sense, text evidence, etc. second, such systems use different inference algorithms to manipulate available knowledge for problem-solving. third, such systems have good interpretability to the predictions. the developments of machine reasoning systems go through several stages. symbolic reasoning methods represent knowledge using symbolic logic (e.g., propositional logic and first order logic) and perform inference using algorithms such as truthtable approach, inference rules approach, resolution, forward chaining and backward chaining. a major defect is that such methods cannot handle the uncertainty in data. probabilistic reasoning methods combine probability and symbolic logic into a unified model. such methods can deal with uncertainty, but suffer the combinatorial explosion when searching in a large discrete symbolic space. with the rapid developments of deep learning, neural reasoning methods attract much attention. neuralsymbolic reasoning methods represent knowledge symbols (such as entities, relationships, actions, logical functions and formulas) as vector or tensor representations, and allow the model to perform end-to-end learning effectively as all components are differentiable. neural-evidence reasoning methods allow the model to communicate with the environment to acquire evidence for reasoning. as such models assume the reasoning layer is not required to be logical, both structured and unstructured data can be used as knowledge. besides, as the interaction with the environment can be conducted multiple times, such approaches are good at solving sequential decision-making problems. however, existing machine reasoning methods face with a dilemma: although they have many merits such as good abstraction, generalization and interpretability, their performance are still worse than black-box neural networks (such as pretrained models) on most downstream tasks such as question answering, text classification, etc. in this tutorial, we will review typical machine reasoning frameworks and talk about the dilemma between black-box neural networks with state-ofthe-art performance and machine reasoning methods with better interpretability. we will also discuss possible research directions to escape this dilemma as the future work. ## description we first review four machine reasoning frameworks. symbolic reasoning this approach, also known as the good, old-fashioned ai (gofai), was the dominant paradigm in the ai community before the late 1980s. by manipulating knowledge in the form of symbolic logic using inference algorithms, a symbolic reasoning system can solve deductive and inductive reasoning tasks. we will use deductive reasoning as an example to show how this task can be solved based on knowledge in the form of propositional logic and first-order logic, respectively. this part is also closely related to probabilistic reasoning and neural-symbolic reasoning. probabilistic reasoning one drawback of symbolic reasoning is that it cannot handle data un-certainty. to alleviate this problem, probabilistic reasoning is proposed, which integrates probabilistic models with symbolic knowledge in a unified framework. in such systems, probabilistic models handle the uncertainty issue while the symbolic logic represents types, relations, and the complex dependencies between them. we will use bayesian network @xcite and markov logic network @xcite as two representative models to show how probabilistic reasoning can solve typical reasoning tasks, such as diagnosis, prediction and maximum probable explanation. neural-symbolic reasoning both symbolic reasoning and probabilistic reasoning support strong abstraction and generalization. such systems have good interpretability but are fragile and inflexible duo to the finite and discrete symbolic representations. on the contrary, neural network models achieve state-of-the-art performance on various ai tasks, due to their good representation and learning capabilities. however, such models cannot capture compositionality and generalization in a systematic way. they cannot provide explicit decision-making evidence to explain their outputs as well, which make such systems look like a black box. so it is straightforward to integrate neural networks with symbolic reasoning, which is called neural-symbolic reasoning in this tutorial. in general, a neural-symbolic reasoning system (1) integrates existing reasoning technologies with symbolic knowledge based on neural networks and (2) implements inference as a chain of differentiable modules, where each module represents a program with a specific function. by doing these, such systems are usually more interpretable than black-box neural networks. we will review knowledge graph reasoning @xcite @xcite @xcite @xcite @xcite @xcite @xcite @xcite , neural semantic parsing @xcite @xcite @xcite , neural module network @xcite @xcite and symbolic knowledge as constraints @xcite @xcite @xcite as four representative models. neural-evidence reasoning previously mentioned three reasoning pipelines have the merits of utilizing abstractive logical or symbolic functions, which are interpretable to developers and users at concept level. the design of such symbolic functions in real applications are typically conducted by domain experts, thus these models cannot be easily extend to broader applications. here, we review neural-evidence models that find external evidence and combine evidence with the input to make predictions. we group existing methods into three categories, including unstructured textual evidence retrieval models, structured fact evidence retrieval models, and iterative evidence retrieval models. applications include open question answering (chen and yih, 2020), commonsenseqa @xcite , fact checking and verification @xcite , inferential text generation @xcite , and multihop question answering @xcite . we then talk about the dilemma between blackbox neural networks with state-of-the-art performance and machine reasoning approaches with better interpretability. dilemma: interpretability vs. performance despite the appealing properties of the previously mentioned machine reasoning approaches in terms of interpretability, the reality is that the leading systems on open benchmarks, evaluated by accuracy, are typically black-box models. we will discuss this dilemma of "interpretability versus performance" by showing the empirical success of pretrained models on natural language understanding challenges, including grade 8 new york regents science exam @xcite , discrete reasoning over natural language @xcite , reasoning over rules in natural language @xcite , and logical reasoning @xcite . afterwards, we will review model interpretation methods, including post-hoc ones and intrinsic ones. post-hoc methods aim to interpret what an existing model learned without making changes to the original model. we will cover saliency maps @xcite , local interpretable model-agnostic explanations (lime) @xcite , testing with concept activation vectors (tcav) @xcite , and visual explanation generation @xcite . intrinsic methods are that inherently interpretable (to some extent). we will cover attention @xcite , interpretable cnn @xcite , and neural module network @xcite . we last summarize the content of this tutorial and discuss possible future directions. summary this tutorial classifies machine reasoning methods into 4 categories based on their modeling mechanisms, including symbolic reasoning, probabilistic reasoning, neural-symbolic reasoning and neural-evidence reasoning. symbolic reasoning can handle complex reasoning tasks by using logical rules. probabilistic reasoning further alleviates the data uncertainty issue in symbolic reasoning systems by introducing probabilistic models. neural-symbolic reasoning provides more robust representation and learning capabilities based on the latest deep learning technologies. neuralevidence reasoning doesn't require the reasoning layer to be logical, so they can leverage both symbolic and non-symbolic evidence. all these methods have good applications in many real-world scenarios like expert system, medical diagnosis, knowledge base completion, question answering, search engine, fact checking, etc. of course, we also notice the dilemma of existing machine reasoning methods. we think this is only a short-term phenomenon. with the continue and rapid developments of different areas at the same time, such as knowledge base engineering, pre-training, interpretability modeling and neural-symbolic computing, we believe machine reasoning will definitely have a brighter future. probabilistic reasoning (20 min.) will review typical methods based on bayesian network and markov logic network, respectively. neural-symbolic reasoning (40 min.) will review typical methods including knowledge graph reasoning, neural semantic parsing, neural module network and symbolic knowledge as constraints. neural-evidence reasoning (40 min.) will review text-base evidence retrieval models, factbased evidence retrieval models, and interactive evidence retrieval models. dilemma: interpretability vs. performance (30 min.) will review post-hoc models and intrinsic models for interpretation, and discuss the dilemma of "interpretability versus performance". summary & future discussion (10 min.) will summarize the content of this tutorial and discuss possible future directions. ## prerequisites for the attendees we expect the attendees to be familiar with typical nlp tasks (such as question answering, semantic parsing, text generation, etc.), basic concepts of logic (such as propositional logic and first order logic) and knowledge graph, recent neural network architectures (such as convolutional neural network, recurrent neural network and transformer) and pretrained language models (such as gpt and bert).