Complex Reasoning in Natural Language (2023)

Abstract: Teaching machines to reason over texts has been a long-standing goal of natural language processing (NLP). To this end, researchers have designed a diverse set of complex reasoning tasks that involve compositional reasoning, knowledge retrieval, grounding, commonsense reasoning, etc. A standard choice for building systems that perform a desired type of reasoning is to fine-tune a pretrained language model (LM) on specific downstream tasks. However, recent research has demonstrated that such a straightforward approach is often brittle. For example, Elazar et al. (2021) and Branco et al. (2021) show that, on question-answering (QA) tasks, similar performance can be achieved with questions removed from the inputs. Min et al. (2019), Chen and Durrett (2019), and Tang et al. (2021) show that models trained on multi-hop QA do not generalize to answer single-hop questions. The reasoning capabilities of these models thus remain at a surface level, i.e., exploiting data patterns. Consequently, augmenting LMs with techniques that make them robust and effective becomes an active research area. We will start the tutorial by providing an overview of complex reasoning tasks where the standard application of pretrained language models fails. This tutorial then reviews recent promising directions for tackling these tasks. Specifically, we focus on the following groups of approaches that explicitly consider problem structures: (1) knowledge-augmented methods, where the knowledge is either incorporated during fine-tuning or pretraining; (2) few-shot prompting methods, which effectively guide the models to follow instructions; (3) neuro-symbolic methods, which produce explicit intermediate representations; and, (4) rationale-based methods, one of the most popular forms of the neuro-symbolic methods, which highlight subsets of input as explanations for individual model predictions.

## problem introduction we will start with nlp tasks that require reasoning over multiple pieces of information in a provided context, covering various reasoning skills such as fact composition, mathematical reasoning, inferring semantic structures, and reasoning about entities @xcite @xcite @xcite @xcite . then, we will discuss benchmarks that combine multiple sources of information (i.e., modalities), e.g., paragraphs, tables, and images @xcite @xcite . we will present open-domain setups where external knowledge should be integrated into the reasoning process @xcite @xcite . in addition, we will review tasks that require commonsense reasoning @xcite @xcite . we will conclude this part by highlighting key practices for dataset creation, that increase data diversity and minimize annotation biases and reasoning shortcuts @xcite @xcite . in this section, we will cover methods for these three forms of knowledge in a variety of reasoning problems. for structured knowledge, kagnet @xcite is a typical method that focuses on fusing retrieved subgraphs from conceptnet for fine-tuning lms to perform commonsense reasoning. follow-up works include mhgrn @xcite , qa-gnn @xcite , and greaselm @xcite . for unstructured knowledge, we will introduce methods that encode a large knowledge corpus as neural memory modules to support knowledge retrieval for reasoning. we will start with dpr @xcite , one of the most popular methods that embed wikipedia as a dense matrix of fact embeddings. then, we will cover drkit @xcite , which improves multi-hop reasoning ability by encoding sparse entity mentions. additionally, we introduce drfact @xcite , a fact-level extension for drkit that focuses on commonsense reasoning. for instance-based knowledge, a promising direction, we will also introduce methods such as raco @xcite , recross @xcite , and qedb @xcite , which aim to exploit annotated examples to enhance reasoning. first, we will introduce prompt-design techniques like chain-of-thought prompting @xcite and least-to-most prompting @xcite , which encourage an lm to generate reasoning steps as part of the solution, helping with problem decomposition and enhanced reasoning. next, we will cover techniques that change the prompt dynamically for each input query. the methods covered in this part include selecting the training examples in the prompt @xcite and editing the prompt to incorporate feedback received on a similar-input @xcite . finally, we will cover techniques that lever-age code-generation models for complex reasoning tasks. representative techniques in this part will cover i) the use of code-generation model for structured commonsense reasoning @xcite , ii) algorithmic reasoning by expanding detailed instructions in the prompt @xcite , and iii) generating chain-of-thought styled reasoning chains in python code to tackle complex symbolic reasoning tasks @xcite . another stream of works called neural module networks @xcite @xcite propose to generate symbolic programs that are further softly executed by the corresponding neural modules. @xcite propose text module networks to solve complex tasks by decomposing them into simpler ones solvable by existing qa models and a symbolic calculator. however, most prior neural-symbolic methods require the elaborate human design of the symbolic language and the calibration of corresponding neural modules to tackle problems in a specific domain with large training data. @xcite propose binder, a new neural-symbolic system based on gpt-3 codex @xcite that supports flexible neural module calls that will enable higher coverage for the symbolic language, while only requiring few annotations. also, @xcite introduce pal, a new method based on codex that generates executable programs as the intermediate reasoning steps and leverages a python interpreter to derive final answers. this section will begin by discussing the highlevel comparison among the end-to-end, chain-ofthought, symbolic (e.g., semantic parsing), and neural-symbolic approaches. we will then move to provide a high-level overview of different neuralsymbolic approaches. in this part, we will mainly focus on neural-symbolic approaches with lms. finally, we will cover recent techniques incorporating gpt-3 codex in neural-symbolic approaches. existing methods for extracting rationales often require supervision; they either apply multi-task loss functions @xcite , or design specialized network architectures to incorporate inductive biases @xcite . because rationale annotations are expensive to collect and not always available, recent effort has been devoted to semi-supervised and unsupervised methods. @xcite leverage entity taggers to build silver reasoning chains used for rationale supervision. @xcite and @xcite design unsupervised objectives for extracting rationales in multi-hop qa systems. finally, latent-variable approaches are a natural fit for unsupervised learning @xcite @xcite . by modeling rationales as a latent variable, it provides a principled way to explicitly impose constraints in the reasoning process.