Neural Module Networks for Reasoning over Text

1 INTRODUCTION . Being formalism-free and close to an end-user task , QA is increasingly becoming a proxy for gauging a model ’ s natural language understanding capability ( He et al. , 2015 ; Talmor et al. , 2018 ) . Recent models have performed well on certain QA datasets , sometimes rivaling humans ( Zhang et al. , 2019 ) , but it has become increasingly clear that they primarily exploit surface level lexical cues ( Jia & Liang , 2017 ; Feng et al. , 2018 ) and compositional QA still remains a challenge . Answering complex compositional questions against text is challenging since it requires a comprehensive understanding of both the question semantics and the text against which the question needs to be answered . Consider the question in Figure 1 ; a model needs to understand the compositional reasoning structure of the questions , perform accurate information extraction from the passage ( eg . extract lengths , kickers , etc . for the field goals and touchdowns ) , and perform symbolic reasoning ( eg . counting , sorting , etc. ) . Semantic parsing techniques , which map natural language utterances to executable programs , have been used for compositional question understanding for a long time ( Zelle & Mooney , 1996 ; Zettlemoyer & Collins , 2005 ; Liang et al. , 2011 ) , but have been limited to answering questions against structured and semi-structured knowledge sources . Neural module networks ( NMNs ; Andreas et al. , 2016 ) extend semantic parsers by making the program executor a learned function composed of neural network modules . These modules are designed to perform basic reasoning tasks and can be composed to perform complex reasoning over unstructured knowledge . NMNs perform well on synthetic visual question answering ( VQA ) domains such as CLEVR ( Johnson et al. , 2017 ) and it is appealing to apply them to answer questions over text due to their interpretable , modular , and inherently compositional nature . We find , however , that it is non-trivial to extend NMNs for answering non-synthetic questions against open-domain text , where a model needs to deal with ∗Work done while at Allen Institute for AI . the ambiguity and variability of real-world text while performing a diverse range of reasoning . Jointly learning the parser and executor using only QA supervision is also extremely challenging ( §2.2 ) . Our contributions are two-fold : Firstly , we extend NMNs to answer compositional questions against a paragraph of text as context . We introduce neural modules to perform reasoning over text using distributed representations , and perform symbolic reasoning , such as arithmetic , sorting , comparisons , and counting ( §3 ) . The modules we define are probabilistic and differentiable , which lets us maintain uncertainty about intermediate decisions and train the entire model via end-to-end differentiability . Secondly , we show that the challenges arising in learning from end-task QA supervision can be alleviated with an auxiliary loss over the intermediate latent decisions of the model . Specifically , we introduce an unsupervised objective that provides an inductive bias to perform accurate information extraction from the context ( §4.1 ) . Additionally , we show that providing heuristically-obtained supervision for question programs and outputs for intermediate modules in a program ( §4.2 ) for a small subset of the training data ( 5–10 % ) is sufficient for accurate learning . We experiment on 21,800 questions from the recently proposed DROP dataset ( Dua et al. , 2019 ) that are heuristically chosen based on their first n-gram such that they are covered by our designed modules . This is a significantly-sized subset that poses a wide variety of reasoning challenges and allows for controlled development and testing of models . We show that our model , which has interpretable intermediate outputs by design , significantly outperforms state-of-the-art black box models on this dataset . We conclude with a discussion of the challenges of pushing NMNs to the entire DROP dataset , where some questions require reasoning that is hard to design modules for . 2 NEURAL MODULE NETWORKS . Consider the question “ Who kicked the longest field goal in the second quarter ? ” in Figure 1 . Multiple reasoning steps are needed to answer such a question : find all instances of “ field goal ” in the paragraph , select the ones “ in the second quarter ” , find their lengths , compute the “ longest ” of them , and then find “ who kicked ” it . We would like to develop machine reading models that are capable of understanding the context and the compositional semantics of such complex questions in order to provide the correct answer , ideally while also explaining the reasoning that led to that answer . Neural module networks ( NMN ) capture this intuition naturally , which makes them a good fit to solve reasoning problems like these . A NMN would parse such a question into an executable program , such as relocate ( find-max-num ( filter ( find ( ) ) ) ) , whose execution against the given paragraph yields the correct answer . These programs capture the abstract compositional reasoning structure required to answer the question correctly and are composed of learnable modules designed to solve sufficiently independent reasoning tasks . For example , the find module should ground the question span “ field goal ” to its various occurrences in the paragraph ; the module find-max-num should output the span amongst its input that is associated with the largest length ; and finally , the relocate module should find “ who kicked ” the field goal corresponding to its input span . 2.1 COMPONENTS OF A NMN FOR TEXT . Modules . To perform natural language and symbolic reasoning over different types of information , such as text , numbers , and dates , we define a diverse set of differentiable modules to operate over these different data types . We describe these modules and the data types in §3 . Contextual Token Representations . Our model represents the question q as Q ∈ Rn×d and the context paragraph p as P ∈ Rm×d using contextualized token embeddings . These are outputs of either the same bidirectional-GRU or a pre-trained BERT ( Devlin et al. , 2019 ) model . Here n and m are the number of tokens in the question and the paragraph , respectively . Appendix §A.1 contains details about how these contextual embeddings are produced . Question Parser . We use an encoder-decoder model with attention to map the question into an executable program . Similar to N2NMN ( Hu et al. , 2017 ) , at each timestep of decoding , the attention that the parser puts on the question is available as a side argument to the module produced at that timestep during execution . This lets the modules have access to question information without making hard decisions about which question words to put into the program . In our model , the data types of the inputs and output of modules automatically induce a typeconstrained grammar which lends itself to top-down grammar-constrained decoding as performed by Krishnamurthy et al . ( 2017 ) . This ensures that the decoder always produces well-typed programs . For example , if a module f1 inputs a number , and f2 outputs a date , then f1 ( f2 ) is invalid and would not be explored while decoding . For example , if a module f1 inputs a number , and f2 outputs a date , then f1 ( f2 ) is invalid and would not be explored while decoding . The output of the decoder is a linearized abstract syntax tree ( in an in-order traversal ) . See §A.2 for details . Learning . We define our model probabilistically , i.e. , for any given program z , we can compute the likelihood of the gold-answer p ( y∗|z ) . Combined with the likelihood of the program under the question-parser model p ( z|q ) , we can maximize the marginal likelihood of the answer by enumerating all possible programs ; J = ∑ z p ( y ∗|z ) p ( z|q ) . Since the space of all programs is intractable , we run beam search to enumerate top-K programs and maximize the approximate marginal-likelihood . 2.2 LEARNING CHALLENGES IN NMN FOR TEXT . As mentioned above , the question parser and the program executor both contain learnable parameters . Each of them is challenging to learn in its own right and joint training further exacerbates the situation . Question Parser . Our model needs to parse free-form real-world questions into the correct program structure and identify its arguments ( e.g . ” who kicked ” , ” field goal ” , etc. ) . This is challenging since the questions are not generated from a small fixed grammar ( unlike CLEVR ) , involve lexical variability , and have no program supervision . Additionally , many incorrect programs can yield the same correct answer thus training the question parser to highly score incorrect interpretations . Program Executor . The output of each intermediate module in the program is a latent decision by the model since the only feedback available is for the final output of the program . The absence of any direct feedback to the intermediate modules complicates learning since the errors of one module would be passed on to the next . Differentiable modules that propagate uncertainties in intermediate decisions help here , such as attention on pixels in CLEVR , but do not fully solve the learning challenges . Joint Learning . Jointly training the parser and executor increases the latent choices available to the model by many folds while the only supervision available is the gold answer . Additionally , joint learning is challenging as prediction errors from one component lead to incorrect training of the other . E.g. , if the parser predicts the program relocate ( find ( ) ) for the question in Fig . 1 , then the associated modules would be incorrectly trained to predict the gold answer . On the next iteration , incorrect program execution would provide the wrong feedback to the question parser and lead to its incorrect training , and learning fails . 3 MODULES FOR REASONING OVER TEXT . Modules are designed to perform basic independent reasoning tasks and form the basis of the compositional reasoning that the model is capable of . We identify a set of tasks that need to be performed to support diverse enough reasoning capabilities over text , numbers , and dates , and define modules accordingly . Since the module parameters will be learned jointly with the rest of the model , we would like the modules to maintain uncertainties about their decisions and propagate them through the decision making layers via end-to-end differentiability . One of the main contributions of our work is introducing differentiable modules that perform reasoning over text and symbols in a probabilistic manner . Table 1 gives an overview of representative modules and §3.2 describes them in detail . 3.1 DATA TYPES . The modules operate over the following data types . Each data type represents its underlying value as a normalized distribution over the relevant support . • Question ( Q ) and Paragraph ( P ) attentions : soft subsets of relevant tokens in the text . • Number ( N ) and Date ( D ) : soft subset of unique numbers and dates from the passage . 1 • Count Number ( C ) : count value as a distribution over the supported count values ( 0− 9 ) . • Time Delta ( TD ) : a value amongst all possible unique differences between dates in the paragraph . In this work , we consider differences in terms of years . • Span ( S ) : span-type answers as two probability values ( start/end ) for each paragraph token .

Summary: This works applies neural module network to reading comprehension that requires symbolic reasoning. There are two main contributions: (1) the authors designed a set of differentiable neural modules for different operations (for example, arithmetics, sorting, and counting) that is required to perform reasoning over a paragraph of text. These modules can be compositionally combined to perform complex reasoning. And the parameters of each module (which can be viewed as executor of each operation) are learned jointly with the parser that generates thee program composed of those modules. (2) To overcome the challenge of weak supervision, the authors proposed to use auxiliary loss (information extraction loss, parser supervision, intermediate output supervision). The model is evaluated on a subset of DROP, and outperforms the state-of-the-art models. Ablation studies supported the importance of the auxiliary losses.