Neural Module Networks for Reasoning over Text
Authors: Nitish Gupta
Kevin Lin
Dan Roth
Sameer Singh
Matt Gardner
Venue:  (2020)
Keywords: question answering
compositionality
neural module networks
multi-step reasoning
reading comprehension

Abstract: Answering compositional questions that require multiple steps of reasoning against text is challenging, especially when they involve discrete, symbolic operations. Neural module networks (NMNs) learn to parse such questions as executable programs composed of learnable modules, performing well on synthetic visual QA domains. However, we find that it is challenging to learn these models for non-synthetic questions on open-domain text, where a model needs to deal with the diversity of natural language and perform a broader range of reasoning. We extend NMNs by: (a) introducing modules that reason over a paragraph of text, performing symbolic reasoning (such as arithmetic, sorting, counting) over numbers and dates in a probabilistic and differentiable manner; and (b) proposing an unsupervised auxiliary loss to help extract arguments associated with the events in text. Additionally, we show that a limited amount of heuristically-obtained question program and intermediate module output supervision provides sufficient inductive bias for accurate learning. Our proposed model significantly outperforms state-of-the-art models on a subset of the DROP dataset that poses a variety of reasoning challenges that are covered by our modules.

Published as a conference paper at ICLR 2020
NEURAL MODULE NETWORKS FOR REASONING OVER TEXT
Nitish Gupta', Kevin Lin"; Dan Roth', Sameer Singh² & Matt Gardner (nitishg, ( upenn. epenn.edu, Deviniin@eece. .erreley. sameer@uci edu,, mattg@allenai org !University of Pennsylvania, Philadelphia, Uniieessity of California, Berkeley, Universiity California, Irvine, *Allen Institute for Al
ABSTRACT
Answering compositional questions that require multiple steps of reasoning against text is challenging, especially when they involve discrete, symbolic operations. Neural module networks (NMNs) learn to parse such questions as executable programs composed of learnable modules, performing well on synthetic visual QA domains. However, we find that it is challenging to learn these models for non-synthetic questions on open-domain text, where model needs to deal with the diversity of natural language and perform broader range of reasoning. We extend NMNs by: (a) introducing modules that reason over a paragraph of text, performing symbolic reasoning (such as arithmetic, sorting, counting) over num- bers and dates in probabilistic and differentiable manner; and (b) proposing an unsupervised auxiliary loss to help extract arguments associated with the events in text. Additionally, we show that a limited amount of heuristically-obtaine question program and intermediate module output supervision provides sufficient inductive bias for accurate learning. Our proposed model significantly outperforms state-of-the art models on a subset of the DROP dataset that poses variety of reasoning challenges that are covered by our modules.
INTRODUCTION
Being formalism-free and close to end-user task, QA increasingly becoming proxy for gauging
a model's natural language understanding capability (He et al. 2015 Talmor et al. 2018). Recent models have performed well on certain QA datasets, sometimes rivaling humans (Zhang et al. 2019)
but it has become increasingly clear that they primarily exploit surface level lexical cues (Jia & Liang 2017: Feng et al. 2018) and compositional QA still remains challenge. Answering complex compositional questions against text is challenging since it requires comprehensive understanding of both the question semantics and the text against which the question needs to be answered. Consider the question in Figure D model needs to understand the compositional reasoning structure of the questions, perform accurate information extraction from the passage (eg. extract lengths, kickers, etc. for the field goals and touchdowns), and perform symbolic reasoning (eg. counting, sorting, etc.).
Semantic parsing techniques, which map natural language utterances to executable programs, have been used for compositional question understanding for long time (Zelle & Mooney] 1996; Zettle- moyer & Collins| 2005 Liang et al. 2011), but have been limited to answering questions against structured and semi-structured knowledge sources. Neural module networks (NMNs: Andreas et al.) 2016) extend semantic parsers by making the program executor a learned finction composed of neural network modules. These modules are designed to perform basic reasoning tasks and can be composed to perform complex reasoning over unstructured knowledge.
NMNs perform well on synthetic visual question answering (VQA) domains such as CLEVR [Johnson] et al., 2017) and it is appealing to apply them to answer questions over text due to their interpretable, modular, and inherently compositional nature. We find, however, that it is non non-trivial to extend NMNs for answering non-synthetic questions against open-domain text, where model needs to deal with
Work done while Allen Institute for AL.

---

Published as a conference paper at ICLR 2020
Who Kicked the longest field goel the second quarter?
In the first quarter, Buffalo tralled as Chiefs Q8 Tyler Thigpen completed a 36-yard Jamaal Charles. The Bills responded with RB Marshawn Lynch getting a 1-yard touchdown run. In the second quarter, Buffalo took the lead as kocker Rian Lindell made 21-yard and 40-yard field goal. Kansas City answered with Thigpen completing a 2-yard TD pass. Bullalo regained the lead as Lindell gol 39-yard field goal. The Chiefs struck with kicker Connor Barth getting 45-yard feed goal, yet the Bills continued their offensive explosion as Lindell got a 34-yard field goal, along with QB Edwards getting a 15-yard TD run. In the third quarter, Buffalo continued its poundings with Edwards getting a 5-yard TD run,
TD pass
Question Porser
zelocate( ind max hunddddnnnn--ntttttttttttt mun(((ttoooot(((())))
Progran Executar
Find
filter
find-nex-nun pelocate
Lindell gol himsell a 48-yard field goal. Kansas City tried to rally as Thigpen completed a Bradley, yet Bills replied with Edwards completing an 8-yard TD pass to WR Josh Reed. In the fourth quarter, Edwards completed 17-yard TD pass to TE Derek Schouman.
fiold gool
second quarter
Who Ricked
45-yard pass WR
Answes:
Connar Barth
Figure 1: Model Overview: Given question, our model parses it into a program composed of neural modules. This program is executed against the context to compute the final answer. The modules operate over soft attention values (on the question, passage, numbers, and dates). For example, filter takes as input attention over the question (in the second quarter) and filters the output of the find module by producing attention mask over tokens that belong to the second quarter.
the ambiguity and variability of real-world text while performing diverse range of reasoning. Jointly learning the parser and executor using only QA supervision is also extremely challenging ($2.2).
Our contributions are two-fold: Firstly, we extend NMNs to answer compositional questions against a paragraph of text as context. We introduce neural modules to perform reasoning over text using distributed representations, and perform symbolic reasoning, such as arithmetic, sorting, comparisons, and counting (33) The modules we define are probabilistic and differentiable, which lets us maintain uncertainty about intermediate decisions and train the entire model via end-to-end differentiability.
Secondly, we show that the challenges arising in learning from end-task QA supervision can be alleviated with an auxiliary loss over the intermediate latent decisions of the model. Specifically, we introduce an unsupervised objective that provides an inductive bias to perform accurate information extraction from the context ($4.I). Additionally, we show that providing heuristically-obtained supervision for question programs and outputs for intermediate modules in program (44.2) for small subset the training data (5-10%) is sufficient for accurate learning.
We experiment on 21,800 questions from the recently proposed DROP dataset (Dua et al. 2019) that are heuristically chosen based on their first n-gram such that they are covered by our designed modules. This is significantly-siin and allows for controlled development and testing of models. We show that our model, which has interpretable intermediate outputs by design, significantly outperforms tfate-of-the-ar black box models on this dataset. We conclude with a discussion of the challenges of pushing NMNs to the entire DROP dataset, where some questions require reasoning that is hard to design modules for.
subset that poses wide variety of reasoning challenges
2 NEURAL MODULE NETWORKS
Consider the question "Who kicked the longest field goal in the second quarter?" in Figure 1 Multiple reasoning steps are needed to answer such a question: find all instances of "field goal" in the paragraph, select the ones "in the second quarter", find their lengths, compute the "longest" of them, and then find "who kicked" it. We would like to develop machine reading models that are capable of understanding the context and the compositional semantics of such complex questions in order to provide the correct answer, ideally while also explaining the reasoning that led to that answer.
Neural module networks (NMN) capture this intuition naturally, which makes them a good fit to solve reasoning problems like these. A NMN would parse such a question into an executable program, such as relocate ( (find-max-nun) ( (filter ( (find () ), whose execution against the given paragraph yields the correct answer. These programs capture the abstract compositional reasoning structure required to answer the question correctly and are composed of learnable modules designed to solve sufficiently independent reasoning tasks. For example, the find module should ground the question span "field goal" to its various occurrences in the paragraph; the module find-max um should output the span amongst its input that is associated with the largest length; and finally, the rel I locattt e module should find "who kicked" the field goal corresponding to its input span.
2

---

Published as a conference paper at ICLR 2020
2.1 COMPONENTS OF A NMN FOR TEXT
Modules. To perform natural language and symbolic reasoning over different types of information, such as text, numbers, and dates, we define diverse set of differentiable modules to operate over these different data types. We describe these modules and the data types in {
Contextual Token Representations. Our model represents the question y as Q E RRadd and the context paragraph p as P E IRx using contextualized token embeddings. These are outputs of either the same bidirectional-Gii the number of tokens in the question and the paragraph, respectively, Appendix A..] contains details about how these contextual embeddings are produced.
or a pre- pre-trained BERT (Devlin et al. 2019) model. Here n and m
mecoder-decoder model with attention to map the question into an executable program. Similar to N2NMN (Hu et al.] (017). at each timestep of decoding, the attention that the parser puts on the question is available as side argument the module produced that timestep during execution. This lets the modules have access to question information without making hard decisions about which question words to put into the program.
Question Parser. We
In our model, the data types of the inputs and output of modules automatically induce a type- constrained grammar which lends itself top-down trammar--onssraine decoding as performed by Krishnamurthy et al. (017). This ensures that the decoder always produces well-typed programs. For example, if module fr inputs a number, and fz outputs a date, then fa((2) is invalid and would not be explored while decoding. For example, if a module f1 inputs number, and fz outputs a date, then (2) invalid and would not be explored while decoding. The output of the decoder is a linearized abstract syntax tree (in in-order traversal). See AA.2 for details.
Learning. We define our model probabilistically, i.e., for any given program z, we can compute the likelihood of the gold-answer ply" zz). Combined with the likelihood of the program under the question-pant model p( p(z)g), we can maximize the marginal likelihood of the answer by enumerating all possible programs; Σg p(y" (z)g(z))) Since the space of all programs is intractable, beam search to enumerate top- K programs and maximize the approximate marginal-likeliooo. likelihood.
2.2 LEARNING CHALLENGES IN NMN FOR TEXT
As mentioned above, the question parser and the program executor both contain learnable parameters. Each of them is challenging to learn in its own right and joint training further exacerbates the situation.
Our model needs to parse free-form real-world questions into the correct program structure and identify its arguments (e.g. "who kicked", "field goal", etc.). This is challenging since the questions are not generated from small fixed grammar (unlike CLEVR), involve lexical variability, and have no program supervision. Additionally, many incorrect programs can yield the same correct answer thus training the question parser to highly score incorrect interpretations.
Question Parser.
Program Executor. The output of each intermediate module in the program is latent decision by the model since the only feedback available for the final output of the program. The absence of any direct feedback to the intermediate modules complicates learning since the eттоrs of one module would be passed on the next. Differentiable modules that propagate uncertainties in intermediate decisions help here, such as attention on pixels in CLEVR, but do not fully solve the learning challenges.
Joint Learning. Jointly training the parser and executor increases the latent choices available to the model by many folds while the only supervision available is the gold answer. Additionally, joint learning is challenging prediction errors from one component lead to incorrect training of the other. E.g.. the parser predicts the program relocate (find ()) ) for the question in Fig. then the associated modules would be incorrectly trained to predict the gold answer. On the next iteration,
incorrect program execution would provide the wrong feedback to the question parser and lead to its incorrect training, and learning fails.
3

---

Published as a conference paper at ICLR 2020
3 MODULES FOR REASONING OVER TEXT
Modules are designed to perform basic independent reasoning tasks and form the basis of the compositional reasoning that the model is capable of. We identify a set of tasks that need to be performed to support diverse enough reasoning capabilities over text, numbers, and dates, and define modules accordingly. Since module parameters will be learned jointly with the rest of the model, we would like the modules to maintain uncertainties about their decisions and propagate them through the decision making layers via end-to-end differentiaailltyy. One the main contributions of our work introducing differentiable modules that perform reasoning over text and symbols in probabilistic manner. Table gives an overview of representative modules and (3.2 describes them in detail.
Module
In
Out
Task
find
Q
P
For question spans in the input, find similar spans in the passage Based on the question. select a subset of spans from the input Find the argument asked for the question for input paragraph spans
filter
P
P
relocate
Q.
find-num
P
} Find the number(s) / date(s) associated to the input paragraph spans
find-date
P
D
count
P
C
Count the number of input passage spans Output the span associated with the smaller number. Difference between the dates associated with the paragraph spans Select span that associated with the largest number Identify contiguous span from the attended tokens
comparc-nun-1t 1t
P
time-diff
P,P
find-max-nin
P
span
P
S
Table 1: Description of the modules we define and their expected behaviour. All inputs and outputs are represented as distributions over tokens, numbers, and dates as described in (3.1
3.1 DATA TYPES
The modules operate the following data types. Each data type represents its underlying value as a normalized distribution over the relevant support.
Question (2) and Paragraph (P) attentions: soft subsets of relevant tokens in the text.
Number (N) and Date (D): soft subset of unique numbers and dates from the passage.
the supported count values (0 9). Time Delta (TD): value amongst all possible unique differences between dates in the
Count Number (C): count value distribution
paragraph. In this work, we consider differences in terms of years.
Span (S): span-type answers as two probability values (start/end) for each paragraph token.
3.2 NEURAL MODULES FOR QUESTION ANSWERING
The question and paragraph contextualized embeddings (Q and P) are available as global variables to all modules in the program. The question attention computed by the decoder during the timestep the module was produced is also available to the module as a side argument, as described in $2.1
find (Q) - - This module is used to ground attended question tokens to similar tokens in the paragraph (e.g., "field goal" in Figure Ip. We use question-to-ppeeee to-paragraph attention matrix A E >x whose i-th row is the distribution of similarity over the paragraph tokens for the i-th question token. The output is an expected paragraph attention; a weighted-sun of the rows A, weighed by the input question attention, P= Σ, Q, A,; R.. A is computed by normalizing (using softmax) the rows a question to- paragraph similarity matrix S E Rm Here S.y is the similarity between the contextual embeddings of the i-th question token and the j-th paragraph token computed as, Sij = wt I Qu: P Qu o p.:,, where Wy E Rd is learnable parameter vector of this module, [:] denotes the concatenation operation, and is elementwise multiplication.
This module masks the input paragraph attention conditioned the question, selecting a subset of the attended paragraph (c.g., selecting fields goals "in the second
filter (Q, P)
We extract numbers and dates pre-pracessing step explained in the Appendix A.3
4

---

Published as a conference paper at ICLR 2020
quarter" in Fig. D. We compute locally-normalizat paragraph-token mask M R where Mj is the masking score for the j-th paragraph token computed as M, o (Wiiter" (q: [q: i PoPy p-
Here q- >. Qi Q. E I2 is weighted sum of question-token embeddings, Wflter T E R is learnable parameter vector, and a is the sigmoid non- non-linearity function. The output is a normalized masked input paragraph attention, Puttered normalize M o P).
relocate (Q, P) + This module re-attends to the paragraph based on the question and is used to find the arguments for paragraph spans (e.g., shifting the attention from "field goals" "who kicked" them). We first compute a paragraph- paragraph
-o-raggaphtt-ppaagrrpphhh attention matrix R E R R based on the question, as R.j = Wrelocate f [(q + P.:): ; (q P..) pj.l. where q E, Q: Qu E Rd, and Wrelocate E 3d is a learnable parameter vector. Each row of R. is also normalized using the softmax operation. The output attention is weighted sum of the rows R weighted by the input paragraph attention, Piclocated = E. P. R
find-num (P) +N This module finds a number distribution associated with the imput paragraph attention. We use a paragraph token-to-number-bee token attention map A num E IR m row is probability distribution over number-containing tokens for the i-th paragraph token. We first compute a token-to-number similarity matrix S E R x Nusen
Nutes whose i-th
as, Saum., ij = P.W W mmm P az: where n2 is the index of the j-th number token and W num R is learnable parame- ter. A mum, = softmax(S (Stum s.). We compute an expected distribution over the number tokens T E; P. A uum,
;: and aggregate the probabilities for number-tokens with the same value to compute the output distribution N. For example, if the values of the number-tokens are [2, 2, 3, 4] and T [0.1, 0.4, 0.3, 0.2]. the output will be a distribution over {2, 3, 4} with N [0.5, 0.3, 0.2).
find-date (P) - D follows the same process as above to compute a distribution over dates for the input paragraph attention. The corresponding learnable parameter matrix is W date Rddd
count (P) + C This module is used to count the number of attended paragraph spans. The idea is to learn a module that detects contiguous spans of attention values and counts each as one. For example, if attention vector is [0. O, 0.3. 0.3, O, 0.4], the count module should produce an output of 2. The module first scales the attention using the values [1, 2, 5, 10] to convert it into matrix Pacated E R A bidirectional-GRU then represents each token attention
hidden vector ı4, A single-layer feed-forward network maps this representation to a soft 0/1 score to indicate the presence of span surrounding it. These scores are summed compute count value, C. = oo (F FcoCuuntGRRU/ (Pscatea)) E R. We hypothesize that the output count value is normally distributed with oy as mean, and constant variance over the supported count values, as p(c) x exp(-(e-o,)3 //20²) Vc E 0. 9] Pretraining this module by generating synthetic data of attention and count values helps (see A.4)..
0.5, and compute categorical distribution
This module performs soft less-than operation between two passage distributions. For example, to find the city with fewer people, cityA or cityB, the module would output linear combination of the two input attentions weighted by which city was associated with a lower number. This module internally calls the find-nnun num module to get a number distribution for each of the input paragraph attentions, N₁ and N2. It then computes two soft boolean values, p(N1 < N2) and p(N2 < N1), and outputs a weighted sum of the input paragraph attentions. The boolean values are computed by marginalizing the relevant joint probabilities:
compare- compare-num-lt (P1, P2)
p(Ni N2) ΣΣ'
Ny <g NiNg
P(No < Ni)
ΣΣ NEENN NANY
The final output is, Pout = =PNN1 < N2) * + P(N, N1) * Pp. When the the predicted number distributions are peaky, p(N < N2) or ((N2 < N1) is close to 1, and the output is either P1 or Pa.
We similarly include the comparison modules Campare-num-o0 compare deieaare--at---ee 1t, and compare-dater gt. defined in an essentially identical manner, but for greater-than and for dates.
time-diff (P1, P2) TD The module outputs the difference between the dates associated with the two paragraph attentions as a distribution over all possible difference values. The module internally calls the f inddate module get a date distribution for the two paragraph attentions,
5

---

Published as a conference paper at ICLR 2020
D1 and D2. The probability of the difference being tu is computed by marginalizing over the joint probability for the dates that yield this value, p(ta) Eig ( dd
find-max-num (P) - P, find-min-num (P) + Given a passage attention attending to multiple spans, this module outputs an attention for the span associated with the largest (or smallest) number. We first compute an expected number token distribution using find-num, then use this to compute the expected probability that each number token is the one with the maximum value, 1 E IR Nuazs (explained below). We then re-distribute this distribution back to the original passage tokens associated with those numbers. The contribution from the i-th paragraph token to the j-th number token, Тj. was P. A num To compute the new attention value for token i, we re- we-weight this contribution based the ratio 73 /1,, and marginalize across the number tokens to get the new token num xj-
attention value: P. = Σj T
Computing 7 m Consider a distribution over numbers N, sorted in an increasing order. Say we sample set S (size n)) of numbers from this distribution. The probability that Nj is the largest number in this set is p(æ I %N₂)" p(r S N i.e. all numbers in are less than or equal to N,, and at least one number is N5- By picking the set size 3 as hyperparameter, we can analytically (and differentiably) convert the expected distribution over number tokens, T, into a distribution over the maximum value 7maax
span (P) S This module is used to convert a paragraph attention into a contiguous answer span and only appears as the outermost module in a program. The module outputs two probability distributions, P. and Pe E IR" denoting the probability of token being the start and end of span, respectively. This module implemented similar to the count module (see A.5
4 AUXILIARY SUPERVISION
As mentioned in 2.2
jointly learning the parameters of the parser and the modules using only end- task QΛ supervision is extremely challenging. To overcome issues in learning, (a) we introduce an unsupervised auxiliary loss provide an inductive bias to the execution of find-noon -nd-num, find dadte and re relocate modules (:44II;; and (b) provide heuristically-obtainee supervision for question program and intermediate module output (44.2) for subset of questions (5-10%).
4.1 UNSUPERVISED AUXILIARY LOSS FOR IE
The find- num, fund-nu, relevant arguments for entities and events mentioned in the context. In our initial experiments we found that these modules would often spuriously predict high attention score for output tokens that appear far away from their corresponding inputs. We introduce an auxiliary objective to induce the idea that the arguments of mention should appear near it. For any token, the objective increases the sum of the attention probabilities for output tokens that appear within window W = 10, letting the model distribute the mass within that window however likes. The objective for the find- num is
and relocate modules perform information extraction by finding
Nusiaa
H loss
log
1 A Aum
==0
We compute a similar loss for the date-attention map A dute (H and the relocate-map R (H) The final auxiliary loss is Hosss = Hn
+ H + H
4.2 QUESTION PARSE AND INTERMEDIATE MODULE OUTPUT SUPERVISION
Question Parse Supervision. Learning to parse questions in a noisy feedback environment is very challenging. For example, even though the questions in CLEVR are programmatically generated, Hu et al. (2017) needed to pre-train their parser using external supervision for all questions. For DROP, have no such external supervision. In order bootstrap the parser, questions manually and come up with a few heuristic patterns to get program and corresponding question attention supervision (for modules that require for subset of the training data (10%
analyze some

---

Published as a conference paper at ICLR 2020
of the questions; see [A.6 For example, for program find-num donddnuu(fiidee (find-max-nun) (find () )
we provide supervision for question tokens to attend to when predicting the find module.
Intermediate Module Output Supervision. Consider the question, "how many yards was the shortest goal?". The model only gets feedback for how long the shortest goal is, but not for other goals. Such feedback biases the model in predicting incorrect values for intermediate modules (only the shortest goal instead of all in find-num) which in turn hurts model generalization.
We provide heuristically-ohtaine
noisy supervision for the output of the find-num and find-date modules for subset of the questions (5%) for which we also provide question program supervision. For questions like 'how many yards was the longest/shortest touchdown?", we identify all instances of the token "touchdown" in the paragraph and assume the closest number to it should be an output of the find-num module. We supervise this as a multi-hot vector N and use an auxiliary loss, similar to inestion-attention loss, against the output distribution N of find-nurt We follow the same procedure for a few other question types involving dates and numbers; see {A.7 for details.
5
EXPERIMENTS
5.1 DATASET
We perform experiments on a portion of the recently released DROP dataset (Dua et al. 2019), which to the best of our knowledge is the only dataset that requires the kind of compositional and symbolic reasoning that our model aims to solve. Our model possesses diverse but limited reasoning capability; hence, we try to automatically extract questions in the scope of our model based on their first n-gram. These n- n-grams were selected by performing manual analysis on a small set of questions. The dataset construct contains 20, 000 questions for training/validation, and 1800 questions for testing (25% of DROP). Since the DROP test set is hidden, this test set is extracted from the validation data. Though this is a subset of the full DROP dataset it is still significantly-sizze dataset that allows drawing meaningful conclusions. We make our subset and splits available publicly with the code.
Based the manual analysis classify these questions into different categories, which are: Pate-Compare e.g. What happened last, commission being granted to Robert or death of his cousin?
Date-Difference e.g. How many years after his attempted assassination was James II coronated? Humber-Comper e.g. Were there more of cultivators or main agricultural labourers in Sweden? Extract-Number e.g. How many yards was Kasay's shortest field goal during the second half? Count e.g. How many touchdowns did the Vikings score in the first half?
Extract-Argument e.g. Who threw the longest touchdown pass the first quarter?
Auxiliary Supervision Out of the 20, 000 training questions, we provide question program su- pervision for 10% (2000), and intermediate module output supervision for 5% (1000) of training questions. We use curriculum learning (Bengio et al. 2009)) where the model is trained only on heuristicalll-suuerriied non-count questions for the first 5 epochs.
5.2 RESULTS
We compare to publicly available best performing models: NAQANet (Dua al. 2019). NABERT+ (Kinley & Lin (019). TAG-NABERT+ (Avia Efrat & Shoham 2019). and MTMSN (Hu jet al.) 2019), all trained on the same data as our model. We implement our model using Al-
lenNLP (Gardner et al. 2018
The hyperparameters used for our model are described in the appendix. All results are reported as an average of 4 model runs.
Overall. Table 2a compares our model's performance to state-of-the-ard models on our full test set.
Our model achieves an Fl score of 73.1 (w/ GRU) and significantly outperforms NAQANet (62.1 F1). Using BERT representations, our model's performance increases to 77.4 F1 and outperforms SoTA models that use BERT representations, such as MTMSN (76.5 F1). This shows the efficacy of our proposed model in understanding complex compositional questions and performing multi-step
Our code is available at http:://agcoop.or////gggg/////

---

Published as a conference paper at ICLR 2020
Model
F1
EM
Question Type
MTMSN
Our Model (w/ BERT)
NAQANET
62.1
57,9
DATE-COMPE COMPARE (18.6%)
85.2
82.6
TAG-NABERT+
74.2
70.6
DATE- DIFFERENCE (17.9%)
72.5
75.4
75.4
72.0
NUMBER- COMPARE (19.3%)
85.1
92.7
MTMSN
76.5
73.1
EXTRACT-NUMBER (13.5%)
80.7
86.1
OUR MODEL (W/ GRU)
73.1
69.6
COUNT (17.6%)
61.6
55.7
OUR MODEL (W/ BERT)
77.4
74.0
EXTRACT-ARGUMENT (12.8%)
66.6
69.7
(a) Performance on DROP (pruned)
(b) Performance by Question Type (F1)
Table 2: Performance of different models on the dataset and across different question types.
reasoning over natural language text. Additionally, this shows that structured models still benefit when used over representations from large pretrained-LMMs, such BERT.
Performance by Question Type. Table 2b shows the performance for different question types as identified by our heuristic labeling. Our model outperforms MTMSN on majority of question types but struggles with counting questions; it outperforms MTMSN on only some of the runs. Even after pre-training the count module using synthetic data, training it is particularly unstable. We believe this because feedback from count questions is weak, i.e., the model only gets feedback about the count value and not what the underlying set is; and because it was challenging to define a categorical count distribution given a passage attention distribution finding a better way parameterize this function is an interesting problem for future work.
Effect of Additional Supervision. Figure shows that the unsupervised auxiliary objective signif- icantly improves model performance (from 57.3 3 to 73.1 FI). The model using BERT diverges while training without the auxiliary objective. Additionally, the intermediate module output supervision has slight positive effect on the model performance.
Effect of Training Data Size. Figure 2b shows that model significantly outperforms MTMSN when training using less data, especially using 10-25% of the available supervision. This shows that by explicitly modeling compositionality, our model is able additional auxiliary supervision effectively and achieves improved model generalization.
Incorrect Program Predictions. Mistakes by our model can be classified into two types; incorrect program prediction and incorrect execution. Here we show few mistakes of the first type that highlight the need to parse the question context conditional manner.
How many touchdown passes did Tom Brady throw the season? count (find) is incorrect since the correct answer requires simple lookup from the paragraph.
2. Which happened last, failed assassination attempt on Lenin, or the Red Terror? date-compare- re-gt (find, find) incorrect since the correct answer requires natural language inference about the order of events and not symbolic comparison between dates.
3. Who caught the most touchdown passes? rel ocate (f Winddmaxx --nu (find) ) Such questions, that require nested counting, are out of scope of our defined modules because the model would first need to to count the passes caught by each player.
RELATED WORK
Semantic parsing techniques have been used for a long time for compositional question understanding. Approaches have used labeled logical-forms (Zelle & Mooney 1996. Zettlemoyer & Collins 2005),
or weak QA supervision (Clarke et al. 2010 Berant et 2013 Reddy et al.) (2014) to learn parsers
to answer questions against structured knowledge bases. These have also been extended for QA
using symbolic reasoning against semi-structured tables (Pasupat & Liang 2013 Krishnamurthy
et al. 2017 Neelakantan al. 2016). Recently, BERT-baaed models for DROP have been been
8

---

Published as a conference paper at ICLR 2020
Supervision Type
w/ BERT
w/ GRU
Hus
MOD-SUP
77.4
73.1
76.3
71.8
Wodel
57.3
80
MTMSN
(a) Effect Auxiliary Supervision: The aux- iliary loss contributes significantly to the perfor- mance, whereas module output supervision has little effect. *Training diverges without Huss for the BERT-based model.
25
50
100
Parcenlaage of Training Data
(b) Performance with less training data: Our model performs significantly better than the baseline with less training data, showing the efficacy of explicitly modeling compositionality.
Figure 2: Effect of auxiliary losses and the size of training data model performance.
proposed (Hu et 2019 Andor al. 2019 Kinley & Lin 2019), but all these models essentially perform a multiclass classification over pre dre-ne programs. Our model on the other hand provides an interpretable, compositional parse of the question and exposes its intermediate reasoning steps.
For combining learned execution modules with semantic parsing, many variations to NMNs have been proposed; NMN (Andreas et al. 2016) use a PCFG parser to parse the question and only learn module parameters. N2NMNs (Hu et al. (017) simultaneously learn to parse and execute but require pre-training the parser. Gupta Lewis (2018) propose NMN model for QA against knowledge graphs and learn execution for semantic operators from QA supervision alone. Recent works (Gupta) Lewis 2018 Mao et al. 2019) also use
to alleviate issues in learning by using curriculum learning to train the executor first simple questions for which parsing is not issue. All these approaches perform reasoning on synthetic domains, while our model is applied to natural language. Concurrently, Jiang & Bansal (2019) apply NMN to HotpotQA (Yang et al. (2018) but their model comprises of only 3 modules and not capable of performing symbolic reasoning.
7 FUTURE DIRECTIONS
We try a trivial extension to our model by adding a module that allows for addition subtraction between two paragraph numbers. The resulting model achieves score of 65.4 Fl on the complete validation data DROP, as compared to MTMSN that achieves 72.8 F1.
Manual analysis of predictions reveals that significant majority of mistakes are due to insufficient reasoning capability in our model and would require designing additional modules. For example, questions such (a) How many languages each had less than 115, 000 speakers in the population? and Which racial groups are smaller than 2% would require pruning passage spans based on the numerical comparison mentioned in the question; (b) Which quarterback threw the most touchdown passes? and In which quarter did the teams both score the same number of points? would require designing modules that considers some key-value representation of the paragraph; (c) How many points did the packers fall behind during the game? would require IE for implicit argument (points scored by the other team). It is not always clear how to design interpretable modules for certain operations; for example, for the last two cases above.
is worth emphasizing here what happens when we try to train our model on these questions for which our modules can't I express the correct reasoning. The modules in the predicted program get updated to try perform the reasoning anyway, which harms their ability to execute their intended operations (cf. 42.2 This is why we focus on only subset of the data when training model.
In part due to this training problem, some other mistakes of our model relative to MTMSN on the full dataset are due to incorrect execution of the intermediate modules. For example, incorrect grounding by the find module, or incorrect argument extraction by the find-num module. For mistakes such as these, our NMN based approach allows for identifying the cause of mistakes and supervising these modules using additional auxiliary supervision that is not possible in black-box models. This additionally opens up avenues for transfer learning where modules can be independently trained

---

Published as a conference paper at ICLR 2020
using indirect distant supervision from different tasks. Direct transfer of reasoning capability in black-box models is not so straight-forward.
To solve both of these classes of errors, one could black-box models, which gain performance on some questions at the expense of limited interpretability. It is not trivial to combine the two approaches, however. Allowing black-box operations inside of neural module network significantly harms the interpretability-- e.g. an operation that directly answers a question after an encoder, mimicking BERT-QA-style models, encourages the encoder to perform complex reasoning in non-interpretable way. This also harms the ability of the model to use the interpretable modules even when they would be sufficient to answer the question. Additionally, due to our lack of supervised programs, training the network to use the interpretable modules instead of a black-box shortcut module is challenging, further compounding the issue. Combining these black-box operations with the interpretable modules that we have presented is an interesting and important challenge for future work.
CONCLUSION
We show how to use neural module networks to answer compositional questions requiring symbolic reasoning against natural language text. We define probabilistic modules that propagate uncertainty about symbolic reasoning operations in a way that is end-to-end differentiable. Additionally, we show that injecting inductive bias using unsupervised auxiliary losses significantly helps learning.
While we have demonstrated marked success in broadening the scope of neural modules and applying them open-domain text, it remains a significant challenge extend these models the full range of reasoning required even just for the DROP dataset. NMNs provide interpretability, compositionality, and improved generalizability, but at the cost of restricted expressivity compared to more black box models. Future research is necessary to continue bridging these reasoning gaps.
ACKNOWLEDGMENTS
We would like to thank Daniel Deutsch and the anonymous reviewers for their helpful comments. This material is based upon work sponsored in part by the DARPA MCS program under Contract No. N66001 1924033 with the United States Office Of Naval Research, an ONR award, the LwLL DARPA program, and grant from A12.
REFERENCES
Daniel Andor, Luheng He, Kenton Lee, and Emily Pitler. Giving BERT a Calculator: Finding Operations and Arguments with Reading Comprehension. ArXiv, abs/1909.00109 2019.
Jacob Andreas, Marcus Rohrbach, Trevor Darrell, and Dan Klein. Neural module networks. In CVPR, 2016.
Elad Segal Avia Efrat and Mor Shoham. Tag-based multi-span extraction in reading comprehension. 2019. URL https://github. com/ http://ceeeeeaeeeeeeeeee http://llaaeelllrpeeeetttct -IP--AM
Yoshua Bengio, Jérôme Louradour, Ronan Collobert, and Jason Weston. Curriculum learning. In ICML, 2009.
Jonathan Berant, Andrew Chou, Roy Frostig, and Percy Liang, Semantic parsing Freebase from Question-Answer pairs. In EMNLP, 2013.
James Clarke, Dan Goldwasser, Ming-Wes Wei Chang, and Dan Roth. Driving semantic parsing from the world's response. CoNLL, 2010.
Jacob Devlin, Ming- Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. In NAACI-HHLT, 2019.
Dheeru Dua, Yizhong Wang, Pradeep Dasigi, Gabriel Stanovsky, Sameer Singh, and Matt Gardner. DROP: A reading comprehension benchmark requiring discrete reasoning over paragraphs. In NAACI-HITT, 2019.
10

---

Published as a conference paper at ICLR 2020
Shi Feng, Eric Wallace, Alvin Grissom, Mohit lyyer, Pedro Rodriguez, and Jordan L. Boyd-Graber. Pathologies of neural models make interpretation difficult. In EMNLP, 2018.
Matt Gardner, Joel Grus, Mark Neumann, Oyvind Tafjord, Pradeep Dasigi, Nelson Liu, Matthew Peters, Michael Schmitz, and Luke S. Zeulemmyer. Allennlp: A deep semantic natural language processing platform. ArXiv, abs/ 1as/1803307640 2018.
Nitish Gupta and Mike Lewis. Neural compositional denotational semantics for question answering. In EMNLP, 2018.
Lubeng He, Mike Lewis, and Luke S. Zettlemoyer. Question-answer driven semantic role labeling: Using natural language annotate natural language. In EMNLP, 2015.
Minghao Hu, Yuxing Peng, Zhiheng Huang, and Dongsheng Li. multi-type multi-span network for reading comprehension that requires discrete reasoning. In EMNLP, 2019.
Ronghang Hu, Jacob Andreas, Marcus Rohrbach, Trevor Darrell, and Kate Saenko. Learning to reason: End-to-end module networks for visual question answering. In ICCV, 2017.
Robin Jia and Perey Liang. Adversarial examples for evaluating reading comprehension systems. In EMNLP, 2017.
Yichen Jiang and Mohit Bansal. Self-assembling modular networks for interpretable multi-hop
reasoning. In EMNLP, 2019.
Justin Johnson, Bharath Hariharan, Laurens van der Maaten, Li Fei-Fei, C. Lawrence Zitnick, and Ross B. Girshick. CLEVR: A diagnostic dataset for compositional language and elementary visual reasoning. CVPR, 2017.
Jambay Kinley and Raymond Lin. Nabert+ Improving numerical reasoning in reading comprehen- sion. 2019. URL
Jayant Krishnamurthy, Pradeep Dasigi, and Matt Gardner. Neural semantic parsing with type constraints for semi-structured tables. In EMNLP, 2017.
Percy S. Liang, Michael I. Jordan, and Dan Klein. Learning dependency-baseee compositional semantics. Computational Linguistics, 2011.
Jiayuan Mao, Chuang Gan, Pushmeet Kohli, Joshua Tenenbaum, and Jiajun Wu. The neuro- symbolic concept learner: Interpreting scenes words and sentences from natural supervision. In ICLR, 2019.
Arvind Neelakantan, Quoc V Le, Martin Abadi, Andrew McCallum, and Dario Amodei. Learning natural language interface with neural programmer. In ICLR, 2016.
Panupong Pasupat and Percy Liang. Compositional semantic parsing on semi-structured tables. In ACL, 2015.
Siva Reddy, Mirella Lapata, and Mark Steedman. Large-scale semantic parsing without question- answer pairs. TACL, 2014.
Alon Talmor, Jonathan Herzig, Nicholas Lourie, and Jonathan Berant. Commonsenseqa: question answering challenge targeting commonsense knowledge. In NAACL-HLT, 2018.
Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William W. Cohen, Ruslan Salakhutdinov, and Christopher D. Manning. Hotpotqa: dataset for diverse, explainable multi-hop question answering. In EMNLP, 2018.
John M Zelle and Raymond J Mooney. Learning to parse database queries using inductive logic programming. Proceedings of the national conference on artificial intelligence, 1996.
Luke $ Zettlemoyer and Michael Collins. Learning to map sentences logical form: structured classification with probabilistic categorial grammars. In UAI '05, 2005.
Zhuosheng Zhang, Yuwei Wu, Junru Zhou, Sufeng Duan, and Hai Zhao. Sg-net: Syntax-guided machine reading comprehension. arXiv preprint arXiv: 1908. /47 2019.
11

---

Published as a conference paper at ICLR 2020
A
APPENDIX
A.1 QUESTION AND PARAGRAPH ENCODER
Our model represents the question y as Q E RRad and paragraph p as P E mx using contextual- ized token embeddings. These embeddings are either produced using a multi-layer GRU network that is trained from scratch, or a pr-trained BERT model that is fine-tuned during training.
We use 2-layer, 64-dimensional (d 128, effectively), bi-directional GRU. The same GRU is used for both, the question and the paragraph. The token embeddings input to the contextual encoder are a concatenation of 100-d pre-trained GloVe embeddings. and 200-d embeddings output from CNN over the token's 's characters. The CNN uses filters of size=5 and character embeddings of 64-d. The pre-trained glove embeddings are fixed, but the character embeddings and the parameters for the CNN are jointly learned with the rest of the model.
GRU:
The input to the BERT model is the concatenation of the question and paragraph in the following format: [CLS] Question [S3P] Context [SEP]] The question and context to- kens input to the BERT model are sub-words extracted by using BERT's tokenizer. We separate the question and context representation from the output of BERT Q and P. respectively. We use
BERT:
-bert-basc-une basceuncassed model for all out experiments.
A.2 QUESTION PARSER DECODER
The decoder for question parsing is single-layer, 100-dimensional, LSTM. For each module, a 100-dimensional embedding to present it as an action in the decoder's 's input/output vocabulary. The attention is computed as a dot-product between the decoder hidden-state and the encoders hidden states which is normalized using the softmax operation.
As the memory-state for the zero-eth time-step in the decoder, we use the last hidden-staal state of the question encoder GRU, or the [CLS] embedding for the BERT-based model.
We use a beam-size of for the approximate maximum marginal likelihood objective. Optmization is performed using the Adam algorithm with learning rate of 0.001 or using BERT's optimizer with learning rate of 5.
A.3 NUMBER AND DATE PARSING
We pre-process the paragraphs to extract the numbers and dates in them. For numbers, use simple strategy where all tokens in the paragraph that can be parsed as a number are extracted. For example, 200 in "200 women". The total number of number-tokens in the paragraph is denoted by Nikenns We do not normalize numbers based on their units and leave it for future work.
To extract dates from the paragraph, we run the spaCy-NERT and collect all "DATE" mentions. To normalize the date mentions we use an off-the-shelf date-parser For example, date mention "19th November, 1961" 1d would be normalized (19, 11, 1961) (day, month, year). The total number of date-tokens is denoted by Drokens
A.4 PRE-TRAINING COUNT MODULE
As mentioned in the paper, training the count module challenging and found that pre-training the parameters of the count module helps.
To re-iterate, the module gets as input a paragraph attention P E T The module first scales the attention using the values [1, 2, 5, 10] to convert it into matrix Pseated E m A bidirectional-GRR then represents each token attention as hidden vector ht, A single-layer feed-forward network maps this representation to soft 0/1 score to indicate the presence of span surrounding it. These scores
'httss//sspcy..i/// http://gghhaaccmmmmmmmpnnnuuaaae
12

---

Published as a conference paper at ICLR 2020
summed to compute count value, Cp.
( (F(coun(GRU((scax))) R
coorss
Σ count,cores
R
We generate synthetic data pre-train this module; each instance is a normalized-attentinn vector R and count value [0,9] 9). This is generated by sampling uniformly between 200 600, then sampling a count value y uniformly in [0, 9]. We then sample y span-lengths between 15 positions in the attention vector æ. For all these spans in
and also sample y non-overlapping we put value of 1.0 and zeros everywhere else. We then add O-mean, 0.01-variance gaussian noise to all elements in and normalize to make the normalized attention vector that can be input the count module.
We train the parameters of the count module using these generated instances using L2-llss between the true count value and the predicted Co-
The CoUntGRU in the count module (spanGRU span module) is 2-layer, bi-directional GRU with input-dim = 4 and output-dim = 20. The final feed-forward comprises of a single-layer to map the output of the countGRU into scalar score.
A.5
SPAN MODULE
The span module is implemented similar the count module. The input paragraph attention is first scaled using [1, 2, 5, 10], then a bidirectional GRU represents each attention as a hidden vector, and single-layer feed-forward network maps this 2 scores, for span start and end. A softmax operation on these scores gives the output probabilities.
A.6 AUXILIARY QUESTION PARSE SUPERVISION
For questions with parse supervision z", we decouple the marginal likelihood into two maximum likelihood objectives, p(z (9) and p(y*lz"). We also add loss for the decoder to attend to the tokens in the question attention supervision when predicting the relevant modules. The question attention supervision is provided
mutli-hot vector a" {0, 1) The loss against the predicted attention vector a is, Qloss = >i= I o log ag Since the predicted attention is a normalized distribution, the objective increases the sum of pog-probabillties of the tokens the supervision.
The following patterns are used to extract the question parse supervision for the training data:
1. what happened first SPANI or SPAN2?
span (compare-date--eee (find (), find ()) 1: with find attentions SPANI and SPAN2, respectively. Use compare-date-gte if second instead of first.
were there fewer SPANI or SPAN2?
span (etmppppe------- respectively. Use compare-num
find )): with find attentions SPANI and SPAN2, if more instead of fewer.
how many yards was the longest [touchdown field goal}?
find-num (find-max-num (find ) with find attention touchdown / field goal. For shortest, the find-min-net module used.
how many yards was the longest (touchdown field goal} SPAN?
find-nit (find-maxx num (filter (find ()))): with find attention touchdown / field goal and filter attention on all SPAN tokens.
how many {field goals, touchdowns, passes] were scored SPAN?
count (filter (find () ): with find attention on {field goals, touchdowns, passes} and filter attention on SPAN.
who (kicked, caught, threw, scored} SPAN?
span (relocate (filter (Eind ) with relocate attention on (kicked, caught, threw,
scored), find attention on (touchdown / field goal), and filter attention on all other tokens in the SPAN.
13

---

Published as a conference paper at ICLR 2020
Question: Which group was smaller, the 25 to 44 year olds or the 45 to 84 year olds?
Program: span (aan(ccaaae--iii mum--trid,,, find))
Answer: 45
compare-num-it passage-attenfion: In the city, the population was spread out with 12.0% under the age of 18, 55.29 from 18 to 24, 15.3% from 25 44, 10.3% from 45 to 64, and 7. 7.1% who were 65 years of age or older. The median age was 22 years. For every 100 females, there were 160.7 males. For every 100 females age 18 and over, there were 173.2 males.
number-distributionn 10.3 12 15.3 18 22 25 45 65.2 64 100 160.7 173.2
question-attention: Which group was smaller, the 25 to 44 year olds or the 45 to 64 year olds?
passage-attention: the city, the population was spread with 12.0% under age of 18, 55.2% from 24, 15.3% from 25 to 44, 10.3% from 64, and 7.1% who were years age or older. The median age was 22 years. For every 100 females, there were 160.7 males. For every females age over, there were 173.2 males.
Our model predicts the program span (compare-nune) -nmaaree--miit it (find, find) ) for the given question. We show the question atten- tions and the predicted passage attentions of the two find operations using color-coded highlights on the same question and paragraph (to space) at the bottom. The number grounding for the two paragraph attentions predicted the compare-num- in wiibber-dittributinn. Since the number associated to the passage span "45 64" is lower (10.3 vs. 15.3), the output of the compare compare-num--a num- 1t module is "45 to 64" as shown in the passage above.
Figure Example usage of num-coomare--ee
module are shown using the same colors
A.7 HEURISTIC INTERMEDIATE MODULE OUTPUT SUPERVISION
As mentioned in Section 4.3, heuristically find supervision for the output of the find- num and and-date module for a subset of questions that already contain question program supervision. These as follows:
1. how many yards was the longest/shortest (touchdown, field goal}? We find all instances touchdown/field goal in the passage and assume that the number appearing closest should be an output of the find-riod module.
2. what happened first EVENTI or EVENT2? Similar above, we perform fuzzy matching find the instance EVENTI and EVENT2 the paragraph and assume that the closest dates should be the output of the two find-date e module calls made by the compare-dates
module in the gold program.
3. were there fewer SPANI or SPAN2?
This is exactly the same previous for find-num module calls by compare-aun-1t.
A.8 EXAMPLE PREDICTIONS
In Figures 3
we show predictions by our model that shows the learned execution of various modules defined the paper.
14

---

Published as a conference paper at ICLR 2020
Question: Which event happened first, when king Chulalongkorn enacted two decrees banning the capture and sale of Kha slaves or when Siam abolished the tributes collected from vassal states?
Program: span(compart -eeedddtttetttttee
find))
Answer: banning the capture and sale of Kha slaves
date-distribution: 1874/-1/-1 1884/-1/-1 1868/-1/-1 1883/-1/-1 1899/-1/-1
question-ateention Which event happened first, when king Chulalongkorn enacted two decrees banning the capture and sale of Kha slaves when Siam abolished the tributes collected from vassal states?
-assage-attentin: Before the Monthon reforms initiated king Chulalongkorn, Slamese territories were divided into three categories: Inner Provinces forming the core of the kingdom, Outer Provinces that were adjacent to the inner provinces and tributary states located on the border regions. The area of southem Lacs that came under Siamese control following the Lao rebellion and destruction of Vientiane belonged to the later category. maintaining relative autonomy. Lao nobles who had received the approval of the Slamese king exercised authority on the Lao population as well as the Alak and Laven- speaking tribesmen. Larger tribal groups often raided weaker tribes abducting people and selling them into slavery at the trading hub of Champasak, while themselves falling prey to Khmer, Lao and Slamese slavers. From Champasak the slaves large profits for the slavers and various middlemen. In 1874 and 1884, king Chulalongkorn enacted two decrees banning the capture and sale of Kha slaves while also freeing all slaves born after 1868. Those abolitionist policies had an immediate effect on slave trading communities. In 1883, France attempted to expand its control in Southeast Asia by claiming that the Treaty of Hué extended into all Vietnamese vassal states. French troops gradually occupied the Kontum Plateau and pushed the Siamese from Laos following the Sranco-Siamsse War. A new buffer zone was thus created on the west bank of Mekong, as the area lacked the presence of the Siamese military local outlaws flocked the newly created safe haven. 1899, Siam abolished the tributes collected from vassal states, replacing them with a new tax collected from able bodied men, undermining authority of officials.
transported to Phnom-Penh and Bangkok, thus creating
Figure 4:
Example usage of centte-coppre--ee -1: Similar to Fig, 3 tions, the output passage attentions of the find module, and the date grounding predicted in the compare- satre-date-lle 1t module in color-coded highlights. The passage span predicted the answer the one associated to a lower-valued date.
we show the question atten-
Question: How many rushing touchdowns were scored in the game?
Program: count(find)
Answer: 5
count-distribution: 2
question-attention:: How many rushing touchdowns were scored in the game?
passage-attention: Coming off their Thursday night home over the Packers, Cowboys flew to Ford Field for Week 14 interconference duel with the Detroit Lions. In the first quarter, Dallas trailed early Lions RB T.J. Duckett getting 32-yard TD run, along with kicker Jason Hanson getting 19- yard field goal. In the second quarter, the Cowboys got the board with RB Marion Barber getting 20-yard TD Detroit would answer with Hanson kicking 36-yard field goal, while RB Kevin Jones getting 2-yard TD run. The Cowboys ended the half with QB Tony Romo completing
8-yard TD pass to Barber. In the third quarter, the Lions replied with Jones getting 3-yard TD run for the only score of the period. In the fourth quarter, Dallas came back and took the lead with Barber getting 1- yard TD run and Romo completing 16-yard TD pass to TE Jason Witten. With the win, the Cowboys improved to 12-1 and clinched the NFC East crown for the first time since 1998.
Figure 5: Example usage of count: For the predicted program count (find), the f find module predicts all "rushing touchdown" spans from the passage and the count module counts their number and represents its output as a distribution over possible count values. In this example, the predicted distribution as a mode of "5".
15

---

Published as a conference paper at ICLR 2020
Question: many yards the longest field goal?
Program: find-num monddnnu(ffnddmmmm caaiindd--x---- -aa--mmmx--mm------ find))
Answer: 39
max-num-distriuuiion: 2 10 11 12 29 31 38 39 40 1986 1990
imput-num-distributii::: 2 10 11 12 13 29 31 38 39 40 1986 1990
find-max-num passage-attention: Coming off their road over the Cardinals, the Glants FedEx Field for Week 13 NFC East rematch with Washington Redskins. the first quarter, New York scored first QB Eli Manning completed 40-yard TD pass WR Amani Toomer, along with kicker John Carney getting 31-yard field goal. In the second quarter, the Giants increased their lead Carney got 38 field goal. The Redskins would close out the half with WR Devin Thomas getting 29-yard TD run. In the third quarter, New York began pull away RB Brandon Jacobs got 1-yard TD run. the fourth quarter, the Giants sealed the deal with Carney connecting on 39-yard field goal. With the season-sweep, the Giants improved to 11-1, exceeding the 1986 and 1990 teams (both started 10-2 and eventually won the Super Bowl) for the best 12-game record in franchise history.
question-attention: How many yards was the longest field goal?
passage-attention: Coming off their road win over the Cardinals, the Giants flew to FedEx Field for Week 13 NFC East rematch with the Washington Redskins. In the first quarter, New York scored first as QB Eli Manning completed 40-yard TD pass to WR Amani Toomer, along with kicker John Carney getting 31 31-yard field goal. In the second quarter, the Giants increased their lead as Carney got 38- yard field goal. The Redskins would close out the half with WR Devin Thomas getting 29-yard TD run. in the third quarter, New York began to pull away RB Brandon Jacobs got 1-yard TD run. In the fourth quarter, the Giants sealed the deal with Camey connecting season-sweep, the Glants improved to 11-1, exceeding the 1986 and 1990 teams (both started 10-2 and eventually the Super Bowl) for the best 12-game record in franchise history.
39-yard field goal. With the
Figure 6: Example usage of f Find-max-num. For the predicted program, the model first grounds the question span "field goal" to all field goals in the passage, shown as attention in the bottom half. The find-max- num first finds the number associated with the input passage spans (shown as input-num-diin -iuum-distribution) then finds the distribution over max value (shown as max momx-num-ee --nn-nnu-itttit -in-distribution). and finally outputs the passage span associated with this max value (shown in the passage attention on top). The f iin-nuum module finally extracts the number associated with this passage attention as the answer.
16

---

Published as a conference paper at ICLR 2020
Question:
Which player scored the longest rushing TD?
Program: Span(relocate( (errrreeeooaeefnnneeeoo (finddmax--iadddde (eix(fimmmnn)))
Answer: T.J. Duckett
reloate-quusston--attrtion Which player scored the longest rushing TD?
relocate / find-max-num passage-attention Coming off their Thursday night home win over the Packers, the Cowboys flew to Ford Field for a Week 14 interconference duel with the Detroit Lions. In the first quarter, Dallas trailed early as Lions RB T.J. Duckett getting 32-yard TD run, along with kicker Jason Hanson getting a 19-yard field goal. In the second quarter, the Cowboys got on the board with RB Marion Barber getting a 20-yard TD run. Detroit would answer with Hanson kicking a 36-yard field goal, while RB Kevin Jones getting a 2-yard TD run. The Cowboys ended the half with QB Tony Romo completing an 8-yard TD pass to Barber. In the third quarter, the Lions replied with Jones getting 3- yard TD run for the only score the period. the fourth quarter, Dallas came back with Barber getting a 1-yard run Romo completing 16-yard pass TE the win, the Cowboys improved to 12-1 clinched NFC East crown for the first time since 1998.
the lead Witten. With
max-num-distribution: 1 2 3 8 12 14 16 19 20 32 36 1998
input-num-distriuutin: 1 2 3 8 12 14 16 19 20 32 36 1998
question-attention: Which player scored the longest rushing TD?
Coming off their Thursday night home win over the Packers, the Cowboys flew to Ford Field for a Week 14 interconference duel with the Detroit Lions. In the first quarter, Dallas trailed early as Lions RB T.J. Duckett getting 32-yard TD run, along with kicker Jason Hanson getting a 19- yard field goal. In the second quarter, the Cowboys got on the board with RB Marion Barber getting 20-yard TD run. Detroit would answer with Hanson kicking 36-yard field goal, while RB Kevin Jones getting 2-yard TD run. The Cowboys ended the half with QB Tony Romo completing an 8-yard TD pass to Barber. In the third quarter, the Lions replied with Jones getting 3-yard TD run for the only score of the period. In the fourth quarter, Dallas came back and took the lead with Barber getting 1- yard TD run and Romo completing 16-yard TD pass to TE Jason Witten. With the win, the Cowboys improved to 12-1 and clinched the NFC East crown for the first time since 1998.
passage attention::::
Figure 7: Example usage of relocate: Similar to the sub-program find-max-num (find) in Fig. [6] the find module finds all mentions of "rushing TD" in the passage and find-max-num selects the one associated to the largest number. This the span "getting a 32-yard TD run" in passage above. The question attention predicted for the relocate module and its output passage attention also shown in the passage above.
17


Reviews:
[{"review_id": "Hyl4DhoAtS", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Paper Claims\n\nThe paper offers a new deep learning approach to symbolic reasoning over text. They propose using Neural Module Networks to perform explicit reasoning steps that are nevertheless differentiable. The process is separated into a semantic parsing of the question, and a resolution using MNMs. Auxiliary tasks improve performance and enable using a BERT pretrained model as a seed. The proposed model's performance surpasses previous SOTA on several question types. \n\nDecision\n\nI'm in favor of accepting this paper because it tackles an extremely challenging and important problem in a novel and successful way. Reasoning has progressed more slowly than other NLP domains, and answering complex multi-reasoning-step questions is a good way to tackle the core of the problem. Regarding the approach, I find the mix of explicit reasoning steps, deep learning modeling, and even heuristics for some of the data preparation, powerful and effective. I see this as a useful step to advance the body of work in this space -- despite not being the desired end-result, it is nevertheless very instructive of what might work for at least some parts of the larger, AI-complete reasoning problem. Also, the paper is clear, well-written, and well-motivated.\n\nFurther details on Decision\n\nI'm more than satisfied with the breadth of question types tackled here, and correspondingly the variety of modules. There's extensive, valuable work in designing these modules and making them work together. As the authors point out, it appears that other types of questions will require more intricate modules (or some other means), and I suspect that predetermined modules will not be what generalizes eventually. Nevertheless, the NMN approach taken here can be a stepping stone to further understanding how to tackle symbolic reasoning in a deep neural network. It will be instructive in designing a more ambitious, generalizable model. \n\nThe auxiliary supervision tasks appear to be essential to obtaining the results, most notably the unsupervised loss for IE. I think this area has room for further improvement, but what is achieved in the paper is sufficient for publication. In particular, the writing of heuristics is a very specific solution targeting specific types of question and this will not scale to the full scope of natural language questions, and much less to all reasoning. Discussion of how to expand on them, scale them (automatic discovery, some other means?), etc. would be very welcome, as it is the main weakness of the paper.\n\nI also think the methodology is sound and the results are obtained in a reasonable and mostly reproducible way.\n\nThis is truly great work that deserves to be published, discussed, and expanded upon.\n\n\n", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}, {"review_id": "BkgvQQW0tH", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper proposes a model and a training framework for question answering which requires compositional reasoning over  the input text, by building executable neural modules and training based on additional auxiliary supervision signals.\n\nI really like this paper and the approach taken: tackling complex QA tasks is an important topic and current state-of-the-art methods rely heavily on lexical similarities as mentioned in the paper. I think learning differentiable programs is a good direction to address this space.\n\nHowever in IMHO, the paper as it stands is premature for publication, the primary reason being the lack of strong experimental evidence that where the strength of this approach is compared to other methods compared. To be specific, the results in Table 2 are very close between MTMSN and the BERT-based model proposed and it's not clear if the difference is because (1) the model is generally better; (2) this is a subset of the dataset that this model performs better; (3) this is because of the additional supervision signals provided (e.g. the results of Fig 2a without the aux-sup is almost the same as MTMSN) and if we provided similar auxiliary supervision for other models they would equally do well; (4) due to lack of reporting variance and error-bars across runs we see a small increase which may not be significant; ...\n\nAgain, the paper is very interesting, but I don't think it's clear and thorough to experimentally prove that the overall approach is working better.\n", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I carefully checked the derivations and theory."}, "extra_text": {}}, {"review_id": "B1xkXk7B5H", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This works applies neural module network to reading comprehension that requires symbolic reasoning. There are two main contributions: (1) the authors designed a set of differentiable neural modules for different operations (for example, arithmetics, sorting, and counting) that is required to perform reasoning over a paragraph of text. These modules can be compositionally combined to perform complex reasoning. And the parameters of each module (which can be viewed as executor of each operation) are learned jointly with the parser that generates thee program composed of those modules. (2) To overcome the challenge of weak supervision, the authors proposed to use auxiliary loss (information extraction loss, parser supervision, intermediate output supervision). The model is evaluated on a subset of DROP, and outperforms the state-of-the-art models. Ablation studies supported the importance of the auxiliary losses.\n\nStrength:\n\n(1) The problem of applying symbolic reasoning over text is important and very challenging. This work has explored a promising direction that applies NMN, which achieved good results in VQA, to QA tasks that requires reasoning, specifically, a subset of the DROP dataset.\n\n(2) The result, although preliminary, seems promising. The design of the modules seems intuitive and the introduction of auxiliary tasks to alleviate the problem of weak supervision is well motivated and works reasonably well. \n\nI am leaning towards rejection because:\n\n(1) The main concern is that the paper, in its current form, seems incomplete. It is understandable that the type of datasets that requires reasoning is not very common nowadays, so only DROP is used for evaluation. However, the current evaluation is only on a subset of DROP, which seems unsatisfying.  \n\nThe paper argues that \"Our model possesses diverse but limited reasoning capability; hence, we try to automatically extract questions in the scope of our model based on their first n-gram\". However, results on the full dataset seems necessary for evaluating the potential of NMN approach over text. Even if the result is negative, it is still good to know the cause of the failure. For example, does the difficulty come from unstable training or does it come from insufficient coverage of the modules. \n\n(2) There are several modules introduced in the paper, but there isn't much analysis of them during the experiments. For example, what are some good and bad samples that uses each type of operations. \n\n(3) Since the modules are learned jointly with the parser, it is good to check whether the learned modules are indeed performing the intended operation instead of just adding more capacity to the model. For example, it might help to show a few examples that demonstrates the \"compare-num-lt\" is actually performing the comparisons. This can support the interpretability claim of the proposed model.  \n\n\nMinor issues:\n\nThe complexities of some modules seem large. For example, \"compare-num-lt\" needs to enumerate all the pairs of numbers, which is quadratic. And the complexity of \"find-max-num\" depends on the choice of n, which could be large (although it is chosen to be 3 in this work). \n\nIt is stated that \"Our model performs significantly better than the baseline with less training data, showing the efficacy of explicitly modeling compositionality.\" However, the comparison with MTMSN using less training data seems a bit unfair since the proposed model is given more supervision (question parse supervision and intermediate module output supervision). Maybe a better argument is that by explicitly modeling compositionality, it is easier to add such extra supervisions than black box models like MTMSN. \n\nFor \"count\", why is the attention scaled using values [1, 2, 5, 10] first?\n\nIn summary, I do like the main idea and the paper has merits, but it requires more evaluation and analysis to be accepted. I am willing to increase my score if more contents are added and I look forward to seeing it in a more complete form.\n\n===================================\n\nUpdate after author response:\n\nThanks for the clarification and adding the content, I have updated my score accordingly. However, I still believe the impact of this paper will be much larger if the evaluation can be more complete, e.g., evaluating over the full DROP dataset or even include some other datasets. In the current form, it looks borderline. \n\nSelecting a subset (~22.7% of DROP dataset) based on the design of the proposed model (\"heuristically chosen based on their first n-gram such that they are covered by our designed modules\"), and compare to other models, which can actually handle a broader set of questions, only on the selected subset seems incomplete and raises concerns about how generally applicable the proposed model is. For example, since the proposed model is handling some types of questions better, it would be good to show that it can be combined with other models to get a better overall result. ", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly.", "review_assessment:_checking_correctness_of_derivations_and_theory": "N/A"}, "extra_text": {}}]

Author rebuttal:

Meta-review: This work extends the previously introduced NMN for VQA for handling reasoning over text using symbolic reasoning components that can perform counting, sorting etc and can be compositionally combined. Moreover, to successfully train the model, the authors introduce a simple unsupervised auxiliary loss for training the IE components as well heuristically incorporating inductive biases in the behaviour on couple of components. All reviews agreed that this is a challenging topic and an interesting approach to symbolic reasoning over text. At the same time, reviewers did point that experiments are borderline thin, since the authors start with DROP and drop questions that are not particularly suited for symbolic reasoning, resulting in a substantially smaller dataset. Despite the fact that the experiments could probably be stronger, I’m recommending acceptance cause this topic is very interesting and this is a good paper to raise discussions at ICLR,
Decision: Accept (Poster)