SlowGuess/NAACL_2021_OCR huggingface dataset:
Tushar Khot Daniel Khashabi Kyle Richardson Peter Clark Ashish Sabharwal
Allen Institute for AI, Seattle, WA, U.S.A.
{tushark,danielk,kyler,peterc,ashishs}  $@$  allenai.org
We propose a general framework called Text Modular Networks (TMNs) for building interpretable systems that learn to solve complex tasks by decomposing them into simpler ones solvable by existing models. To ensure solvability of simpler tasks, TMNs learn the textual input-output behavior (i.e., language) of existing models through their datasets. This differs from prior decomposition-based approaches which, besides being designed specifically for each complex task, produce decompositions independent of existing sub-models. Specifically, we focus on Question Answering (QA) and show how to train a next-question generator to sequentially produce sub-questions targeting appropriate sub-models, without additional human annotation. These sub-questions and answers provide a faithful natural language explanation of the model's reasoning. We use this framework to build MODULARQA, $^{1}$  a system that can answer multi-hop reasoning questions by decomposing them into sub-questions answerable by a neural factoid single-span QA model and a symbolic calculator. Our experiments show that MODULARQA is more versatile than existing explainable systems for DROP and HotpotQA datasets, is more robust than state-of-the-art blackbox (uninterpretable) systems, and generates more understandable and trustworthy explanations compared to prior work.
An intuitive way to solve more complex tasks, such as multi-hop question-answering (Yang et al., 2018; Khashabi et al., 2018; Khot et al., 2020) and numerical reasoning (Dua et al., 2019), would be to decompose them into already solved simpler problems, e.g., single-fact QA (Rajpurkar et al., 2016). Besides allowing reuse of existing simpler models, this approach would yield an interpretable system that provides a faithful explanation (Jacovi and
Figure 1: MODULARQA learns to ask sub-questions to existing simple QA models, including a symbolic calculator, to answer a given complex question. Notably, the approach does not rely on annotated decompositions. Despite this, the system learned to add "start to take a dip" in the DROP dataset question.
Goldberg, 2020) of its reasoning as a composition of simpler sub-tasks, as shown in Fig. 1. Motivated by this, we ask the following question:
Given a set of existing QA models, can one leverage them to answer complex questions by communicating with these existing models?
We propose a general framework, Text Modular Networks (TMNs), that answers this question by learning to decompose complex questions (of any form) into sub-questions that are answerable by existing QA models—symbolic or neural (henceforth referred to as sub-models). Unlike previous approaches (Talmor and Berant, 2018; Min et al., 2019a), the decompositions are not based on splits of the complex questions and aren't built independent of the sub-model. Instead, our framework learns to generate sub-questions in the scope
$^{2}$ TMNs, in fact, treat sub-models as blackboxes, and can thus use any model or function as a module.
$^{1}$ https://github.com/allenai/modularqa
Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1264-1279 June 6-11, 2021. ©2021 Association for Computational Linguistics
of existing models. For instance, the second subquestion in the DROP dataset example in Fig. 1 requires the introduction of a new phrase, "start to take a dip", which is beyond the scope of standard decomposition approaches. Additionally, the final sub-question targets a symbolic calculator, which operates over a different input language.
The core of our TMN framework is a next-question generator that sequentially produces the next sub-question to ask as well as an appropriate sub-model for answering it. The resulting sequence of sub-questions and their answers provides a human-interpretable description of the model's neuro-symbolic reasoning (McCarthy, 1988; Smolensky, 1988), as illustrated in Fig. 1. Notably, TMNs learn to produce these decompositions using only distant supervision, without the need for any explicit human annotation.
One of our key insights is that the capabilities of existing sub-models can be captured by training a text-to-text system to generate the questions in the sub-model's training dataset (e.g., SQuAD), given appropriate hints. In our case, we train a BART model (Lewis et al., 2020) to generate questions given the context, answer, and preferred vocabulary as hints. We then use these sub-task question models to generate sub-questions (and identify appropriate sub-models) that could lead to the likely intermediate answers extracted for each step of the complex question ("Raymond S." and "American" in the HotpotQA example in Fig. 1). The resulting sub-questions, by virtue of our training, are in the language (i.e., within-scope) of the corresponding sub-models. These sub-question sequences can now be used to train the next-question generator to sequentially produce the next sub-question. We use this trained generator, along with existing QA models, to answer complex questions, without the need for any intermediate answers.
We use the TMN framework to develop MODULARQA, a modular system that explains its reasoning in natural language, by decomposing complex questions into those answerable by two sub-models: a neural factoid single-span QA model and a symbolic calculator. MODULARQA's implementation<sup>1</sup> covers multi-hop questions that can be answered using these two sub-models via five classes of reasoning found in existing QA datasets: composition, conjunction, comparison, difference, and complementation.<sup>3</sup>
We evaluate MODULARQA on questions from two datasets, DROP (Dua et al., 2019) and HotpotQA (Yang et al., 2018), resulting in the first cross-dataset decomposition-based interpretable QA system. Despite its interpretability and versatility, MODULARQA scores only  $3.7\%$  F1 lower than NumNet+V2 (Ran et al., 2019), a state-of-the-art blackbox model designed for DROP. MODULARQA even outperforms this blackbox model by  $2\%$  F1 in a limited data setting and demonstrates higher  $(+7\%)$  F1 robustness (Gardner et al., 2020). MODULARQA is competitive with and can even outperform task-specific Neural Module Networks (Gupta et al., 2020; Jiang and Bansal, 2019) while producing textual explanations. Further, our human evaluation against a split-point based decomposition model trained on decomposition annotation (Min et al., 2019b) for HotpotQA finds our explanations to be more trustworthy, understandable, and preferable in  $67\% -78\%$  of the cases.
Contributions. (1) Text Modular Networks (TMNs), a general framework that leverages existing simpler models—neural and symbolic—as blackboxes for answering complex questions. (2) MODULARQA, $^{1}$  an interpretable system that learns to automatically decompose multi-hop and discrete reasoning questions. (3) Experiments on DROP and HotpotQA demonstrating MODULARQA's cross-dataset versatility, robustness, sample efficiency and ability to explain its reasoning in natural language.
Many early QA systems were designed as a combination of distinct modules, often composing outputs of lower-level language tasks to solve higher-level tasks (Moldovan et al., 2000; Harabagiu and Hickl, 2006). However, much of this prior work is limited to pre-determined composition structures (Berant et al., 2013; Seo et al., 2015; Neelakantan et al., 2017; Roy and Roth, 2018).
Various modular network architectures have been proposed to exploit compositionality (Rosenbaum et al., 2018; Kirsch et al., 2018). The closest models to our work are based on neural module networks (NMN) (Andreas et al., 2016) which compose task-specific simple neural modules. We
to as 'bridge' questions. Complementation refers to questions such as 'What percentage of X is not Y?' MODULARQA can be easily extended to other reasoning types by defining the corresponding hints (§4.3).
<sup>3</sup>Composition and conjunction questions are also referred
compare against formulations of NMNs for HotpotQA (Jiang and Bansal, 2019) and DROP (Gupta et al., 2020), both of which target only one dataset and do not reuse existing QA systems. Moreover, they provide attention-based explanations whose interpretability is unclear (Serrano and Smith, 2019; Brunner et al., 2020; Wegreffe and Pinter, 2019).
Question decomposition has been pursued before for ComplexWebQuestions (Talmor and Berant, 2018) and HotpotQA. Both approaches (Talmor and Berant, 2018; Min et al., 2019b) focus on directly training a model to produce sub-questions using question spans—an approach not suitable for DROP questions (as illustrated in Fig. 1). Our next-question generator overcomes this limitation by generating free-form sub-questions in the language of existing models. Perez et al. (2020) also use a text-to-text model to generate sub-questions for HotpotQA. However, they generate simpler questions without capturing the requisite reasoning, and hence use them mainly for evidence retrieval.
BREAK (Wolfson et al., 2020) follows an alternative paradigm of collecting full question decomposition meaning representations (QDMR) annotations. While this can be effective, it relies on costly human annotation that may not generalize to domains with new decomposition operations. Its decompositions are generated in a model-agnostic way and still need QA systems to answer the subquestions, e.g., high-level QDMR questions such as "Which is earlier?" and "Which is longer?" would need special systems that can map these to symbolic comparisons. In contrast, TMNs start with pre-determined models and learn to generate decompositions in their language.
While many multi-hop QA models exist for HotpotQA and DROP, these are often equally complex models (Tu et al., 2020; Fang et al., 2020; Ran et al., 2019) focusing on just one of these datasets. Only on HotpotQA, where supporting sentences are annotated, can these models also produce post-hoc explanations, but these explanations are often not faithful and shown to be gameable (Trivedi et al., 2020). TMNs are able to produce explanations for multiple datasets without needing such annotations, making it more generalizable to future datasets.
3 Text Modular Networks
TMNs are a family of architectures consisting of modules that communicate through language learned from these modules, to accomplish a cer
Figure 2: A sample inference on a DROP question using TMNs with the text-to-text interactions between the next-question generator  $\mathbb{D}$  and existing QA models  $\mathbb{A}$ .
tain goal (e.g., answering a question). Figure 2 illustrates this general idea in the context of answering a DROP question. The core of our system is a next-question generator  $\mathbb{D}$ , a component in charge of generating and distributing sub-tasks among sub-models  $\mathbb{A}_s$ . The system alternates between using  $\mathbb{D}$  to produce the next question (NextGen) and using the corresponding sub-model to answer this question. Formally, solving a complex question  $qc$  is an alternating process between the following two steps:
Generate the next question  $q_{i}$  for submodel  $t_i$ :
$$
\langle t _ {i}, q _ {i} \rangle = \mathbb {D} (q c, q _ {1}, a _ {1}, \dots , q _ {i - 1}, a _ {i - 1})
$$
Find answer  $a_i$  by posing  $q_i$  to submodel  $t_i$ :
$$
a _ {i} = \mathbb {A} _ {t _ {i}} (q _ {i}, p)
$$
where  $q_{i}$  is the  $i^{th}$  generated sub-question and  $a_{i}$  is the answer produced by a sub-model  $t_i$  based on a given context paragraph  $p$ . This simple iterative process ends when  $q_{i + 1}$  equals a special end-of-sequence symbol (denoted throughout as [EOQ]) with the final output answer  $a_{i}$ .
Building a Text Modular Network. The key challenge in building a Text Modular Networks is developing the next-question generator model. Training this model requires a next-question prediction dataset where each example is a step in the iterative progression of sub-question generation. For example, the second step in Fig. 2 is:
In  $\left\{ \begin{array}{l}qc: How\text{many years did it take for the services sector to rebound?}\\ q_1:\text{In what year did the services sector rebound?}\\ a_1:2003 \end{array} \right.$    
Out  $\langle t_2,q_2\rangle = \langle SQUAD, "When did the services sector start to take a dip?"\rangle$
While it may be possible to collect task-specific datasets or design a task-specific next-question generator (Min et al., 2019b; Talmor and Berant, 2018), our goal is to build a framework that can be easily extended to new complex QA tasks reusing existing QA sub-models. To achieve this, we present a general framework to generate the next-question training dataset by: (1) Modeling the language of sub-models; (2) Building decompositions in the language of these sub-models using minimal distant supervision hints.
3.1 Modeling QA Sub-Models
To ensure the sub-questions are answerable by existing sub-models, we train a text-to-text sub-task question model on the original sub-task to generate a plausible  $q_{i}$  conditioned on hints, e.g., a BART model trained on SQuAD to generate a question given the answer. We can view this utility as characterizing the question language of the sub-model. For example, such a model trained on the SQuAD dataset would produce factoid questions—the space of questions answerable by a model trained on this dataset.
While an unconditional text generation model can also capture the space of questions, it can generate a large number of possibly valid questions, making it hard to effectively train or use such a model. Instead, we scope the problem down to conditional text generation of questions given hints  $z$ . Specifically, we use the context  $p$ , answer  $a$  and question vocabulary  $v$  as input conditions to train a question generator model  $\mathbb{G} : z \to q$  where  $z = \langle p, a, v \rangle$ . Such a generator,  $\mathbb{G}_S$ , produces the first two subquestions in the example in Fig. 4, when using  $a = 2003$  (or 2002, resp.) and  $v = \Phi(qc) = \{ "service", "sector", "year", "rebound" \}$  as hints.
3.2 Training Decompositions via Distant Supervision
To generate training decompositions for a complex question using a sub-task question model, we extract distant-supervision hints  $z$  corresponding to each reasoning step. This is akin to the distant supervision approaches used to extract logical forms in semantic parsing (Liang et al., 2013; Berant et al., 2013) and the intermediate entities in a reasoning chain (Gupta et al., 2020; Jiang and Bansal, 2019).
In our running DROP example, under the definition of  $z = \langle p, a, v \rangle$ , we would need to provide the context, answer, and question vocabulary for each reasoning step. We can derive intermediate
answers by finding the two numbers whose difference is the final answer (see Fig. 4). We can use words from the input question as vocabulary hints. $^{4}$
As shown in Fig. 4, we generate the training subquestions in the language of appropriate systems for each step  $i$  using the question generation model  $\mathbb{G}_{t_i}\colon q_i = \mathbb{G}_{t_i}(z_i)$  where  $z_{i} = \langle p_{i},a_{i},v_{i}\rangle$  and the model  $t_i$  is determined by the answer type (or can be a hint too).
Note that our framework does not depend on the specific choice of  $z$ . Our key idea is to train the sub-task question model conditioned on the same  $z$  that we can provide for the complex task. The hint  $z$  could be very general (just the context) or very specific (exact vocabulary of the question), trading off the ease of extracting hints with the quality of the generated decomposition. Similarly, these hints don't have to be  $100\%$  accurate as they are only used to build the training data and play no role during inference.
Finally, we convert the decompositions into training data for the next-question generator. For each question  $q_{i}$  generated using the sub-task question model  $\mathbb{G}_{t_i}$ , we create the training example:
$$
I n p u t: q c, q _ {1}, a _ {1}, \dots , q _ {i - 1}, a _ {i - 1}
$$
$$
\text {O u t p u t}: \langle t _ {i}, q _ {i} \rangle
$$
Training Data Generation Summary. Fig. 3 illustrates the complete process for generating the training data for the next-question generator. For each complex question, we extract a set of possible hints for each potential reasoning chain (e.g., all number pairs that lead to the final answer). For each step, we use the corresponding sub-task question models to generate potential sub-questions that lead to the expected answer. Finally we use these generated sub-question decompositions as the training data for the next-question generator model.
4 MODULARQA System
We next describe a specific instantiation of the Text Modular Network: MODULARQA – a new QA system that works across HotpotQA and DROP. To handle these datasets, we first introduce the two QA sub-models(\$4.1), the sub-task question models for these models(\$4.2), our approach to build training data (\$4.3), and the inference procedure used for question-answering(\$4.4).
As mentioned before, these are soft hints and the model can be trained to handle noise in these hints.
Figure 3: The overall flow of building the training data for the next-question generator, given a complex question.
Figure 4: An example decomposition generated for a DROP example using hints and sub-question generators  $\mathbb{G}$ .  $\phi(\mathrm{qc})$  indicates words in the input question qc.
4.1 QA Sub-Models, A
We use two QA models with broad coverage on the two datasets:
SQuAD model,  $\mathbb{A}_S$ , A RoBERTa-Large model trained on the entire SQuAD 2.0 dataset including the no-answer questions; and
Math calculator model,  $\mathbb{A}_{\mathcal{C}}$ , a symbolic Python program that can perform key operations needed for DROP and HotpotQA, namely:
diff(X, Y, Z) that computes the difference between X and Y in unit Z (days/months/years);
not(X) that computes the complement  $\%$  of X, i.e.,  $100 - X$ ;
if_thenth(X <op> Y, Z, W) that returns Z if X <op> Y is true, otherwise returns W.
4.2 Sub-task Question Models,  $\mathbb{G}$
We define two sub-task question models corresponding to each of our QA sub-models.
SQuAD Sub-task Question Model,  $\mathbb{G}_{\mathcal{S}}$ . We train a BART-Large model on the answerable subset of SQuAD 2.0 to build our sub-task question model for SQuAD. We use the gold paragraph and
answer from the dataset as the input context and answer. For the estimated question vocabulary, we select essential words<sup>5</sup> from the gold questions (referred as the function  $\Phi$ ) with additional irrelevant words sampled from other questions.<sup>6</sup>
To train the text-to-text  $\mathrm{BART}_S$  model, we use a simple concatenation of the passage, vocabulary, and answer (with markers such as "H:" and "A:" to indicate each field) as the input sequence and the question as the output sequence. While a constrained-decoding approach (Hokamp and Liu, 2017; Hu et al., 2019a) could be used here to further promote the use of the vocabulary hints, this simple approach was effective and more generally applicable to other hints in our use-case.
Once this model is trained, we use it with nucleus sampling (Holtzman et al., 2020) to generate  $k$  sub-questions,  $Q$ , and filter out those that lead an incorrect or no answer using  $\mathbb{A}_S$ :
$$
\mathbb {G} _ {\mathcal {S}} (p, a, v) = \{q \in Q \mid \text {o v e r l a p s} (\mathbb {A} _ {\mathcal {S}} (p, q), a) \}
$$
Math Sub-task Question Model,  $\mathbb{G}_{\mathcal{C}}$ . Given the symbolic nature of this solver, rather than training a neural generator, we simply generate all possible numeric questions given the context. Similar to  $\mathbb{G}_{\mathcal{S}}$ , we first generate potential questions  $Q$  and then filter down to those that lead to the expected answer using  $\mathbb{A}_{\mathcal{C}}$ :
$$
\mathbb {G} _ {\mathcal {C}} (p, a, v) = \{q \in Q \mid \mathbb {A} _ {\mathcal {C}} (p, q) = a \}
$$
4.3 Generating Training Decompositions
We broadly identify five classes of questions in HotpotQA and DROP dataset that can be answered using our two models. These question classes, how
$^5\Phi (q) =$  Non-stopword tokens with pos tags  $\in$  {NOUN, VERB, NUM, PROPN, ADJ, RB}
$^{6}$ More details in Appendix A
7Other questions require a QA model that can return multiple answers or a Boolean QA model, as discussed in §6.
Example 1: How many days passed between the Sendling Christmas Day Massacre and the Battle of Aidenbach?
$\gg$  Q: When was the Battle of Aidenbach? A: 8 January 1706 Q: When was the Sendling Christmas Massacre? A: 25 December 1705 Q: diff(8 January 1706, 25 December 1705, days) A: 14
Example 2: Which ancestral group is smaller: Irish or Italian?
» Q: How many of the group were Irish? A: 12.2 Q: How many Italian were there in the group? A: 6.1 Q: if_then(12.2 < 6.1, Irish, Italian) A: Italian
Example 3: How many percent of the national population does not live in Bangkok?
$\gg \mathrm{Q}$  : What percent of the national population lives in Bangkok? A: 12.6 Q: not(12.6) A: 87.4
HotpotQA
Example 4: 12 Years a Slave starred what British actor born 10 July 1977)
>> Q: Who stars in 12 Years a Slave? A: Chiwetel Ejiofor Q: Who is the British actor born 10 July 1977? A: Chiwetel Umeadi Ejiofor
Example 5: How many children's books has the writer of the sitcom Maid Marian and her Merry Men written?
>>Q: What writer was on Maid Marian and her Merry Men? A: Tony Robinson Q: How many children's books has Tony Robinson written? A: sixteen
Example 6: Did Holland's Magazine and Moondance both begin in 1996?
>>Q:When did Holland's Magazine begin?A:1876Q:When did Moondance begin?A:1996Q:if_then(  $1876\neq 1996$  ,no,yes)A:no
Table 1: Sample Reasoning Explanations generated by MODULARQA. Note that the system learns to generate such explanations without relying on manually designed rules such as "smaller"  $\Rightarrow x < y$ .
they are identified and how we extract hints for each question type is described next. Note that similar rules for extracting distant supervision hints have been used by prior work for DROP (Gupta et al., 2020) and HotpotQA (Jiang and Bansal, 2019) too.
1. Difference (How many days before  $X$  did  $Y$  happen?): We identify these questions based on the presence of term indicating a measurement: "how many" and terms indicating difference such as "shorter", "more", "days between", etc. Also we check for two dates or numbers in the context such that their difference (in all units) can lead to the final answer. If these conditions are satisfied, for every pair  $n_1, n_2$  where the difference (in units  $u$ ) can lead to the final answer, we generate the hints:
$$
\begin{array}{l} p _ {1} = p; a _ {1} = n _ {1}; v _ {1} = \Phi (q c) \\ p _ {2} = p; a _ {2} = n _ {2}; v _ {2} = \Phi (q c) \\ p _ {3} = \varepsilon ; a _ {3} = a; v _ {3} = [ \text {d i f f}, n _ {1}, n _ {2}, u ] \\ \end{array}
$$
where  $\varepsilon$  refers to the empty string.
2. Comparison (Which event happened before:  $X$  or  $Y$ )? We identify the two entities  $e_1$  and  $e_2$  in such questions and find dates/numbers that are mentioned in documents. For every  $n_1, n_2$  number/date mentioned close to  $e_1$  and  $e_2$  respectively, we create the hints:
$$
p _ {1} = p; a _ {1} = n _ {1}; v _ {1} = \Phi (q c) \backslash e _ {2}
$$
$$
p _ {2} = p; a _ {2} = n _ {2}; v _ {2} = \Phi (q c) \backslash e _ {1}
$$
$$
p _ {3} = \varepsilon ; a _ {3} = a; v _ {3} = [ \text {i f \_ t h e n}, n _ {1}, n _ {2}, e _ {1}, e _ {2} ]
$$
The final set of hints are for use by the calculator generator to create the questions: if_then  $(n_1 > n_2, e_1, e_2)$  and if_then  $(n_1 < n_2, e_1, e_2)$ .
3. Complementation (What percent is not  $X$ )? We identify these questions mainly based on the presence of “ $*$  not  $*$ ” in the question and a number  $n_1$  such that the  $a = 100 - n_1$ . The hints are:
$$
p _ {1} = p; a _ {1} = n _ {1}; v _ {1} = \Phi (q c)
$$
$$
p _ {2} = \varepsilon ; a _ {2} = a; v _ {2} = [ \text {n o t}, n _ {1} ]
$$