SlowGuess/ACL_2020_OCR huggingface dataset:
Jianxing Yu, Wei Liu, Shuang Qiu, Qinliang Su\*, Kai Wang, Xiaojun Quan, Jian Yin
School of Data and Computer Science, Sun Yat-sen University
Guangdong Key Laboratory of Big Data Analysis and Processing, China
{yujx26, liuw259, qiush9, suqliang}@mail.sysu.edu.cn
{wangk73@mail2,quanxj3@mail，issjyin@mail}.sysu.edu.cn
This paper focuses on generating multi-hop reasoning questions from the raw text in a low resource circumstance. Such questions have to be syntactically valid and need to logically correlate with the answers by deducing over multiple relations on several sentences in the text. Specifically, we first build a multi-hop generation model and guide it to satisfy the logical rationality by the reasoning chain extracted from a given text. Since the labeled data is limited and insufficient for training, we propose to learn the model with the help of a large scale of unlabeled data that is much easier to obtain. Such data contains rich expressive forms of the questions with structural patterns on syntax and semantics. These patterns can be estimated by the neural hidden semi-Markov model using latent variables. With latent patterns as a prior, we can regularize the generation model and produce the optimal results. Experimental results on the HotpotQA data set demonstrate the effectiveness of our model. Moreover, we apply the generated results to the task of machine reading comprehension and achieve significant performance improvements.
Question generation (QG) is a hot research topic that aims to create valid and fluent questions corresponding to the answers by fully understanding the semantics on a given text. QG is widely used in many practical scenarios: including providing practice exercises from course materials for educational purposes (Lindberg et al., 2013), initiating the dialog system by asking questions (Mostafazadeh et al., 2017), and reducing the labor cost of creating large-scale labeled samples for the QA task (Duan et al., 2017). The mainstream QG methods can be summarized into the rule-based and neural-based models. The first method often transforms the input
text into an intermediate symbolic representation, such as a parsing tree, and then convert the resulting form into a question by well-designed templates or general rules (Hussein et al., 2014). Since rules and templates are hand-crafted, the scalability and generalization of this method are limited. Respectively, the neural model usually directly maps the text to question based on neural network (Du and Cardie, 2017), which is entirely data-driven with far less labor. Such a model can be typically regarded as learning a one-to-one mapping between the text and question. The mapping is mainly used to generate simple questions with a single sentence. However, due to the lack of fine-grained modeling on the evidential relations on the text, such a method has minimal capability to form the multihop questions that require sophisticated reasoning skills. These questions have to be grammatically valid. Besides, they need to logically correlate with the answers by deducing over multiple entities and relations in several sentences and paragraphs of the given text. As shown in Fig.(1), the question asks the director of a film, where the film was shot at the Quality Cafe in Los Angeles and Todd Phillips directed it. These two relations can form a reasoning chain from question to answer by logically integrating the pieces of evidence "Los Angeles," "Quality Cafe," and "Old School" as well as the pronoun "it" distributed across  $S_{1}$  in paragraph 1 and  $S_{1}$ ,  $S_{2}$  in paragraph 2. Without capturing such a chain, it is difficult to precisely produce the multi-hop question by using "Old School" as a bridging evidence and marginal entity "Todd Phillips" as the answer.
For the task of multi-hop QG, a straightforward solution is to extract a reasoning chain from the input text. Under the guidance of the reasoning chain, we learn a neural QG model to make the result satisfy the logical correspondence with the answer. However, the neural model is data-hungry, and the scale of training data mostly limits its performance.
Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 6729-6739 July 5 - 10, 2020. ©2020 Association for Computational Linguistics
Paragraph 1:  $(S_1)$  Old School Black is a 2003 American comedy film released by DreamWorks Pictures and The Montecito Picture Company and directed by Todd Phillips.  $(S_2)$  The story ...
Paragraph 2:  $(S_{1})$  The Quality Cafe is a now-defunct diner at 1236 West 7th Street in Los Angeles, California.  $(S_{2})$  It ceased to function as a diner in late 2006 but has appeared as a location featured in a number of Hollywood films, including "Training Day", "Old School", "Se7en", "Ghost World", "Gone in 60 Seconds", "The Stepfather", "What's Love Got to Do with It", "Sex and Death 101", and ....
Paragraph 3: ...
Question: Who is the director of the 2003 film which has scenes in it filmed at the Quality Cafe in Los Angeles?
Answer: Todd Phillips
Supporting sentences:  $S_1$  in Paragraph 1;  $S_1, S_2$  in Paragraph 2.
Figure 1: Sample that requires reasoning skills.
Each training example is a triple combined with the text, answer, and question. Since labeling such a combination is labor-intensive, it is difficult to ensure that we can always obtain sufficient training data in real-world applications. We thus formalize the problem as the low-resource generation of multi-hop questions, which is less explored by existing work. This task has substantial research value since reasoning is crucial in quantifying the high-level cognitive ability of machines, and low resource is the key to promote the extensive application. In order to address the problem, we propose to utilize unlabeled data, which is usually abundant and much easier to obtain. Although such data does not combine the questions with the texts and answers, the unlabeled questions contain plentiful expressive forms with structural patterns on the syntax and semantics. These patterns can be seen as the "template" to produce the questions. Thus, we can use the patterns as the prior to regularize the QG model and obtain better results accordingly.
Motivated by the above observations, we propose a practical two-stage approach to learn a multi-hop QG model from both a small-scale labeled data and a large-size unlabeled corpus. In particular, we first exploit the neural hidden semi-Markov model (Dai et al., 2016) to parameterize the sophisticated structural patterns on the questions by latent variables. Without domain knowledge, the variables can be estimated by maximizing the likelihood of the unlabeled data. We then heuristically extract a reasoning chain from the given text and build a holistic QG model to generate a multi-hop question. The evidential relations in the reasoning chain are leveraged to guide the QG model, so as to let the generated result meet multi-hop logical correspondence with the answer. Simultaneously, we naturally incorporate the prior patterns into the QG
model. In this way, we can regularize the model and inform it to express a question reasonably. That can improve the syntactic and semantic correctness of the result. With the parameterized patterns, the whole model can be learned from the labeled and unlabeled data in an end-to-end and explainable manner. In order to better balance the supervision of the labeled data and the usage of prior patterns, we propose to optimize the model by reinforcement learning with an augmented evaluated loss. Experiments are conducted on the HotpotQA (Yang et al., 2018) data set, which contains a large number of reasoning samples with manual annotation. Evaluated results in terms of automatic metrics and human judgment show the effectiveness of our approach. Moreover, we apply our generated results to the task of machine reading comprehension. We view the results as pseudo-labeled samples to enrich the training data for the task. That can alleviate the labeled data shortage problem and boost the performance accordingly. Extensive experiments are performed to show the efficacy of our approach in this application with the help of low-resource QG.
The main contributions of this paper include,
- We dedicate to the topic of low-resource generation of multi-hop questions from the text.
- We propose a practical approach to generate multi-hop questions with a minimal amount of labeled data. The logical rationality of the results is guided by the reasoning chain extracted from the text. Besides, the results are regularized to ensure the correctness of syntax and semantics by using the prior patterns estimated from a large-size of unlabeled data.
- We show the potential of our approach in a real-world application on machine reading comprehension by using the generated results.
The rest of this paper is organized as follows. Section 2 elaborates on the proposed low-resource QG framework. Section 3 presents experimental results, while Section 4 shows the QG application and demonstrates its usefulness. Section 5 reviews related works and Section 6 concludes this paper.
In this section, we first define some notations and then present the details of the proposed QG framework, including the learning of prior patterns from the unlabeled data, and the multi-hop QG network guided by the reasoning chain and prior patterns.
6730
2.1 Notations and Problem Formulation
Let  $D_{L} = \{(B_{i},A_{i},Y_{i})\}_{i = 1}^{n}$  denote a small set of labeled data that consists of  $n$  examples on the text  $B$ , answer  $A$ , and question  $Y$ . Besides, we assume that there are a large number of unlabeled data  $D_{U} = \{Q_{j}\}_{j = 1}^{\mathbb{N}}$  available, where  $Q_{j}\in D_{U}$  shares the same characteristics with  $Y_{i}\in D_{L}$  and  $\mathbb{N} > n$ . Each text contains multiple paragraphs and sentences, involving several logically correlated entities. We aim to generate the new question  $Y^{\prime}$  and answer  $A^\prime$  given the evaluated text  $B^{\prime}$  by a QG model, where the answer  $A^{\prime}$  often is a salient entity in the text  $B^{\prime}$ . The question  $Y^{\prime}$  is produced by finding the best  $\hat{Y}$  to maximize the conditional probability in  $\hat{Y} = \arg \max_{Y^{\prime}}\prod_{t = 1}^{T}p(y_{t}|B^{\prime},A^{\prime},Y^{\prime}_{< t})$ , where  $Y^{\prime}_{< t}$  represents the outputted  $1^{th}$  to  $(t - 1)^{th}$  terms,  $y_{t}$  is the  $t^{th}$  term. The question has to be syntactically and semantically correct. Also, it needs to correspond to the answer by logically deducing over multiple evidential entities and relations scattered across the text. Since the resource in  $D_{L}$  may not be enough to support accurately learning of the  $p(\cdot)$ , we transfer the linguistic knowledge in  $D_{U}$  and combine it with  $D_{L}$  to enhance the training.
2.2 Learning Patterns from Unlabeled Data
The expressive pattern on the question can be viewed as a sequence of groups. Each group contains a set of term segments that are semantically and functionally similar. Such segmentation is not explicitly given but can be inferred from the text's semantics. It is difficult to characterize this structural pattern by simple hand-crafted rules, while we do not have extra labeled data to learn the pattern by the methods like Variational Auto-Encoder (VAE) (Kingma and Welling, 2014). In order to tackle this problem, we propose to employ the neural hidden semi-Markov model. The model parameterizes the similar segments on the input questions by probabilistic latent variables. Through unsupervised learning, these variables can be trained on the unlabeled data. That can well represent the intricate structural patterns without domain knowledge. Besides, the variables can be incorporated into the generation model naturally, which makes the results more interpretable and controllable.
Given a question  $Q$  with a sequence of terms  $\{q_t\}_{t=1}^T$ , we model its segmentation by two variables, including a deterministic state variable  $z_t \in \{1, \dots, K\}$  that indicates the segment to which the  $t^{th}$  term belongs, and a length variable  $l_t \in$
$\{1,\dots ,L\}$  , which specifies the length of the current segment. We assume the question is generated based on a joint distribution as Eq.(1) by multi-step emissions, where  $\mathrm{i}(\cdot)$  is the index function; the index on  $t^{th}$  term is  $\mathrm{i}(t) = \sum_{j = 1}^{t}l_{j}$  with  $\mathrm{i}(0) = 0$  and  $\mathrm{i}(T^{\prime}) = T$  .  $q_{\mathrm{i}(t - 1) + 1:\mathrm{i}(t)}$  is the sequence of terms  $(q_{\mathrm{i}(t - 1) + 1},\dots ,q_{\mathrm{i}(t)})$  . That is, we first produce a segment based on the latent state  $z_{t}$  , and then emits term with a length of  $l_{t}$  on that segment.
$$
\prod_ {t = 0} ^ {T ^ {\prime} - 1} p \left(z _ {t + 1}, l _ {t + 1} \mid z _ {t}, l _ {t}\right) \prod_ {t = 1} ^ {T ^ {\prime}} p \left(q _ {\mathrm {i} (t - 1) + 1: \mathrm {i} (t)} \mid z _ {t}, l _ {t}\right) \tag {1}
$$
$p(z_{t + 1},l_{t + 1}|z_t,l_t)$  is the transition distribution, where the  $(t + 1)^{th}$  latent state and length are conditioned on their previous ones. Since the length mainly depends on the segment, we can further factorize the distribution as  $p(l_{t + 1}|z_{t + 1})\times p(z_{t + 1}|z_t)$ .  $p(l_{t + 1}|z_{t + 1})$  is the length distribution, and we fix it to be uniform up to a maximum length  $L$ . In this way, the model can be encouraged to bring together the functionally similar emissions of different lengths.  $p(z_{t + 1}|z_t)$  is the state distribution, which can be viewed as a  $K\times K$  matrix, where each row sums to 1. We define this matrix to be Eq.(2), where  $e_o,e_j,e_k\in \mathbb{R}^d$  are the embeddings of the state  $o,j,k$  respectively, and  $b_{o,j},b_{o,k}$  are the scalar bias terms. Since the adjacent states play different syntactic or semantic roles in the expressive patterns, we set  $b_{o,o}$  as negative infinity to disable self-transition. We apply a row-wise softmax to the resulting matrix to obtain the desired probabilities.
$$
p \left(z _ {t + 1} = j \mid z _ {t} = o\right) = \frac {\exp \left(e _ {j} ^ {\top} e _ {o} + b _ {o , j}\right)}{\sum_ {k = 1} ^ {K} \exp \left(e _ {k} ^ {\top} e _ {o} + b _ {o , k}\right)} \tag {2}
$$
$p(q_{\mathrm{i}(t - 1) + 1:\mathrm{i}(t)}|z_t,l_t)$  is the term emission distribution conditioned on a latent state and a length. Based on the Markov process, the distribution can be written as a product over the probabilities of all the question terms, as Eq.(3). In order to compute the term probability, we leverage a neural decoder like the Gated Recurrent Unit (GRU) (Cho et al., 2014). We first formulate the hidden vector  $h_t^j$  for yielding  $j^{th}$  term as  $h_t^j = GRU(h_t^{j - 1},[e_{zt};e_{q_{\mathrm{i}(t - 1) + j - 1}}])$  where  $[\cdot ;\cdot ]$  is a concatenation operator,  $e_{q_{\mathrm{i}(t - 1) + j - 1}}$  and  $e_{zt}$  are the embedding of the term and corresponding segment, respectively. By attending over the given question using  $h_t^j$  , we can produce a context vector  $v_{t}^{j}$  ,as  $g_{z_t}\odot h_t^j$  ,where  $\odot$  refers to
the element-wise multiplication,  $g_{z_t}$  is a gate for the latent state  $z_t$ , and there are  $K$  gate vectors as trainable parameters. We then pass the vector  $v_t^j$  through a softmax layer to obtain the desired distribution as  $p(q_{\mathrm{i}(t-1)+j}|q_{\mathrm{i}(t-1)+j-1}, z_t, l_t) = \text{softmax}(\mathrm{W}_q v_t^j + b_q)$ , where  $\mathrm{W}_q$  and  $b_q$  are the trainable parameters.
$$
\begin{array}{l} p \left(q _ {\mathrm {i} (t - 1) + 1: \mathrm {i} (t)} \mid z _ {t}, l _ {t}\right) = p \left(q _ {\mathrm {i} (t - 1) + 1} \mid z _ {t}, l _ {t}\right) \tag {3} \\ \times \prod_ {j = 2} ^ {l _ {t}} p (q _ {\mathrm {i} (t - 1) + j} | q _ {\mathrm {i} (t - 1) + j - 1}, z _ {t}, l _ {t}) \\ \end{array}
$$
Considering that the latent variables are unobserved, we then learn the model by marginalizing over these variables to maximize the log marginal-likelihood of the observed question sequence  $Q$ , i.e.,  $\max(\log p(Q))$ .  $p(Q)$  can be formulated as Eq.(4) by the backward algorithm (Murphy, 2002), with the base cases  $\beta_T(o) = 1, \forall o \in \{1, \dots, K\}$ . The quantities in Eq.(4) are obtained from a dynamic program, which is differentiable. Thus, we can estimate the model's parameters from the unlabeled data  $D_U$  by back-propagation.
$$
\begin{array}{l} \beta_ {t} (o) = p \left(q _ {t + 1: T} | z _ {t} = o\right) \\ = \sum_ {k = 1} ^ {K} \beta_ {t} ^ {*} (k) p \left(z _ {t + 1} = k \mid z _ {t} = o\right) \\ \end{array}
$$
$$
\begin{array}{l} \beta_ {t} ^ {*} (k) = p \left(q _ {t + 1: T} \mid z _ {t + 1} = k\right) \\ = \sum_ {j = 1} ^ {L} \left[ \beta_ {t + j} (k) p \left(l _ {t + 1} = j \mid z _ {t + 1} = k\right) \right. \\ \end{array}
$$
$$
\begin{array}{l} p \left(q _ {t + 1: t + j} \mid z _ {t + 1} = k, l _ {t + 1} = j\right) ] \\ p (Q) = \sum_ {k = 1} ^ {K} \beta_ {0} ^ {*} (k) p \left(z _ {1} = k\right) \tag {4} \\ \end{array}
$$
2.3 Multi-hop QG Net with Regularization
Afterward, we incorporate the learned patterns into the generation model as the prior. Such prior can be acted as a soft template to regularize the model. That can ensure the correctness of the results in syntax and semantics, especially when the labeled data is insufficient to learn the correspondence between the text and question. Fig.(2) illustrates the architecture of our model. We first estimate the prior pattern by sampling a sequence of latent states  $z$  with the length  $l$ . We then extract the reasoning chain and other textual contents involved in asking and solving a specific question from the given text. Under the guidance of both the reasoning chain and the prior patterns, we build a multi-hop QG model on the extracted contents by the sequence-to-sequence framework (Bahdanau et al., 2015). The evidential relations in the chain are used to enhance the logical rationality of the results. The prior pattern helps to facilitate the performance in low-resource conditions by specifying the segmentation of the generated results.
Figure 2: Flow chart of the low-resource multi-hop QG network.
2.3.1 Prior Patterns Estimation
Using the Viterbi algorithm (Zucchini et al., 2016), we can obtain the typed segmentation of a given question. Such segmentation can be characterized by a sequence of latent states  $z$ . Each segment, like the phrase, is associated with a state, reflecting that the state frequently produces that segment. Based on the labeled data  $D_{L}$ , we can collect all sequences of latent states, which can be seen as a pool of prior patterns. We sample one from the pool uniformly. And then, we view it as a question template with  $S$  distinct segments, as  $\{< z_t^k,l_t^k >\}_{k = 1}^S$ , where  $z_{t}$  is a state variable for the  $t^{th}$  term,  $l_{t}$  is the length variable derived by the probability  $p(l_{t}|z_{t})$ ,  $z_{t}^{k}$  and  $l_{t}^{k}$  are obtained by collapsing adjacent  $z_{t}$  and  $l_{k}$  with the same value. In order to easily incorporate into the generation model, we encode the template as a vector  $v_{k}^{m} = g_{z_{t}}\odot h_{k}^{m}$ , where  $h_k^m$  is the hidden vector for generating  $m^{th}$  term, as GRU  $(h_{k - 1}^{m},[e_{zm};e_{yt - 1}])$ ,  $m$  satisfies  $\mathrm{i}(m - 1) < t\leq \mathrm{i}(m)$ ,  $k = t - \mathrm{i}(m - 1)$ .
2.3.2 Question-Related Content Extraction
Given a text, we use the method proposed by Yu et al. (2020) to extract the question-related content. In order to make the paper self-contained, we briefly describe the approach in this section. It first extracted the entities from the text, and view them as the potential answers and evidences. It then links the entities to create a graph by three kind
6732
s of relations, including dependency, coreference, and synonym. Based on the graph, it heuristically extracts a sub-graph as the reasoning chain. The textual contents on the sub-graph are then gathered, including the answer, reasoning type, evidential entities, and sentences on the entities. The extraction is based on three question types, consisting of the Sequence, Intersection, and Comparison. These types account for a large proportion of the multi-hop questions on most typical data sets, for example,  $92\%$  in HotpotQA data set (Min et al., 2019).
2.3.3 Question Generation with Guidance
We then develop a multi-hop QG model based on the extracted contents. This model is guided by the reasoning chain and prior pattern, so that the generated results are not only logical but also fluent. In the pre-processing phase, we first mask the answer from the input contents by a special token  $<\mathrm{UNK}>$ , to avoid the answer inclusion problem (Sun et al., 2018). That is, the answer words may appear in the question that would reduce the rationality.
Encoder: The reasoning chain is encoded via an  $N$  head graph transformer (Vaswani et al., 2017), so as to integrate all evidential relations fully. Each node is represented by contextualizing on its neighbors, as  $h_v^g = e_v + \left\|_{n=1}^N \sum_{j \in \mathcal{N}_v} a^n(e_v, e_j) \mathrm{W}^n e_j$ , where  $\|\cdot\|$  denotes the concatenation,  $e_v$  is the embedding of node's entity,  $a^n(\cdot, \cdot)$  is  $n^{th}$  head attention function,  $\mathcal{N}_v$  is the set of neighbors. By aggregation with  $N$ -head attention, we can get a relation-aware vector  $c_g$  as Eq.(5), where  $\mathrm{W}_g^n$ ,  $\mathrm{W}_h$ ,  $\mathrm{W}_d$  are trainable matrices,  $\mathcal{C}$  is the set of nodes in the chain.
$$
\begin{array}{l} a ^ {n} \left(s _ {t}, h _ {v} ^ {g}\right) = \frac {\exp \left(\left(\mathrm {W} _ {h} h _ {v} ^ {g}\right) ^ {\top} \mathrm {W} _ {d} s _ {t}\right)}{\sum_ {k \in \mathcal {N} _ {v}} \exp \left(\left(\mathrm {W} _ {h} h _ {k} ^ {g}\right) ^ {\top} \mathrm {W} _ {d} s _ {t}\right)} \tag {5} \\ c _ {g} = s _ {t} + \left\| _ {n = 1} ^ {N} \sum_ {v \in \mathcal {C}} a ^ {n} \left(s _ {t}, h _ {v} ^ {g}\right) \mathrm {W} _ {g} ^ {n} h _ {v} ^ {g} \right. \\ \end{array}
$$
Other textual inputs are encoded in two steps: (1) each text term is embedded by looking up the pre-trained vectors, such as BERT (Devlin et al., 2019). (2) The resulting embeddings are fed into a bi-directional GRU to incorporate a sequential context. In detail, the sentences are represented by concatenating the final hidden states of GRU, as  $\overleftarrow{[h_1^b; h_J^b]}$ , where  $j^{th}$  term is  $h_j^b = \overleftarrow{[h_j^b; h_j^b]}$ ,  $\overleftarrow{h_j^b} = GRU(e_j^b, \overleftarrow{h_{j+1}^b})$ ,  $\overrightarrow{h_j^b} = GRU(e_j^b, \overrightarrow{h_{j-1}^b})$ ; [.;] denotes the concatenation of two vectors;  $e_j^b$  is the augmented embedding of  $j^{th}$  term;  $J$  is the size of all terms. Similarly, the answer and evidence entities are integrally encoded as  $h^a = \overleftarrow{[h_1^a; h_O^a]}$ .
Attention: For the textual inputs, we fully integrate the encodings and their correlations by attention. First, we use self-attention (Wang et al., 2017) to grasp the long-term dependency in the sentences, as  $[\hat{h}_j^b]_{j=1}^J = \text{SelfAtt}([\hat{h}_j^b]_{j=1}^J)$ . Subsequently, we exploit multi-perspective fusion (Song et al., 2018) to grasp the answer-related context in the sentences and strengthen their cross interactions. That is,  $[h_j^{b'}]_{j=1}^J = \text{MulPerFuse}([\hat{h}_j^b]_{j=1}^J, [h_o^a]_{o=1}^O)$ . By aggregating the significant information over all the terms, we can obtain a context vector  $c_t$  as Eq.(6), where  $\alpha_{tj}$  is the normalized attention weight,  $a_{tj}$  denotes the alignment score,  $s_t$  refers to the  $t^{th}$  hidden state of the decoder,  $v, b, W_s, W_b$  are trainable parameters.
$$
\begin{array}{l} a _ {t k} = v ^ {\mathsf {T}} \tanh  \left(\mathrm {W} _ {s} s _ {t} + \mathrm {W} _ {b} h _ {k} ^ {b ^ {\prime}} + b\right) \\ \alpha_ {t j} = \exp \left(a _ {t j}\right) / \sum_ {k = 1} ^ {J} \exp \left(a _ {t k}\right) \tag {6} \\ c _ {t} = \sum_ {j = 1} ^ {J} \alpha_ {t j} h _ {j} ^ {b} \\ \end{array}
$$