SlowGuess/ACL_2020_OCR huggingface dataset:
Yifan Gao†, Chien-Sheng Wu‡, Shafiq Joty‡, Caiming Xiong‡, Richard Socher‡, Irwin King†, Michael R. Lyu†, and Steven C.H. Ho‡
$\ddagger$  Salesforce Research
$\dagger$  {yfgao,king,lyu}@cse.cuhk.edu.hk
$\ddagger$  {cswu,sjoty,cxiong,rsocher,shoi}@salesforce.com
The goal of conversational machine reading is to answer user questions given a knowledge base text which may require asking clarification questions. Existing approaches are limited in their decision making due to struggles in extracting question-related rules and reasoning about them. In this paper, we present a new framework of conversational machine reading that comprises a novel Explicit Memory Tracker (EMT) to track whether conditions listed in the rule text have already been satisfied to make a decision. Moreover, our framework generates clarification questions by adopting a coarse-to-fine reasoning strategy, utilizing sentence-level entailment scores to weight token-level distributions. On the ShARC benchmark (blind, held-out) testset, EMT achieves new state-of-the-art results of  $74.6\%$  micro-averaged decision accuracy and 49.5 BLEU4. We also show that EMT is more interpretable by visualizing the entailment-oriented reasoning process as the conversation flows. Code and models are released at https://github.com/Yifan-Gao/explicit_memory_tracking.
In conversational machine reading (CMR), machines can take the initiative to ask users questions that help to solve their problems, instead of jumping into a conclusion hurriedly (Saeidi et al., 2018). In this case, machines need to understand the knowledge base (KB) text, evaluate and keep track of the user scenario, ask clarification questions, and then make a final decision. This interactive behavior between users and machines has gained more attention recently because in practice users are unaware of the KB text, thus they cannot provide all the information needed in a single turn.
Figure 1: Example of Conversational Machine Reading tasks from the ShARC dataset (Saeidi et al., 2018). At each turn, given the rule text, a user scenario, an initial user question, and previous interactions, a machine can give a certain final answer such as Yes or No to the initial question. If the machine cannot give a certain answer because of missing information from the user, it will ask a clarification question to fill in the information gap. Clarification questions and their corresponding rules are marked in the same colors.
For instance, consider the example in Figure 1 taken from the ShARC dataset for CMR (Saeidi et al., 2018). A user posts her scenario and asks a question on whether her employer can take money from her final pay. Since she does not know the relevant rule text, the provided scenario and the initial question(s) from her are often too underspecified for a machine to make a certain decision. Therefore, a machine has to read the rule text and
* This work was mostly done when Yifan Gao was an intern at Salesforce Research Asia, Singapore.
Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 935-945 July 5 - 10, 2020. ©2020 Association for Computational Linguistics
ask a series of clarification questions until it can conclude the conversation with a certain answer.
Most existing approaches (Zhong and Zettlemoyer, 2019; Sharma et al., 2019) formalize the CMR problem into two sub-tasks. The first is to make a decision among Yes, No, Irrelevant, and Inquire at each dialog turn given a rule text, a user scenario, an initial question and the current dialog history. If one of Yes, No, or Irrelevant is selected, it implies that a final decision (Yes/No) can be made in response to the user's initial question, or stating the user's initial question is unanswerable (Irrelevant) according to the rule text. If the decision at the current turn is Inquire, it will then trigger the second task for follow-up question generation, which extracts an underspecified rule span from the rule text and generates a follow-up question accordingly.
However, there are two main drawbacks to the existing methods. First, with respect to the reasoning of the rule text, existing methods do not explicitly track whether a condition listed in the rule has already been satisfied as the conversation flows so that it can make a better decision. Second, with respect to the extraction of question-related rules, it is difficult in the current approach to extract the most relevant text span to generate the next question. For example, the state-of-the-art  $\mathrm{E}^3$  model (Zhong and Zettlemoyer, 2019) has only  $60.6\%$  F1 for question-related span extraction.
To address these issues, we propose a new framework of conversational machine reading with a novel Explicit Memory Tracker (EMT), which explicitly tracks each rule sentence to make decisions and generate follow-up questions. Specifically, EMT first segments the rule text into several rule sentences and allocates them into its memory. Then the initial question, user scenario, and dialog history are fed into EMT sequentially to update each memory module separately. At each dialog turn, EMT predicts the entailment states (satisfaction or not) for every rule sentence, and makes a decision based on the current memory status. If the decision is Inquire, EMT extracts a rule span to generate a follow-up question by adopting a coarse-to-fine reasoning strategy (i.e., weighting token-level span distributions with its sentence-level entailment scores). Compared to previous methods which only consider entailment-oriented reasoning for decision making or follow-up question generation, EMT utilizes its updated memory modules to
reason out these two tasks in a unified manner.
We compare EMT with the existing approaches on the ShARC dataset (Saeidi et al., 2018). Our results show that explicitly tracking rules with external memories boosts both the decision accuracy and the quality of generated follow-up questions. In particular, EMT outperforms the previous best model  $\mathrm{E}^3$  by 1.3 in macro-averaged decision accuracy and 10.8 in BLEU4 for follow-up question generation. In addition to the performance improvement, EMT yields interpretability by explicitly tracking rules, which is visualized to show the entailment-oriented reasoning process of our model.
As illustrated in Figure 2, our proposed method consists of the following four main modules.
(1) The Encoding module uses BERT (Devlin et al., 2019) to encode the concatenation of the rule text, initial question, scenario and dialog history into contextualized representations.
(2) The Explicit Memory Tracking module sequentially reads the initial question, user scenario, multi-turn dialog history, and updates the entailment state of each rule sentence.
(3) The Decision Making module does entailment-oriented reasoning based on the updated states of rule sentences and makes a decision among Yes, No, Irrelevant, and Inquire.
(4) If the decision is Inquire, the Question Generation module is activated, which reuses the updated states of rule sentences to identify the underspecified rule sentence and extract the most informative span within it in a coarse-to-fine manner. Then it rephrases the extracted span into a well-formed follow-up question.
Let  $x_{R}, x_{Q}, x_{S}, [x_{H,1}, x_{H,2}, \ldots, x_{H,P}]$  denote the input of rule text, initial question, user scenario, and  $P$  turns of dialog history, each of which is a sequence of tokens. We first split the rule text  $x_{R}$  into several rule sentences  $[x_{R,1}, x_{R,2}, \ldots, x_{R,M}]$  according to sentence boundary or bullet points, insert [CLS] tokens at the start of each sentence, and concatenate them into one sequence:
[CLS],  $x_{R,1}$  ... [CLS],  $x_{R,M}$ ; [CLS],  $x_{Q}$ ; [CLS],  $x_{S}$ ; [CLS],  $x_{H,1}$ ; ... [CLS],  $x_{H,P}$ ]. Then we use BERT (Devlin et al., 2019), a pretrained transformer encoder (Vaswani et al., 2017)
Figure 2: The Explicit Memory Tracker with Coarse-to-Fine Reasoning for Conversational Machine Reading (CMR). The CMR process includes (1) BERT encoding, (2) Explicit Memory Tracking for entailment state of each rule sentence, (3) Decision Making on updated entailment states of all rule sentences, (4) Question Generation via span extraction with coarse-to-fine reasoning and question rephrasing of the extracted span. (Best viewed in color)
to encode the sequence into a sequence of vectors with the same length. We treat each [CLS] representation as feature representation of the sentence that follows it. In this way, we receive both token-level representation and sentence-level representation for each sentence. We denote sentence-level representation of the rule sentences as  $\mathbf{k}_1,\dots ,\mathbf{k}_M$  and their token-level representation as  $[(u_{1,1},\dots,u_{1,n_1}),\dots,(u_{M,1},\dots,u_{M,n_M})]$ , where  $n_i$  is number of tokens for rule sentence  $i$ . Similarly, we denote the sentence-level representation of the initial question, user scenario, and  $P$  turns of dialog history as  $\mathbf{s}_Q,\mathbf{s}_S$ , and  $\mathbf{s}_1,\dots,\mathbf{s}_P$ , respectively. All these vectorized representations are of  $d$  dimensions (768 for BERT-base).
2.2 Explicit Memory Tracking
Given the rule sentences  $\mathbf{k}_1, \dots, \mathbf{k}_M$  and the user provided information including the initial question  $\mathbf{s}_Q$ , scenario  $\mathbf{s}_S$ , and  $P$  turns of dialog history  $\mathbf{s}_1, \dots, \mathbf{s}_P$ , our goal is to find implications between the rule sentences and the user provided information. Inspired by Recurrent Entity Network (Henaff et al., 2017) which tracks the world state
given a sequence of textual statements, we propose the Explicit Memory Tracker (EMT), a gated recurrent memory-augmented neural network which explicitly tracks the states of rule sentences by sequentially reading the user provided information.
As shown in Figure 2, EMT explicitly takes rule sentences  $\mathbf{k}_1,\dots,\mathbf{k}_M$  as keys, and assigns a state  $\mathbf{v}_i$  to each key to save the most updated entailment information (whether this rule has been entailed from the user provided information). Each value state  $\mathbf{v}_i$  is initialized with the same value of its corresponding rule sentence:  $\mathbf{v}_{i,0} = \mathbf{k}_i$ . Then EMT sequentially reads user provided information  $\mathbf{s}_Q,\mathbf{s}_S,\mathbf{s}_1,\dots,\mathbf{s}_P$ . At time step  $t$ , the value state  $\mathbf{v}_{i,t}$  for  $i$ -th rule sentence is updated by incorporating the user provided information  $\mathbf{s}_t\in \{\mathbf{s}_Q,\mathbf{s}_S,\mathbf{s}_1,\dots,\mathbf{s}_P\}$ ,
$$
\tilde {\mathbf {v}} _ {i, t} = \operatorname {R e L U} \left(\mathbf {W} _ {k} \mathbf {k} _ {i} + \mathbf {W} _ {v} \mathbf {v} _ {i, t} + \mathbf {W} _ {s} \mathbf {s} _ {t}\right), \tag {1}
$$
$$
g _ {i} = \sigma \left(\mathbf {s} _ {t} ^ {\top} \mathbf {k} _ {i} + \mathbf {s} _ {t} ^ {\top} \mathbf {v} _ {i, t}\right) \in [ 0, 1 ], \tag {2}
$$
$$
\mathbf {v} _ {i, t} = \mathbf {v} _ {i, t} + g _ {i} \odot \tilde {\mathbf {v}} _ {i, t} \in \mathbb {R} ^ {d}, \mathbf {v} _ {i, t} = \frac {\mathbf {v} _ {i , t}}{\| \mathbf {v} _ {i , t} \|} \tag {3}
$$
where  $\mathbf{W}_k, \mathbf{W}_v, \mathbf{W}_s \in \mathbb{R}^{d \times d}$ ,  $\sigma$  represents a sigmoid function, and  $\odot$  is scalar product. As the
user background input  $\mathbf{s}_t$  may only be relevant to parts of the rule sentences, the gating function in Equation 2 matches  $\mathbf{s}_t$  to the memory. Then EMT updates state  $\mathbf{v}_{i,t}$  only in a gated manner. Finally, the normalization allows EMT to forget previous information, if necessary. After EMT sequentially reads all user provided information (the initial question, scenario, and  $P$  turns of history dialog) and finishes entailment-oriented reasoning, keys and final states of rule sentences are denoted as  $(\mathbf{k}_1,\mathbf{v}_1),\dots,(\mathbf{k}_M,\mathbf{v}_M)$ , which will be used in the decision making module (Section 2.3) and question generation module (Section 2.4).
The key difference between our Explicit Memory Tracker and Recurrent Entity Network (REN) (Henaff et al., 2017) is that each key  $\mathbf{k}_i$  in our case has an explicit meaning (the corresponding rule sentence) and thus it changes according to different rule texts while in REN, the underlined meaning of keys are learned through training and they are fixed throughout all textual inputs. Moreover, the number of keys is dynamic in our case (according to the number of sentences parsed from the rule text) while that is predefined in REN.
2.3 Decision Making
Based on the most up-to-date key-value states of rule sentences  $(\mathbf{k}_1,\mathbf{v}_1),\dots,(\mathbf{k}_M,\mathbf{v}_M)$  from the EMT, the decision making module predicts a decision among Yes, No, Irrelevant, and Inquire. First, we use self-attention to compute a summary vector  $\mathbf{c}$  for the overall state:
$$
\alpha_ {i} = \mathbf {w} _ {\alpha} ^ {\top} \left[ \mathbf {k} _ {i}; \mathbf {v} _ {i} \right] + b _ {\alpha} \in \mathbb {R} ^ {1} \tag {4}
$$
$$
\tilde {\alpha} _ {i} = \operatorname {s o f t m a x} (\alpha) _ {i} \in [ 0, 1 ] \tag {5}
$$
$$
\mathbf {c} = \sum_ {i} \tilde {\alpha} _ {i} \left[ \mathbf {k} _ {i}; \mathbf {v} _ {i} \right] \in \mathbb {R} ^ {d} \tag {6}
$$
where  $[\mathbf{k}_i;\mathbf{v}_i]$  denotes the concatenation of the vectors  $\mathbf{k}_i$  and  $\mathbf{v}_i$ , and  $\alpha_{i}$  is the attention weight for the rule sentence  $\mathbf{k}_i$  that determines the likelihood that  $\mathbf{k}_i$  is entailed from the user provided information. Then the final decision is made through a linear transformation of the summary vector  $\mathbf{c}$ :
$$
\mathbf {z} = \mathbf {W} _ {z} \mathbf {c} + \mathbf {b} _ {z} \in \mathbb {R} ^ {4} \qquad (7)
$$
where  $\mathbf{z} \in \mathbb{R}^4$  contains the model's score for all four possible classes. Let  $l$  indicate the correct decision, the decision making module is trained with the following cross entropy loss:
$$
\mathcal {L} _ {\mathrm {d e c}} = - \log \operatorname {s o f t m a x} (\mathbf {z}) _ {l} \tag {8}
$$
In order to explicitly track whether a condition listed in the rule has already been satisfied or not, we add a subtask to predict the entailment states for each rule sentence. The possible entailment labels are Entailment, Contradiction and Unknown; details of acquiring such labels are described in Section 3.1. With this intermediate supervision, the model can make better decisions based on the correct entailment state of each rule sentence. The entailment prediction is made through a linear transformation of the most up-to-date key-value state  $[\mathbf{k}_i;\mathbf{v}_i]$  from the EMT module:
$$
\mathbf {e} _ {i} = \mathbf {W} _ {e} [ \mathbf {k} _ {i}; \mathbf {v} _ {i} ] + \mathbf {b} _ {e} \in \mathbb {R} ^ {3} \qquad (9)
$$
where  $\mathbf{e}_i\in \mathbb{R}^3$  contains scores of three entailment states  $[\beta_{\mathrm{entailment},i},\beta_{\mathrm{contradiction},i},\beta_{\mathrm{unknown},i}]$  for the  $i$ -th rule sentence. Let  $r$  indicate the correct entailment state. The entailment prediction subtask is trained with the following cross entropy loss, normalized by the number of rule sentences  $M$ :
$$
\mathcal {L} _ {\text {e n t a i l}} = - \frac {1}{M} \sum_ {i = 1} ^ {M} \log \operatorname {s o f t m a x} \left(\mathbf {e} _ {i}\right) _ {r} \tag {10}
$$
2.4 Follow-up Question Generation
When the decision making module predicts Inquire, a follow-up question is required for further clarification from the user. In the same spirit of previous studies (Zhong and Zettlemoyer, 2019; Sharma et al., 2019), we decompose this problem into two stages. First, we extract a span inside the rule text which contains the underspecified user information (we name it as underspecified span hereafter). Second, we rephrase the extracted underspecified span into a follow-up question. We propose a coarse-to-fine approach to extract the underspecified span for the first stage, and finetune the pretrained language model UniLM (Dong et al., 2019) for the follow-up question rephrasing, as we describe below.
Coarse-to-Fine Reasoning for Underspecified Span Extraction. Zhong and Zettlemoyer (2019) extract the underspecified span by extracting several spans and retrieving the most likely one. The disadvantage of their approach is that extracting multiple rule spans is a challenging task, and it will propagate errors to the retrieval stage. Instead of extracting multiple spans from the rule text, we propose a coarse-to-fine reasoning approach to directly identify the underspecified
span. For this, we reuse the Unknown scores  $\beta_{\mathrm{unknown},i}$  from the entailment prediction subtask (Eqn. 9), and normalize it (over the rule sentences) with a softmax to determine how likely that the  $i$ -th rule sentence contains the underspecified span:
$$
\zeta_ {i} = \operatorname {s o f t m a x} \left(\beta_ {\text {u n k n o w n}}\right) _ {i} \in [ 0, 1 ] \tag {11}
$$
Knowing how likely a rule sentence is underspecified greatly reduces the difficulty to extract the underspecified span within it. We adopt a soft selection approach to modulate span extraction (i.e., predicting the start and end points of a span) score by rule sentence identification score  $\zeta_{i}$ . We follow the BERTQA approach (Devlin et al., 2019) to learn a start vector  $\mathbf{w}_s \in \mathbb{R}^d$  and an end vector  $\mathbf{w}_e \in \mathbb{R}^d$  to locate the start and end positions from the whole rule text. The probability of  $j$ -th word in  $i$ -th rule sentence  $\mathbf{u}_{i,j}$  being the start/end of the span is computed as a dot product between  $\mathbf{w}_s$  and  $\mathbf{u}_{i,j}$ , modulated by its rule sentence score  $\zeta_{i}$ :
$$
\gamma_ {i, j} = \mathbf {w} _ {s} ^ {\top} \mathbf {u} _ {i, j} * \zeta_ {i}, \delta_ {i, j} = \mathbf {w} _ {e} ^ {\top} \mathbf {u} _ {i, j} * \zeta_ {i} \tag {12}
$$
We extract the span with the highest span score  $\gamma *\delta$  under the restriction that the start and end positions must belong to the same rule sentence. Let  $s$  and  $e$  be the ground truth start and end position of the span. The underspecified span extraction loss is computed as the pointing loss
$$
\mathcal {L} _ {\text {s p a n}, \mathrm {s}} = - \mathbb {1} _ {l = \text {i n q u i r e}} \log \operatorname {s o f t m a x} (\gamma) _ {s} \tag {13}
$$
$$
\mathcal {L} _ {\text {s p a n}, e} = - \mathbb {1} _ {l = \text {i n q u i r e}} \log \operatorname {s o f t m a x} (\delta) _ {e} \tag {14}
$$
The overall loss is the sum of the decision loss, entailment prediction loss and span extraction loss
$$
\mathcal {L} = \mathcal {L} _ {\mathrm {d e c}} + \lambda_ {1} \mathcal {L} _ {\text {e n t a i l}} + \lambda_ {2} \mathcal {L} _ {\text {s p a n}} \tag {15}
$$
where  $\lambda_{1}$  and  $\lambda_{2}$  are tunable hyperparameters.
Question Rephrasing. The underspecified span extracted in the previous stage is fed into the question rephrasing model to generate a follow-up question. We finetune the UniLM (Dong et al., 2019) to achieve this goal. UniLM is a pretrained language model which demonstrates its effectiveness in both natural language understanding and generation tasks. Specifically, it outperforms previous methods by a large margin on the SQuAD question generation task (Du and Cardie, 2018).
As shown in Figure 2, UniLM takes the concatenation of rule text and the extracted rule span as
input, separated by the sentinel tokens: [CLS] rule-text [SEP] extracted-span [SEP]. The training target is the follow-up question we want to generate. Please refer to Dong et al. (2019) for details on finetuning UniLM and doing inference with it.
Dataset. We conduct experiments on the ShARC CMR dataset (Saeidi et al., 2018). It contains 948 dialog trees, which are flattened into 32,436 examples by considering all possible nodes in the trees. Each example is a quintuple of (rule text, initial question, user scenario, dialog history, decision), where decision is either one of  $\{\mathrm{Yes},\mathrm{No},$  Irrelevant} or a follow-up question. The train, development, and test dataset sizes are 21890, 2270, and 8276, respectively.
End-to-End Evaluation. Organizers of the ShARC competition evaluate model performance as an end-to-end task. They first evaluate the micro- and macro-accuracy for the decision making task. If both the ground truth decision and the predicted decision are Inquire, then they evaluate the generated follow-up question using BLEU score (Papineni et al., 2002). However, this way of evaluating follow-up questions has one issue. If two models have different Inquire predictions, the follow-up questions for evaluation will be different, making the comparison unfair. For example, a model could classify only one example as Inquire in the whole test set and generate the follow-up question correctly, achieving a  $100\%$  BLEU score. Therefore, we also propose to evaluate the follow-up question generation performance in an oracle evaluation setup as described below.
Oracle Question Generation Evaluation. In this evaluation, we ask the models to generate follow-up questions whenever the ground truth decision is Inquire, and compute the BLEU score for the generated questions accordingly. In this setup, there are 6804 examples for training and 562 examples for evaluation.
Data Augmentation. In the annotation process of the ShARC dataset, the scenario is manually constructed from a part of the dialog history, and that excerpt of the dialog is not shown as input to
Leaderboard: https://sharc-data.github.io/leaderboard.html
Models	End-to-End Task (Leaderboard Performance)
Micro Acc.	Macro Acc.	BLEU1	BLEU4
Seq2Seq (Saeidi et al., 2018)	44.8	42.8	34.0	7.8
Pipeline (Saeidi et al., 2018)	61.9	68.9	54.4	34.4
BERTQA (Zhong and Zettlemoyer, 2019)	63.6	70.8	46.2	36.3
UrcaNet (Sharma et al., 2019)	65.1	71.2	60.5	46.1
BiSon (Lawrence et al., 2019)	66.9	71.6	58.8	44.3
E3 (Zhong and Zettlemoyer, 2019)	67.6	73.3	54.1	38.7
EMT (our single model)	69.1	74.6	63.9	49.5
the model. Instead, it is treated as the evidence which should be entailed from the scenario. To effectively utilize this additional signal, we construct more examples by replacing the scenario with the evidence. This leads to additional 5800 training instances. We use this augmented dataset for the EMT model and its ablations in our experiments.
Labeling Underspecified Spans. To supervise the process of coarse-to-fine reasoning, we follow Zhong and Zettlemoyer (2019) to label the rule spans. We first trim the follow-up questions in the conversation by removing question words "do, does, did, is, was, are, have" and the question mark "?". For each trimmed question, we find the shortest span inside the rule text which has the minimum edit distance from the trimmed question, and treat it as an underspecified span.
Acquiring Labels for Entailment. To supervise the subtask of entailment prediction for each rule sentence, we use a heuristic to automatically label its entailment state. For each rule sentence, we first find if it contains any underspecified span for the questions in the dialog history (and evidence text), and use the corresponding Yes/No answers to label the rule text as Entailment/Contradiction. The rule text without any underspecified span is labeled as Unknown.
Implementation Details. We tokenize all text inputs with spaCy (Honnibal and Montani, 2017). The EMT model and the follow-up question generation model UniLM are trained separately and pipelined together at test time. For EMT, we use the uncased BERT base model (Wolf et al., 2019) for encoding. We train EMT with Adam (Kingma and Ba, 2015) optimizer with a learning rate of 5e-5, a warm-up rate of 0.1 and a dropout rate of 0.35. The loss weights  $\lambda_{1}$  and  $\lambda_{2}$  in Eq. 15 are set to 10 and 0.6 respectively, based on the development set results. For UniLM, we fine-tuning it with a batch
Table 1: Performance on the blind, held-out test set of ShARC end-to-end task.
Models	Yes	No	Inquire	Irrelevant
BERTQA	61.2	61.0	62.6	96.4
E3	65.9	70.6	60.5	96.4
UrcaNet*	63.3	68.4	58.9	95.7
EMT	70.5	73.2	70.8	98.6