Explicit Alignment and Many-to-many Entailment Based Reasoning for Conversational Machine Reading (2023)

Abstract: Conversational Machine Reading (CMR) requires answering a user’s initial question through multi-turn dialogue interactions based on a given document. Although there exist many effective methods, they largely neglected the alignment between the \textit{document} and the \textit{user-provided information} , which significantly affects the intermediate decision-making and subsequent follow-up question generation. To address this issue, we propose a pipeline framework that (1) aligns the aforementioned two sides in an explicit way, (2) makes decisions using a lightweight many-to-many entailment reasoning module, and (3) directly generates follow-up questions based on the document and previously asked questions. Our proposed method achieves state-of-the-art in micro-accuracy and ranks the first place on the public leaderboard of the CMR benchmark dataset ShARC.

## introduction the conversational machine reading (cmr) task @xcite requires an agent to answer an initial question from users through multi-turn dialogue interactions based on a given document. as shown in figure 1 , a typical process involves two steps, (1) the agent first makes a decision classification among irrelevant, yes, no and more, (2) if the decision is more, the agent generates a question to clarify an unmentioned condition in the given document, otherwise responds directly. recent research @xcite @xcite @xcite @xcite has explored how to improve the abilities of decision-making and question generation. @xcite . for decision-making, one common approach first segments the document into many text spans at different granularity levels (e.g., sentences or elementary discourse units (edus)). then complex modules are adopted to predict the entailment state for each document span based on user scenario and previous dialogue history (both are user-provided information). finally, decisions are made based on the entailment states of all document spans. one effective module for predicting entailment states is transformer blocks @xcite , which are widely adopted @xcite @xcite . however, the aforementioned approach has overlooked the explicit alignment between the document and the userprovided information, such as text spans marked with the same color as shown in figure 1 . since not all user-provided information is relevant to a particular document span, the lack of explicit alignment leads to sparse attention and introduces noises that affect the prediction of the entailment state. furthermore, recent work @xcite tries to leverage relational graph convolutional networks, which results in a heavyweight decision module, therefore greatly imposing a substantial burden on computation and memory resources. for question generation, most works @xcite @xcite first extract an unmentioned span in the document and then rewrite it into a follow-up question. the extract-then-rewrite method relies heavily on extracted spans, the failure to properly extract an unmentioned span results in generating a redundant or even irrelevant followup question. to address these issues, we propose a pipeline approach consisting of a reasoning model based on bipartite alignment and many-to-many entailment (biae) for decision-making, and a directly finetuned model for question generation foot_0 . our approach (1) explicitly aligns the document and the user-provided information by introducing supervison from an external model, (2) uses a lightweight core decision module with only linear layers, which predicts many-to-many entailment states using aligned information and feature vectors, (3) directly uses the whole document and previously asked questions to generate follow-up questions without extracting underspecified document spans. through extensive experiments on the cmr benchmark dataset sharc @xcite , we demonstrate that biae significantly outperforms baselines with lightweight decision modules by at least 12.7% in micro accuracy and the finetuned model outperforms all baselines using extract-thenrewrite generation method. our contributions can be summarized as follows: machine reading comprehension (mrc) is a classical and fruitful research field with various tasks and focuses, such as extractive tasks @xcite @xcite , cloze-style and multiple choice tasks @xcite @xcite @xcite , multi-document tasks @xcite @xcite . among them, we focus on conversational machine reading (cmr) @xcite , which is a critical but more challenging task: pretrained language model d 1 d 2 d 3 𝑈 𝑞 𝑈 1 (𝑆 1 ) 𝑈 2 (𝑆 2 ) d 4 𝑈 3 (𝑇 1 ) 𝑑 1 𝑑 2 𝑑 3 𝑢 𝑞 𝑢 1 𝑢 2 𝑑 4 𝑢 3 1. alignment and entailment based reasoning alignment scores a decision probabilities p irrelevant yes no more entailment probabilities 𝐸 𝐸 (𝐸) 𝐸 (𝐶) 𝐸 (𝑁) 𝑑 1 𝑑 2 𝑑 3 𝑑 4 𝑒 1 𝑒 2 𝑒 3 𝑒 4 𝑠 𝑢 𝑞 learnable vectors 𝒆 𝐸 𝒆 𝐶 𝒆 𝑁 𝐸 𝑖𝑗 (𝐶) 𝐸 𝑖𝑗 (𝑁) 𝐸 𝑖𝑗 (𝐸) + × cat&att decision layer alignment layer entailment layer 2. generation pretrained language model d 𝑓 1 𝑓 n … follow up question in comparison with these works, our work focuses on the explicit alignment of information from both the document and the user, and employs a simpler entailment reasoning structure. then we adopt a new approach to directly generate followup questions based on the document and the questions asked. ## methodology the cmr task can be formulated as follows: give input @xmath0, q, s, h), d is the document, q is the user's initial question, s is the user's scenario, @xmath1, a 1 ), • • • , (f n h , a n h ), f i is a follow-up question that was already asked, a @xmath2, no }, is the dialogue history, a cmr system g makes a response @xmath3. we propose a classification model based on bipartite alignment and entailment (biae) for intermediate decision-making. if the decision is ir-relevant, yes or no, the system provides a direct response. if the decision is more, a finetuned model is used for question generation. the overall architecture of classification and generation is displayed in figure 2 . ## conclusion we propose a new framework for conversational machine reading in this paper. our classification model, biae, leverages many-to-many entailment reasoning, enhanced by explicit alignment, for decision-making. and our t5 biae is directly finetuned for generating follow-up questions to clarify underspecified document spans. biae significantly reduces the parameters in the core decision-making module and achieves results comparable to strong baselines. extensive experiments demonstrate the effectiveness of our framework. through analysis, we believe that improving the ability of entailment reasoning among the overall hypotheses of a document is crucial for enhancing decision-making ability.