Smoothing Dialogue States for Open Conversational Machine Reading (2021)

Abstract: Conversational machine reading (CMR) requires machines to communicate with humans through multi-turn interactions between two salient dialogue states of decision making and question generation processes. In open CMR settings, as the more realistic scenario, the retrieved background knowledge would be noisy, which results in severe challenges in the information transmission. Existing studies commonly train independent or pipeline systems for the two subtasks. However, those methods are trivial by using hard-label decisions to activate question generation, which eventually hinders the model performance. In this work, we propose an effective gating strategy by smoothing the two dialogue states in only one decoder and bridge decision making and question generation to provide a richer dialogue state reference. Experiments on the OR-ShARC dataset show the effectiveness of our method, which achieves new state-of-the-art results.

## introduction the ultimate goal of multi-turn dialogue is to enable the machine to interact with human beings and solve practical problems @xcite @xcite @xcite . it usually adopts the form of question answering (qa) according to the user's query along with the dialogue context @xcite @xcite . the machine may also actively ask questions for confirmation @xcite @xcite . in the classic spoken language understanding tasks @xcite @xcite , specific slots and intentions are usually defined. according to these predefined patterns, the machine interacts with people according to the dialogue states, and completes specific tasks, such as ordering meals @xcite and air tickets @xcite . in real-world scenario, annotating data such as intents and slots is expensive. inspired by the studies of reading comprehension @xcite @xcite , there appears a more general task -conversational machine reading (cmr) @xcite : given the inquiry, the machine is required to retrieve relevant supporting rule documents, the machine should judge whether the goal is satisfied according to the dialogue context, and make decisions or ask clarification questions. a variety of methods have been proposed for the cmr task, including 1) sequential models that encode all the elements and model the matching relationships with attention mechanisms @xcite @xcite ; 2) graph-based methods that capture the discourse structures of the rule texts and user scenario for better interactions @xcite . however, there are two sides of challenges that have been neglected: open-retrieval rule text-1: eligible applicants may obtain direct ... rule text-2: guaranteed loans for up to a maximum ... rule text-3: $1,392,000 ... user scenario: i got my loan last year. it was for 450,000. ## related work most of the current conversation-based reading comprehension tasks are formed as either spanbased qa @xcite or multi-choice tasks @xcite , both of which neglect the vital process of question generation for confirmation during the human-machine interaction. in this work, we are interested in building a machine that can not only make the right decisions but also raise questions when necessary. the related task is called conversational machine reading @xcite which consists of two separate subtasks: decision making and question generation. compared with conversation-based reading comprehension tasks, our concerned cmr task is more challenging as it involves rule documents, scenarios, asking clarification questions, and making a final decision. existing works @xcite ques: am i entitled to the national minimum wage? scen: i am not following a european union programme. the following are not entitled to the national minimum wage: higher students on a work placement up to 1 year workers on government pre-apprenticeships schemes people on the european union programmes people working on a jobcentre plus work trial for 6 weeks rule documents document 1 r 1 r 2 r 3 u s u q h 1 relations user scenario gcn @xmath0 ## open-retrieval setting for cmr in the cmr task, each example is formed as a tuple {r, u s , u q , c}, where r denotes the rule texts, u s and u q are user scenarios and user questions, respectively, and c represents the dialogue history. for open-retrieval cmr, r is a subset retrieved from a large candidate corpus d. the goal is to train a discriminator f(•, •) for decision making, and a generator g(•, •) on {r, u s , u q , c} for question generation. ## model our model is composed of three main modules: retriever, encoder, and decoder. the retriever is employed to retrieve the related rule texts for the given user scenario and question. the encoder takes the tuple {r, u s , u q , c} as the input, encodes the elements into vectors and captures the contextualized representations. the decoder makes a decision or generates a question once the decision is "inquiry". figure 1 overviews the model architecture, we will elaborate the details in the following part. ## analysis 6.1 comparison of open-retrieval methods we compare two typical retrievals methods, tf-idf and dense passage retrieval (dpr), which are widely-used traditional models from sparse vector space and recent dense-vector-based ones for opendomain retrieval, respectively. we also present the results of tf-idf+dpr (denoted dpr++) follow-dpr top1 top5 top10 top20 ing @xcite , using a linear combination of their scores as the new ranking function. the overall results are present in