Focus-aware Response Generation in Inquiry Conversation (2023)

Abstract: Inquiry conversation is a common form of conversation that aims to complete the investigation (e.g., court hearing, medical consultation and police interrogation) during which a series of focus shifts occurs. While many models have been proposed to generate a smooth response to a given conversation history, neglecting the focus can limit performance in inquiry conversation where the order of the focuses plays there a key role. In this paper, we investigate the problem of response generation in inquiry conversation by taking the focus into consideration. We propose a novel Focus-aware Response Generation (FRG) method by jointly optimizing a multi-level encoder and a set of focal decoders to generate several candidate responses that correspond to different focuses. Additionally, a focus ranking module is proposed to predict the next focus and rank the candidate responses. Experiments on two orthogonal inquiry conversation datasets (judicial, medical domain) demonstrate that our method generates results significantly better in automatic metrics and human evaluation compared to the state-of-the-art approaches.

## introduction thanks to the high effectiveness of machine learning techniques, natural language processing (nlp) has made tremendous progress in a variety of tasks; for example, in conversation response generation which empowers many applications such as chatbots (e.g., siri). the performance of response generation was significantly improved after applying neural network models such as recurrent neural networks (rnn) @xcite and transformers @xcite . however, existing studies on response generation mainly concentrate on relevance and fluency, rarely paying attention to the focus, which is important from the viewpoint of the rationality of generated responses. inquiry conversation (inquiry dialogue) is a common form of conversation that aims to complete the investigation @xcite (e.g., court hearing, medical consultation, police interrogation). focus shifts tend to occur often in inquiry conversation, and their order plays a key role. for example, as shown in fig. 1 , a judge will not issue the verdict before the defendants have finished pleading, and the doctor will not prescribe drugs before stating a diagnosis. the latent focuses of utterances often affect dialogue development, and hence it is beneficial to incorporate the notion of focus in the response generation process. in this paper, we focus on response generation in inquiry conversation, and we aim at improving the rationality of generated content. for practical reasons, we only generate the responses of the leading role speaker in a conversation (e.g., judge, doctor). when addressing this problem one faces the following challenges: (1) the focuses are sequential yet latent. the next response should be generated considering the focuses underlying in the conversation history, and the next focus needs to be predicted. (2) the focuses are discrete and different focuses correspond to different responses. thus, the generator needs to determine the focus before and then generate a response guided by the established focus. to address these challenges, we propose a novel focus-aware response generation (frg) method by jointly optimizing a multi-level encoder, a set of focal decoders (to generate responses with different focuses), and a synergistic focus ranking module. specifically, the multi-level encoder is designed to better learn the latent focuses from the conversation history based on the aggregated characteristics of speakers and the content in each block (defined in sec.3) through a speaker level attention layer and a block level attention layer. then, each decoder in the set of focal decoders generates a candidate response guided by its corresponding focus. finally, the focus ranking module ranks all the candidate responses generated by the focal decoders and predicts the next focus for the final output. to test the proposed method, we employ two inquiry conversation datasets from two diverse domains -court hearing and medical consultation. due to the difficulty and high cost of annotating focuses in different domains which typically require input from domain experts, we use a two-stage training paradigm to assure the generalizability of our method. we first warm-up the decoders together with a large number of unlabeled data to ensure the generation ability, and then we fine-tune them separately on a small number of labeled data to ensure the generation quality of particular focus. extensive experiments show that the proposed frg model achieves the best performance on both automatic metrics and human evaluation. to sum up, our contributions are as follows: • we investigate the response generation task in inquiry conversation by involving the focus in the generation process. ## problem formulation in this section, we define the problem of response generation in inquiry conversation. we first describe the key concepts as below: inquiry conversation is a form of conversation that aims to complete an investigation @xcite (e.g., court hearing, medical consultation). focus is the center of the conversation at a certain stage of its progress. the focuses tend to shift during the conversation. leading role is the speaker (e.g., interrogating speaker such as a judge, doctor) who controls the focus shifts in the inquiry conversation. block consists of several consecutive utterances and is regarded as the smallest unit of the focus shifting. therefore, the conversation can be divided into several blocks according to the actions of the leading role speaker. response utterance refers to the interrogating utterance of the leading role speaker (examples are shown in figure 1 ). the problem of response generation in inquiry conversation is then defined here as follows: given the conversation history @xmath0, s t )} nu @xmath1, s t )} is tth pair of utterance u t and the role of speaker s t , the task is to determine the next focus f and based on it generate the corresponding response denoted as @xmath2. ## method in this section, we describe our focus-aware response generation (frg) model. fig. 2 shows the overall framework. our model consists of a shared multi-level encoder, a focus ranking module, and a set of focal decoders. the model works in a multitask learning manner. the ranking module and decoders take the output of the encoder as an input. ## conclusion and future work in this paper, we investigate the response generation task in inquiry conversation from a focal view and propose a novel focus-aware response generation (frg) method. we design a multi-level encoder to represent the conversation history at different levels, as well as a set of focal decoders to generate responses guided by different focuses. thanks to the focus ranking module, the generated responses are sorted for the final output. the experiment results show the effectiveness of our method. in the future, we will explore the following directions based on the frg method: (1) adding external knowledge to constrain the ranking module and (2) using the feedback of users to optimize the ranking module in practical applications.