U ni GDD : A Unified Generative Framework for Goal-Oriented Document-Grounded Dialogue (2022)

Abstract: The goal-oriented document-grounded dialogue aims at responding to the user query based on the dialogue context and supporting document. Existing studies tackle this problem by decomposing it into two sub-tasks: knowledge identification and response generation. However, such pipeline methods would unavoidably suffer from the error propagation issue. This paper proposes to unify these two sub-tasks via sequentially generating the grounding knowledge and the response. We further develop a prompt-connected multi-task learning strategy to model the characteristics and connections of different tasks and introduce linear temperature scheduling to reduce the negative effect of irrelevant document information. Experimental results demonstrate the effectiveness of our framework.

## introduction recent years have seen significant progress in goaloriented dialogues @xcite @xcite @xcite , which aim at assisting end users in accomplishing certain goals via natural language interactions. however, due to the lack of external knowledge, most goal-oriented dialogue systems are restricted to providing information that can only be handled by given databases or apis @xcite and completing certain tasks in a specific domain such as restaurant booking. to address this challenge, goal-oriented document-grounded dialogue has been proposed to leverage external documents as the knowledge source to assist the dialogue system in satisfying users' diverse information needs @xcite . your application for renewal of a driving school license must be submitted between 30 and 60 days before the license expires (the expiration date is printed on your license. ) ## our unigdd framework unigdd is a multi-task generative framework for the goal-oriented document-grounded dialogue problem. main task given the dialogue context @xmath0, a 1 , . . . , u t-1 , a t-1 , u t ) and grounding document d, where u i is the i-th user utterance and a i is the i-th agent utterance, our main task aims to generate the target sequence @xmath1, a t ), where k t is the grounding knowledge from d and a t is the response to u t . specifically, for the example in figure 1 , the input and output of the main task are as follows: input: @xmath2: <@xmath3... ? <@xmath4... <@xmath5... ? <@xmath6... your application for renewal ... output: <@xmath7... <@xmath8... we use different special tokens to identify different elements in the input and output. for example, we add "<@xmath9, "<@xmath10, and "<@xmath11. the prompt "@xmath12:" is added to the dialogue context and supporting document to form the input and guide the model to generate the grounding knowledge and the response in order. the input-to-target generation can be modeled with a pre-trained encoder-decoder model m : (c, d, t p ) → (k t , a t ) such as t5 @xcite ), where t p is the task prompt. prompt-connected multi-task learning we introduce two auxiliary tasks to steer our framework to model the respective characteristics of knowledge identification and response generation. given the dialogue context c and grounding document d, these two tasks aim to generate the grounding knowledge k t and the response a t with the same model m. as depicted in figure 2 , we construct prompts "@xmath13:" and "@xmath14:" for them. these prompts indicate the model that the goals of the two auxiliary tasks are to generate the first part and the second part of the target sequence of the main task, respectively. as a result, the connections between different tasks are naturally modeled. instead of using discrete language phrases, we randomly initialize the embeddings of those special tokens in the prompts and train them end-to-end to better encode the characteristics and connections of these tasks. linear temperature scheduling for a specific user query in the dialogue, many document contents are actually irrelevant. to force the model to pay less attention to the irrelevant parts, we propose a linear temperature scheduling strategy to make the attention distribution of cross-attention gradually sharper during the training process. specifically, we design the softmax function in the cross-attention module of each decoder layer as follows: compared with the original cross-attention module, the ending temperature 0 < τ @xmath15, giving more attention weight to the relevant content. training the model is trained with a maximum likelihood objective. given the training example @xmath16, d, t p, y ), the objective l θ is defined as inference after training, for each pair of dialogue context and document (c, d), we generate the target sequence of the main task for obtaining the grounding knowledge k t and the response a t . ## conclusion our unigdd framework unifies knowledge identification and response generation and models their characteristics via a multi-task generative modeling strategy. both automatic evaluation and human evaluation demonstrate the effectiveness of our framework.