OPAL : Ontology-Aware Pretrained Language Model for End-to-End Task-Oriented Dialogue (2023)

Abstract: This paper presents an ontology-aware pretrained language model (OPAL) for end-to-end task-oriented dialogue (TOD). Unlike chit-chat dialogue models, task-oriented dialogue models fulfill at least two task-specific modules: Dialogue state tracker (DST) and response generator (RG). The dialogue state consists of the domain-slot-value triples, which are regarded as the user’s constraints to search the domain-related databases. The large-scale task-oriented dialogue data with the annotated structured dialogue state usually are inaccessible. It prevents the development of the pretrained language model for the task-oriented dialogue. We propose a simple yet effective pretraining method to alleviate this problem, which consists of two pretraining phases. The first phase is to pretrain on large-scale contextual text data, where the structured information of the text is extracted by the information extracting tool. To bridge the gap between the pretraining method and downstream tasks, we design two pretraining tasks: ontology-like triple recovery and next-text generation, which simulates the DST and RG, respectively. The second phase is to fine-tune the pretrained model on the TOD data. The experimental results show that our proposed method achieves an exciting boost and obtains competitive performance even without any TOD data on CamRest676 and MultiWOZ benchmarks.

## introduction a task-oriented dialogue (tod) system aims to assist users in accomplishing a specific task by interacting with natural language, for example, reserving a hotel or booking flight tickets. with the popularity of the industrial dialogue system, the * the corresponding authors are lu chen and kai yu. the existing task-oriented dialogue system can be classified into two categories: pipeline format and end-to-end format. the pipeline tod system @xcite is composed of four modules: natural language understanding (nlu) @xcite , dialogue state tracking (dst) @xcite , dialogue policy (dp) @xcite @xcite , and natural language generation (nlg) @xcite @xcite . since each module of the system is trained separately and executes sequentially, it faces two serious issues: error accumulation and high annotation cost. thus, the end-to-end dialogue system @xcite gradually becomes the research focus, which formulates the task-oriented dialogue as a sequence-to-sequence task. the dialogue state, database (db) state, and the corresponding system response are directly concatenated together and flattened as a token sequence. the db state is the status of the domainrelated database searched with the dialogue state, as shown in figure 1 . thanks to the success of pretraining language models @xcite , effective application has shed light on open-domain (chit-chat) dialogues @xcite . nevertheless, utilizing such pretrained language models on tod systems remains challenging due to the limited tod data with annotated dialogue state. unlike the open-domain dialogue, tod is restricted by a dialogue ontology, which defines the dialogue domains, the slots and their candidate values. the tod system needs to predict the dialogue state and feedback the db content to accomplish a task. the dialogue state is structured information extracted from the dialogue context, which is a set of domain-slot-value triples. recently, some works @xcite try to directly leverage the pretrained language models, e.g., gpt-2 @xcite and bart @xcite , in the end-to-end tod system. such models @xcite are pretrained on the large-scale contextual text with the general self-supervised method, e.g., language modeling and language denoising. however, in the task-oriented dialogue task, the dialogue state is structured information rather than a contextual text. the inconsistency between the pretrained and downstream tasks will impact the performance of the plms on the tod benchmarks. to alleviate this problem, soloist @xcite fine-tunes the pretrained gpt-2 with the existing annotated tod data and then transfers it to the other task-oriented dialogue generation tasks. similarly, ncm @xcite first warm-ups the transformerbased model with large-scale reddit 1 @xcite data and then fine-tunes the model on the tod data. however, the existing tod data is too limited to pretrain a large-scale language model. to alleviate the problems above and advance pretrained language model research, especially its application on tod, we propose an ontologyaware pretrained language model (opal). from the high-level perspective, we can abstract the end-to-end tod task into two sub-tasks: ontology-like triple recovery and next-text generation, which corresponds to dialogue state tracking task and response generating task. the ontology-like triple recovery in the tod means to predict the corresponding value given the domain and the slot. the next-text generation is easy to design for the contextual text, which directly fulfills with masking the last sentence. the challenge is how to design the ontology-like triple recovery task, which needs to obtain the structured information from the contextual text. in this paper, we utilize the external openie tools @xcite foot_1 to extract the relation triples (subject-relation-object) from the contextual text as the structured information. in most cases, the domain-slot-value triple can be regarded as relation triple, for example, train-arrive-12:30. the relation triples extracted from the contextual text can be regarded as the ontology-like triples. we design self-supervised ontology-like triple recovery task and next-text generation task to pretrain the model. the main contributions of this paper are summarized below: as previously introduced, the pipeline dialogue system consists of four modules. the nlu module is to recognize the user's intents and the corresponding slot values. the dst module combines the previous state and the results of the nlu to update the current dialogue state. the dp module chooses the discrete dialogue acts according to the dialogue state and the database state to respond to the user. the nlg module generates the natural language based on the chosen dialogue acts. there are at least four kinds of annotation in such systems: the user's intent, the slot value, the dialogue state, and the dialogue act. the heavy annotation labor enormously increases the cost of building a pipeline system. its poor scalability further influences the pipeline dialogue system development. compared with the pipeline system, this paper's end-to-end task-oriented dialogue system only requires the annotated dialogue state. the end-to-end tod system is fed with the dialogue context c and generates the dialogue state b and delexicalized response r, where the database (db) state d is retrieved from the results searched with b. the delexicalized response means that the specific slot values are replaced with the corresponding slot placeholders. the lexicalized response is recovered from the delexicalized one with the generated dialogue state and db state. the training sample at each dialogue turn of the end-to-end tod model is defined as: for the task-oriented dialogue, the dialogue context not only consists of the dialogue history h but also includes the dialogue ontology schema s, which is usually ignored by the existing end-to-end models. the ontology can be seen as prior knowledge designed by the dialogue expert, which defines the dialogue domain, slots, and candidate values. the end-to-end tod model needs to fulfill two sub-tasks: dialogue state tracking (dst) and response generation (rg). formally, the learning goal of the tod model is to maximize the joint probability p θ (x), which can be factorized in an auto-regressive manner as: where the factorization from ( 3 ) to ( 4 ) is based on the fact that the database-lookup operation is a deterministic process. the p(h, s) is the prior probability of the paired dialogue and ontology (as the input of the model), which depends on the distribution of the (pre-)training data and is independent on the model. the dialogue state tracker intrinsically extracts the ontology-related constraints demanded by the user, where the ontology schema is given in advance. ## ontology-aware pretraining method the existing task-oriented dialogue data with the given ontology is limited to pretrain the language model. to increase the scale of the pretraining data, we divide the pretraining process into two phases. the first phase pretrains the model on the large-scale contextual text. the triples of the text are extracted by the latest neural-based open-ie6 @xcite . there is still a glaring discrepancy between the contextual text and the dialogue. for example, the dialogue always contains co-reference and information ellipsis @xcite . we pretrain the model on the smaller tod data at the second phase to further decrease the gap between the pretrained model and the downstream tasks. the two phases are complementary to each other introduced as below: phase-1: pretrained on contextual text in traditional dialogue pretrained models @xcite , the crawled reddit data is popular to be used as pretrained corpus. however, reddit data contain lots of the co-reference and information ellipsis, which seriously impact the performance of the external information extraction tool. different from the dialogue data, the co-reference and information ellipsis are infrequent in the contextual text of the wikipedia. 3 more details are shown in section 5.1 to validate the effects of pretrained corpora. we use the neural-based open-ie6 to extract the ontology-like knowledge of contextual text automatically. we directly simulate the extracted subject-relation-object triples as the domain-slot-value triples. as shown in figure 2 , the object values in the extracted ontology are masked during the pretraining process. one of our designed pretraining tasks is to recover the ontology-like triples (named ontology-like triple recovery [or]), which is similar to the dst task. to increase the inference ability of the pretrained model, we mask the next text (one or two sentences, which are randomly chosen) and push the model to infer the next text (named next-text generation [ntg]), which is similar to the rg task. thus, the pretraining sample is composed of four elements: masked ontology-like triples ŝ, the masked document context ĥ, ontology-like triples b, and the next text r. similar to equation 4, the goal of the pretaining model is to maximize the joint probability: to obtain the qualified triples of a sentence using openie6, we remove all the stopwords in the triples and filter the triples in which one of the triple components is a blank space. it is also the main reason that we do not choose the reddit at this pretraining phase. there are many pronouns in the text, with which is hard to extract qualified triples. this pretraining phase vastly increases the scale of the pretraining data. there are four steps to filter the triples of the sentence: ## analysis the analysis experiments evaluate the proposed opal on the end-to-end tod tasks to answer three main questions: q1: what role do the different pretraining corpora (wikipedia and outof-domain tod) play? q2: what is the main factor that affects the pretrained model? q3: does opal have a higher sample efficiency than the original bart in the limited-resource setting? ## conclusion and future work in this paper, we propose an ontology-aware pretraining method for modeling the end-to-end task-oriented dialogue. the scale of the existing task-oriented dialogue data is far from the need for the pretrained model. thus, we leverage the external tool openie6 in extracting the ontology-like knowledge of the large-scale contextual texts. to bridge the gap between the pretrained and end-to-end tod models, we design two adaptive self-supervised learning tasks: ontology-like triple recovery and next-text generation. the pretraining process is divided into two phases, where phase-1 pretrains on the large-scale ontology-aware contextual texts and phase-2 pretrains on the ontology-aware tod data. our proposed opal achieves excellent performance on the end-to-end tod tasks and dialogue state tracking tasks. in the future, we will evaluate the effect of the different ontology-building methods.