T ransfer TOD : A Generalizable C hinese Multi-Domain Task-Oriented Dialogue System with Transfer Capabilities (2024)

Abstract: Task-oriented dialogue (TOD) systems aim to efficiently handle task-oriented conversations, including information collection. How to utilize TOD accurately, efficiently and effectively for information collection has always been a critical and challenging task. Recent studies have demonstrated that Large Language Models (LLMs) excel in dialogue, instruction generation, and reasoning, and can significantly enhance the performance of TOD through fine-tuning. However, current datasets primarily cater to user-led systems and are limited to predefined specific scenarios and slots, thereby necessitating improvements in the proactiveness, diversity, and capabilities of TOD. In this study, we present a detailed multi-domain task-oriented data construction process for conversations, and a Chinese dialogue dataset generated based on this process, **TransferTOD**, which authentically simulates human-computer dialogues in 30 popular life service scenarios. Leveraging this dataset, we trained a model using full-parameter fine-tuning called **TransferTOD-7B**, showcasing notable abilities in slot filling and questioning. Our work has demonstrated its strong generalization capabilities in various downstream scenarios, significantly enhancing both data utilization efficiency and system performance. The data is released in https://github.com/KongLongGeFDU/TransferTOD.

## introduction the task-oriented dialogue (tod) system is a human-computer interaction system aims to aid users in accomplishing specific tasks or acquiring particular information, which has found extensive use in daily life and commercial applications. at present, tod systems have displayed the capability to adapt effectively to diverse tasks, domains, i am 23 years old. please tell me your address. can you tell me your age? "name": tom "age": none "address": none "name": tom "age": 23 "address": none ## related work task-oriented dialogue datasets the performance of intelligent dialogue systems is profoundly influenced by the quality of the dialogue datasets, making dataset construction an active research area. initial generations of task-oriented dialogue datasets often focused on a single task or even a single scenario, atis @xcite , dstc2 @xcite , woz2.0 @xcite , etc. included. the emergence of these databases not only enhanced the conversational fluency of conversational agents but also made task completion through natural dialogues between machines and humans possible. considering that user dialogues often involve domain transitions, datasets multi-woz (budzianowski et al., 2020), crosswoz @xcite etc. encompassing more scenes and larger volumes of data were subsequently proposed. however, these dialogues are user-led discussions on relevant topics, requiring a user to pose questions or set tasks for the dialogue agent to respond accordingly. ## experiment in this section, we detail the experiments conducted. the primary experiments were carried out on the test set of transfertod. additionally, we conducted ablation studies on the dataset construction phase, as well as supplementary experiments to further investigate the effects of secondary finetuning. ## conclusion empirical evidence substantiates that our transfer-tod dataset possesses substantial noise resilience and superior linguistic performance. utilizing this dataset for supervised fine-tuning, the resultant model, designated transfertod-7b, attains a joint accuracy of 75.09% in out-of-domain evaluations, accompanied by a slot f1 of 96.20%. when it comes to question-asking ability, the accuracy of transfertod-7b is only slightly inferior to gpt-4, whereas its fluency in generating questions surpasses all other models we tested. furthermore, our findings suggest that appropriate secondary fine-tuning of the transfertod-7b model can further enhance its generalization capabilities. by employing a small portion of the out-of-domain test set for secondary fine-tuning, the resulting model surpasses the performance of gpt-3.5-turbo, which was fine-tuned with an equivalent amount of data. in summary, we have proposed a highly versatile data construction process that enhances the quality of task-oriented dialogue data for information collection tasks. the models fine-tuned with this data exhibit strong generalization capabilities, performing well in out-of-domain scenarios.