On the Compositional Generalization in Versatile Open-domain Dialogue (2023)

Abstract: Previous research has demonstrated the potential of multi-task learning to foster a conversational agent’s ability to acquire a variety of skills. However, these approaches either suffer from interference among different datasets (also known as negative transfer), or fail to effectively reuse knowledge and skills learned from other datasets. In contrast to previous works, we develop a sparsely activated modular network: (1) We propose a well-rounded set of operators and instantiate each operator with an independent module; (2) We formulate dialogue generation as the execution of a generated programme which recursively composes and assembles modules. Extensive experiments on 9 datasets verify the efficacy of our methods through automatic evaluation and human evaluation. Notably, our model outperforms state-of-the-art supervised approaches on 4 datasets with only 10% training data thanks to the modular architecture and multi-task learning.

## introduction building an open-domain dialogue system is an intriguing and challenging task. a good open-domain chatbot should be equipped with a well-rounded set of skills @xcite including but not limited to providing an informative response, showing different emotions, keeping a consistent persona and conducting commonsense inference. up to now, with more and more datasets proposed to train multiple conversation skills (e.g., wizard of wikipedia @xcite , personachat @xcite ), multi-task learning is an efficient way to grasp all the versatile skills and quickly transfer to newly emerging datasets @xcite . however, as a core problem in multi-task learning, it is not easy to strike a balance between transfer and interference (negative transfer) among multiple datasets @xcite . for this, recent researchers mainly follow two lines of research. on one line, @xcite and @xcite simply mix all the datasets together to embody the blended skill required in dialogue. they update all the model parameters to minimize the loss of all of the data, which is also dubbed dense training @xcite . in spite of its simplicity, it easily incurs interference among different datasets @xcite . on the other line, @xcite learn multiple skills and store the knowledge from different datasets with different sets of parameter-efficient architectures. this approach eliminates underlying negative transfer among different corpora, but hinders positive transfer at cost. the model has to learn from scratch rather than reuse past knowledge every time a new corpus comes. inspired by recent advancements in neuroscience @xcite suggesting that the human brain represents knowledge in a modular way, we incorporate this as an inductive bias and present a compositional modular architecture to balance transfer and interference @xcite . by decomposing the knowledge for dialogue into relative independent modules @xcite , a neural model thus decides which module to invoke for different tasks or different samples. however, there are two challenges in applying modular architecture to building a versatile open-domain chatbot. first, the generation task is different from question answering, where the neural module network accomplishes impressive performance @xcite @xcite . it is untouched how to apply the ideology of modularity to the auto-regressive generation process. second, the modules used in neural module network @xcite are typically trained with end-task supervision. without intermediate supervision or specialized training data for each module @xcite , modules might perform homogeneous functions rather than perform their predefined functions as intended @xcite . to deal with the above problems, in this paper, we present a neural modular framework for blended-skill dialogue generation. the principle of our approach is to decompose the generation process into the recursive execution of basic operators by various modules. specifically, (1) as an attempt to conduct generation tasks in a modular way, we introduce content modules for basic content synthesis and linguistics modules for linguisticallyrelated surface realization. in addition, a programmer is trained to produce a reverse-polish-style code @xcite which schedules the modules to produce the final response. (2) to overcome the homogeneity of modules, we construct pseudo labels and provide weak supervision signals to facilitate the training of each module. since the output of the programmer and the modules are discrete and thus not differentiable, we employ gumbel-softmax trick to produce "soft" sentences as the output of modules at training time, and employ reinforcement learning to bridge the gap between the programmer and the modules. extensive experiments are conducted on 9 opendomain datasets. our approach surpasses other models on a similar parameter scale and achieves a new state-of-the-art by multi-task training on all the 9 corpora. notably, our model outperforms state-of-the-art supervised approaches on dailydialog, empatheticdialog, light and cornell movie with only 10% training data, demonstrating that our modular framework could compose existing skills more efficiently to attain superior performance on out-of-distribution data. ## preliminary for open-domain dialogue generation, each datum can be thought of a pair (x, y), where y is the response and x is the dialogue context composed of history utterances and other external resources such as background knowledge (cmu_dog @xcite ), persona of speakers (convai2 @xcite al., 2018) based on the generated code, different modules are activated to perform different functions. the execution of the code produces a response in the end. the workflow of our framework is shown in figure 1 . the rest of our paper is structured as follows. we illustrate the modular architecture in §4, including the implementation of the programmer and execution of the code with modules. in §5, we elaborate the training algorithm to cope with the paucity of human annotation and discrete optimization. the experiment results and further analysis are displayed in §6 and §7 respectively. ## modular framework in this section, we elaborate on how the modular generation framework works. briefly, a programmer first generates a code in a special reversepolish-style programme language. then we execute the code with a stack to store the intermediate result. when encountering some specific operators in code, we just activate corresponding modules to fulfill the function of the operator. ## experiment 6.1 experimental setup setting. to comprehensively evaluate the multitask learning ability and the generalization ability, suppose we have n datasets, we evaluate our proposed framework in three settings: (1) all task mtl. in this setting, we train our model on the mixed union of n datasets and evaluate it on each individual dataset. (2) leave-one-out. in this setting, we train our model on n -1 datasets and test on the left one dataset to evaluate a model's zero-shot generalization ability. (3) low-resource. to further evaluate the generalization capability of our method, after training on other n -1 datasets in the leave-one-out setting, we fine-tune the model on the left dataset with only 10% data available, and test the model on the left one dataset. datasets. we use @xmath0: cornell movies (danescu-niculescu-mizil and lee, 2011), dailydialog @xcite , cmu_dog @xcite , light @xcite , empatheticdialog (rashkin et al., 2019), convai2 @xcite , wizard of wikipedia @xcite , mutual @xcite and commonsensedialog @xcite . each dataset embodies one or more specific skills. more details about the datasets could be found in appendix b. baselines. we use bart @xcite as one of our baselines, which is a standard sequenceto-sequence transformer pre-trained on the same corpus as @xcite ; we also compare against r2c2, a blenderbot-like open-domain dialogue model trained in a multi-task way by @xcite and hold the current stateof-the-art on many datasets @xcite . for parameter-efficient technique in multi-task learning, we compare our method with prefixtuning @xcite . we also draw a comparison with the recent proposed modular skill (ms) @xcite , a modular network that allows each task to choose its skill toolkit and optimize the global skill inventory together with the choice of each task jointly. for a fair comparison, we use bart-large ( 406m) and r2c2-base ( 400m) in our experiments. the parameter scale of prefix-tuning ( 415m) and ms ( 448m) are both comparable with ours. ## conclusions in this work, we utilize the ideology of modular networks to address the transfer-interference prob-lem in multi-task learning. we implement a model architecture that allows the composition of different modules to fulfill complicated functions and eliminate interference among modules. we apply our method to dialogue generation and conduct extensive experiments to verdict its efficacy. we hope our work would inspire relevant research in the community.