Dialogue Generation on Infrequent Sentence Functions via Structured Meta-Learning (2020)

Abstract: Sentence function is an important linguistic feature indicating the communicative purpose in uttering a sentence. Incorporating sentence functions into conversations has shown improvements in the quality of generated responses. However, the number of utterances for different types of fine-grained sentence functions is extremely imbalanced. Besides a small number of high-resource sentence functions, a large portion of sentence functions is infrequent. Consequently, dialogue generation conditioned on these infrequent sentence functions suffers from data deficiency. In this paper, we investigate a structured meta-learning (SML) approach for dialogue generation on infrequent sentence functions. We treat dialogue generation conditioned on different sentence functions as separate tasks, and apply model-agnostic meta-learning to high-resource sentence functions data. Furthermore, SML enhances meta-learning effectiveness by promoting knowledge customization among different sentence functions but simultaneously preserving knowledge generalization for similar sentence functions. Experimental results demonstrate that SML not only improves the informativeness and relevance of generated responses, but also can generate responses consistent with the target sentence functions. Code will be public to facilitate the research along this line.

## introduction humans express intentions in conversations through sentence functions, such as interrogation for acquiring further information, declaration for making statements, and imperative for making requests and instructions. for machines to interact with humans, it is therefore essential to enable them to make use of sentence functions for dialogue generation. sentence function is an important linguistic feature indicating the communicative purpose of a sentence in a conversation. there are four query 今天心情不错 i am in a good mood today sentence function positive declarative response 1 什么让你这么开心？ what makes you so happy? sentence function 1 wh-style interrogative response 2 我今天心情很不好 i feel bad today sentence function 2 negative declarative major sentence functions: declarative, interrogative, exclamatory and imperative @xcite . each major sentence function can be further decomposed into fine-grained ones according to different purposes indicated in conversations. for example, interrogative is divided into wh-style interrogative, yes-no interrogative and other types. these fine-grained sentence functions have great influences on the structures of utterances in conversations including word orders, syntactic patterns, and other aspects @xcite . figure 1 presents how sentence functions influence the responses. given the same query expressed in positive declarative, the responses expressed in wh-style interrogative and in negative declarative are completely different. although the use of sentence functions improves the overall quality of generated responses @xcite , it suffers from the data imbalance issue. for example, in the recently released response generation dataset with manually annotated sentence functions stc-sefun @xcite , more than 40% of utterances are positive declarative while utterances annotated with declarative with interrogative words account for less than 1% of the entire dataset. therefore, dialogue generation mod-els suffer from data deficiency for these infrequent sentence functions. recently, model-agnostic meta-learning (maml) @xcite has shown promising results on several low-resource natural language generation (nlg) tasks, including neural machine translation @xcite , personalized response generation @xcite and domain-adaptive dialogue generation @xcite . they treat languages of translation, personas of dialog and dialog domains as separate tasks in maml respectively. in the same spirit of previous works, we first treat dialogue generation conditioned on different sentence functions as separate tasks, and meta-train a dialogue generation model using high-resource sentence functions. moreover, we observe that sentence functions have hierarchical structures: four major sentence functions can be further divided into twenty fine-grained types. some fine-grained sentence functions may share some similarities while some others are disparate. for example, utterances belong to wh-style interrogative and yes-no interrogative may share some transferable word patterns while utterances in wh-style interrogative and in exclamatory with interjections totally differ from each other. motivated by this observation, we explore a structured meta-learning (sml) considering inherent structures among fine-grained sentence functions. inspired from recent advances on learning several initializations with a set of meta-learners @xcite , we develop our own approach to utilize the underlying structure of sentence functions. more specifically, our proposed sml explicitly tailors transferable knowledge among different sentence functions. it utilizes the learned representations of fine-grained sentence functions as parameter gates to influence the globally shared parameter initialization. therefore, conversation models for similar sentence functions can share similar parameter initializations and vice versa. as a result, sml enhances meta-learning effectiveness by promoting knowledge customization among different sentence functions but simultaneously preserving knowledge generalization for similar sentence functions. the experimental results on stc-sefun dataset @xcite show that responses generated from our proposed structured meta-learning algorithm are of better quality over several baselines in both human and automatic evaluations. moreover, our proposed model can generate responses consistent with the target sentence functions while baseline models may ignore the target sentence functions or generate some generic responses. we further conduct a detailed analysis on our proposed model and show that it indeed can learn word orders and syntactic patterns for different fine-grained sentence functions. meta-learning for low-resource nlg. humans can learn quickly with a few examples while data-driven models are mostly compute-intensive. in meta-learning, the goal of the trained model is to quickly learn a new task from a small amount of data. therefore, the model should be able to learn transferable knowledge on a large number of different tasks. recently, model-agnostic metalearning (maml) @xcite has shown promising results on several few-shot classification tasks. maml directly optimizes the gradient towards a good parameter initialization for easy fine-tuning on low-resource scenarios. because of the model-agnostic nature of maml, it can be directly applied to low-resource nlg tasks with modifications on corresponding training strategies. @xcite frame machine translation between two language pairs as a single task in meta-learning, and learn to adapt to low-resource languages based on multilingual high-resource language tasks. in a similar spirit, recent works apply maml to personalized response generation @xcite and task-oriented dialogue agents @xcite . in this paper, we not only investigate how maml helps for open domain dialogue generation on infrequent sentence functions, but also develop a structured approach to fit the hierarchical structure of sentence functions. ## problem formulation we define response generation conditioned on every query-response sentence function pair (d x , d y ) as a single task. as the number of utterances for different sentence functions is extremely imbalanced, some tasks have abundant utterances while some others are low-resource. we take k high-resource tasks as training data, denoted as: then, we take t tasks with infrequent sentence functions as target tasks, denoted as: the initialization parameters of model f learned from the training process, denoted by θ 0 , are used as the initialization parameters in the adaptation process. the adaptation process on each target task d t target can be formulated as follows: ## proposed approach in this section, we first introduce the conditional sequence-to-sequence (c-seq2seq) model for open domain dialogue generation with fine-grained sentence function. then we describe how to meta-train c-seq2seq under the algorithm of model-agnostic meta-learning. finally, we explore the structure of fine-grained sentence functions and propose the structured meta-learning (sml) algorithm. ## results performance of human evaluation. human evaluation on "fluency" (flue), "relevance" (rele), "informativeness" (info) and "accuracy" (accu) are shown in this tells us exploring the structure of sentence functions can balance knowledge generation and knowledge customization. the task-specific initialized model can leverage the knowledge of similar tasks and thus adapts the target tasks better. performance of automatic evaluation. we also show the results of automatic evaluations in compared to transfer learning based model mtl+ft, our meta-learning based models adapt faster (lower fine-tune step) and better (lower perplexity). although bleu is not reliable enough to evaluate response generation, maml and sml still achieve slightly better results than baselines. presumably, they can capture frequent word pattern in low-resource tasks. finally, mtl+ft, maml and sml achieve comparable performance with regard to the unigram/bigram diversity (dist1/2) of generated responses. effect of structure modeling. to get more insight into how our proposed sml balances the knowledge generalization and customization, in figure 3 , we visualize the heatmap of self-attention weight a k in eqn. 12 for all 9 training sentence function representations. each row in figure 3 demonstrates the similarity between sentence function s k and all sentence functions [s 1 ; ...; s k ]. take the first row in figure 3 for example, it tells us how all nine query-response sentence function pairs contribute to the representation of current query-response sentence function pair "positive declarative (de) | positive declarative (de)". we see that sentence functions containing interrogative(in) have nearly zero contribution while sentence functions containing declarative(de) in both query and response have a certain amount of contribution. therefore, similar sentence functions trig-query (am i stupid for not passing the 4th part of the driving-test?) query sf a-not-a interrogative target response sf negative declarative mtl (so do i) mtl+ft (i have not taken a make-up driving-test) maml (i have not taken the driving-test yet) sml (i did not pass too.) query (i register for a marriage today) query sf positive declarative target response sf yes-no interrogative mtl (congratulations) mtl+ft (congratulations to you) maml ( you held the wedding ceremony ) sml ( you hold the wedding ceremony ) figure 4: responses of all models. words in red are related to the target sentence function. sf: sentence function. ger similar initializations and dissimilar sentence functions trigger different ones. case study. we present two examples in figure 4, each of which shows a test query with the target sentence function and responses generated by all models. we see that responses generated by mtl are generic and can be used to reply to a large number of queries. with fine-tuning on responses of the target sentence function, mtl+ft can capture the correct response pattern in some cases. however, it is inferior to our proposed models maml and sml, which can not only generate words related to the target sentence function but also keep the coherence and informativeness of responses. 7 conclusion in this paper, we propose a structured meta-learning algorithm for open domain dialogue generation on infrequent sentence functions. to tackle the lowresource issue, our proposed model, based on the recently proposed model-agnostic meta-learning, can find both transferable internal representations and sensible parameters which can produce large improvement under a few adaptation steps. moreover, we further explore the structure across fine-grained sentence functions and such that the model can balance knowledge generalization and knowledge customization. extensive experiments show that our structured meta-learning (sml) algorithm outperforms existing approaches under the lowresource setting.