Knowledge-Grounded Conversational Data Augmentation with Generative Conversational Networks (2022)

Abstract: While rich, open-domain textual data are generally available and may include interesting phenomena (humor, sarcasm, empathy, etc.) most are designed for language processing tasks, and are usually in a non-conversational format. In this work, we take a step towards automatically generating conversational data using Generative Conversational Networks, aiming to benefit from the breadth of available language and knowledge data, and train open domain social conversational agents. We evaluate our approach on conversations with and without knowledge on the Topical Chat dataset using automatic metrics and human evaluators. Our results show that for conversations without knowledge grounding, GCN can generalize from the seed data, producing novel conversations that are less relevant but more engaging and for knowledge-grounded conversations, it can produce more knowledge-focused, fluent, and engaging conversations. Specifically, we show that for open-domain conversations with 10% of seed data, our approach performs close to the baseline that uses 100% of the data, while for knowledge-grounded conversations, it achieves the same using only 1% of the data, on human ratings of engagingness, fluency, and relevance.

## introduction conversational artificial intelligence has progressed a lot in the recent past, partly due to advances in large pre-trained language models (plm) and partly due to commercial conversational agents (alexa, siri, cortana, google assistant, and others). it is evident, however, that many challenges still remain, such as handling idioms, humour, expressing empathy, processing unstructured knowledge, and so on. one big factor for this is the lack of large and rich conversational data that include these complex aspects of human communication. while the research community is making great efforts in collecting such data (e.g. empathetic dialogues @xcite , persuasion @xcite , and others), these are still small compared to the amount of data needed to train deep neural networks. furthermore, these expensive data collections usually target a single phenomenon at a time, and hence do not necessarily scale to the richness of human conversations. another challenge for real world applications is privacy, preventing the use of much of the publicly available conversational data. in this work, we take a first step into automatically generating conversational data from unstructured textual knowledge (e.g. web sources) using generative conversational networks (gcn) @xcite . gcn is a meta-learning method initially proposed for intent detection and slot tagging; we extend that approach and demonstrate that we can learn how to generate responses grounded in unstructured knowledge. specifically, gcn learns how to generate labelled, diverse, and targeted data that are optimised with reinforcement learning (rl). this is achieved by using a generator model that produces new data which is used to train a separate learner model. the performance of the learner model is used as a reward signal to train the generator, so that over time the quality of the generated data increases. this reward signal can allow us to guide the data generation towards dimensions of interest, for example, knowledge-grounded, empathetic, or polite dialogues and can be derived from automatic metrics or human feedback if the system is deployed. in our case, the generator produces open-domain dialogues and the learner is a conversational agent that is trained on that data. selecting an appropriate reward signal can be difficult, since we want to generate good quality dialogues that do not exist in the training data, but dialogue evaluation is a challenging open problem. we therefore investigate a combination of multiple metrics that capture different aspects: bleu @xcite and rouge @xcite to ensure some similarity with the reference data, bertscore @xcite foot_0 to encourage good quality dialogues, and knowledge f1 foot_1 @xcite to encourage knowledge integration. it should be noted that while the focus in this work is knowledge grounding in open-domain response generation, our approach is extensible to other conversational phenomena with appropriate reward signals. our main contributions are: a) we generate knowledge-grounded conversational data from unstructured textual knowledge (e.g. the kind of knowledge available on the web); b) we improve response generation quality over a baseline that uses fine-tuning on seed data, eliminating the need for additional human-human data collection; and c) we demonstrate improved performance on knowledgegrounded response generation on topical chat, as measured by kf1 and human evaluations. ## related work language data augmentation approaches. there are a lot of recent works on data augmentation, but most of them are geared towards individual language processing tasks rather than training complete conversational agents. due to lack of space we only mention the ones that are most relevant to our work. protoda @xcite uses prototypical networks to augment data for intent classification while gensf @xcite uses dialogpt @xcite for zero-shot slot tagging; dino @xcite uses plm to generate data for semantic textual similarity; @xcite focus on zeroshot dialogue state tracking and use an abstract dialogue model to generate data. soloist @xcite uses a plm fine-tuned on large dialogue corpora and is designed for transactional (goal-oriented) dialogues. @xcite use plm to train user simulators from crowdgenerated conversations and their instructions. @xcite train domain-independent user simulators for transactional dialogues. @xcite augment data for data-to-text nlg by generating text in two steps: replacing values with alternatives and using gpt-2 to produce surface text. they then do automatic labelling and enforce cycle-consistency (make sure text can be generated from data and vice versa). @xcite focus on data generation for grammatical error correction and propose a method that can generate an erroneous sentence given a correct sentence and an error tag. @xcite use data augmentation to improve out of scope (oos) detection models. specifically, they extract utterances from a different dataset than the one they are targeting that can be labelled as oos and then do some smart filtering to select good candidates. @xcite propose neuralwoz, a framework to generate dialogue state tracking data given goal descriptions and api calls. neuralwoz has a data generator and a data labeler that annotates the data. gcn does not need a separate labeler model and has the added option of being continually trained with rl. promda @xcite is a soft-prompt learning method for low-resource nlp tasks, that addresses the problem of overfitting (memorizing) when fine-tuning a plm with a very small number of examples. the authors generate data for sequence classification and labelling. however, this approach is not tested on full dialogues which require significantly more context in the input. @xcite propose a three step method, where they first fine-tune a plm and then generate new data-points by adjusting the temperature of the generation. they then filter the generated data by putting a threshold on embedding similarity with respect to the target class centroid. gcn uses rl to guide the generation process, alleviating the need for explicit post-processing. @xcite present a data augmentation approach for aspect-based sentiment analysis that can generate data along two dimensions: aspects and polarity. the resulting data are then used in a contrastive learning setting to train a sentiment classifier. similarly to other approaches, it is not clear how it would perform in knowledge-grounded dialogue generation, with large inputs (context and available knowledge). for a more comprehensive review of data augmentation for language tasks, please see @xcite @xcite . regarding data augmentation for conversational agents, one of the most prominent methods is user simulation @xcite @xcite @xcite . these approaches, however, have been designed to work with well-structured databases whereas we are concerned with grounding open-domain conversational responses in unstructured knowledge. dg 2 @xcite focuses on data augmentation for document-grounded dialogues, using doc2dial @xcite . the authors use an agent bot and a user bot to conduct simulated conversations and generate data. however, unlike gcn, the bots are not continually updated and may not generalise well to produce novel content. the code was not available for a direct comparison on our dataset, however, in the few-shot learning experiments, they demonstrate good performance with as little as 25% of the data (869 doc2dial dialogues), whereas we demonstrate competitive performance by only using 1% of the training data (86 topical chat dialogues). few-shot approaches. another line of related work is based on few-/zero-shot transfer learning for dialogue tasks. again due to space we only mention the most relevant works. earlier studies have focused on improving the generalizability of natural language understanding problems such as intent classification @xcite and slot filling @xcite for unseen labels or domains. then, focus was placed on other dialogue problems including dialogue state tracking @xcite , next action prediction @xcite , and natural language generation (nlg) @xcite . ## notation we conduct experiments under two settings: conversations without explicit knowledge-grounding (we call them open-domain) and knowledgegrounded conversations. ## generative conversational networks gcn @xcite (figure 1 ) consist of two models in a meta-learning architecture: a data generator and a learner. the generator creates a labeled dataset that is used to train a new learner (a conversational agent in our case) in a supervised fashion. the learner is then evaluated on an external validation set and its performance is used as a proxy for the quality of the dataset. this quality measure is used as a reward in a rl setup that trains the generator. over time, the generator learns to create data of better and better quality, with respect to the learner's task, leading the learner to perform well. to avoid overfitting the validation set, we can limit the number of meta-iterations or include domain-independent performance metrics, such as fluency, perplexity, or even human feedback. when deployed, the generator is directly optimized on the test set (i.e. real interactions). both models can be pre-trained with seed data, if available, and paired with reward estimation, gcn can be used for continuous learning from user feedback. this approach has been proven to work well for intent detection and slot tagging in goal-oriented conversations @xcite and we here apply it to train social conversational agents. different from generative adversarial networks (goodfellow et al., 2014) 3 where the model tries to mimic the data, gcn models are guided by an external reward signal -that does not need to be differentiable -and can therefore generalize better. depending on the optimization criteria, we can set the direction towards which the models will go, for example more polite conversations, more technical terminology, different dialect, knowledge grounding, and even directions that are not easily quantifiable (e.g. engagingness ratings from humans). for open-domain conversations, as a proof of concept, we conduct few-shot experiments using 10% of the data and for knowledge-grounded conversations which is the main focus of this work, we use 1%, 5%, and 10% of the data; we call these the seed data (d seed ). at the beginning of training, we sample d seed from the data d, fine-tune the generator on d seed (see g.train(d seed ), line ## experiments to evaluate gcn as a data augmentation method for conversations with and without knowledge, we conduct few-shot experiments on topical chat (tc) @xcite . tc is a set of humanhuman conversations, without explicitly defined roles for each participant, collected over amazon mechanical turk. each participant had access to a set of facts or articles with some conversations being symmetric (participants had access to the same knowledge) and some being asymmetric. all experiments were conducted on 2 tesla v100 gpus with 32gb memory each. ## conclusion we presented a meta-learning method based on gcn to generate conversational data grounded on unstructured textual knowledge such as what can be found on the web. we show that given a small seed (1% of tc or 86 examples) our approach can generate high quality data that can be used to train a competitive conversational agent able to do knowledge selection and grounding. lower reference-based metric scores ( context hello, do you watch football? hi, yes. i love football! i did not realize the season lasted 17 weeks! yeah pretty long. i guess with 32 teams playing it takes that much time did you see they only rest one weekend during the season? that is crazy, so much work goes in there. did you know u of iowa painted their visitor locker room pink? oh yes,that is so funny. they really want to intimidate the other team, but is more like a joke wow, what an idea. did you know bowlers used to make more than football players in the 60s responses baseline that's a good point. i wonder how many of the top bowlers make twice as much as top football stars. do you have a favorite gcn-rl that's a good point. i'm not a big fan of football, but i do like to watch it.