Low-Resource Adaptation of Open-Domain Generative Chatbots (2022)

Abstract: Recent work building open-domain chatbots has demonstrated that increasing model size improves performance (Adiwardana et al., 2020; Roller et al., 2020). On the other hand, latency and connectivity considerations dictate the move of digital assistants on the device (Verge, 2021). Giving a digital assistant like Siri, Alexa, or Google Assistant the ability to discuss just about anything leads to the need for reducing the chatbot model size such that it fits on the user’s device. We demonstrate that low parameter models can simultaneously retain their general knowledge conversational abilities while improving in a specific domain. Additionally, we propose a generic framework that accounts for variety in question types, tracks reference throughout multi-turn conversations, and removes inconsistent and potentially toxic responses. Our framework seamlessly transitions between chatting and performing transactional tasks, which will ultimately make interactions with digital assistants more human-like. We evaluate our framework on 1 internal and 4 public benchmark datasets using both automatic (Perplexity) and human (SSA – Sensibleness and Specificity Average) evaluation metrics and establish comparable performance while reducing model parameters by 90%.

## introduction recent progress on end-to-end neural approaches for building open-domain chatbots @xcite @xcite has demonstrated that large-scale pre-training using heavy-weight models combined with careful selection of datasets for fine-tuning to acquire specific skills can deliver superior performance. however, ⭑ equal contribution. ♥ work done while at apple. for one model to perform several tasks -such as dialogue state tracking or reference resolution, response generation, mitigating toxic responses, avoiding in-turn contradictions, and avoiding incorrect or "i don't know" responses due to lack of knowledge -in a reliable fashion, there is still a long way to go. despite much research, these limitations from the recently proposed approaches prevent practical adoption. in addition, due to huge model sizes, these approaches lack practical utility in a low-resource setting. some complex frameworks @xcite @xcite use a mix of templates and dialogue managers with rulebased systems. these complex frameworks often have problems: the produced responses are vague and generic, and they lack engagingness @xcite . other complex frameworks address this issue by employing modularizing design assigning each conversational task to a specific component, which can help improve overall performance of the dialogue systems @xcite . prior works have shown that generative neural response models outperform template-based or hybrid response generation methods as measured using various human evaluation techniques @xcite . in this work, we propose a generic, modular and light-weight framework that blends the desired characteristics of both classes of methods. a snippet of sample dialogue with our proposed framework is shown in figure 1 . our contributions are as follows: (1) demonstrating that a light-weight response generation model in a modular framework achieves comparable performance to recent models @xcite that have billions of parameters; (2) providing evidence that adding a reference resolution component improves the quality of the generated response for multi-turn conversations, compared to previous approaches that state track conversational context explicitly or use latent representations @xcite ; (3) providing a generic end-to-end framework that can process both objective (factual) and subjective questions. ## lightweight entertainment domain chatbot lightweight entertainment domain (led) chatbot interacts with the user through a pipeline of models. the led chatbot architecture is illustrated in figure 2 . each module in our pipeline architecture handles a specific conversational task and passes the output for further processing to the downstream modules. in the following subsections, we describe these modules with their respective tasks and training details. ## training data we use various datasets for training and evaluation focused on different tasks. in this section, we describe each dataset along with the corresponding modules that use the dataset for training. qrecc @xcite ) contains around 81,000 conversation turns. every turn contains a question which may have anaphoric references, a rewritten version of the question with references resolved, an answer span to the question and a corresponding web url. qrecc data is used to train the reference resolution, passage retrieval and answer span extraction models. wizard of wikipedia @xcite ) (wow) contains 194,000 turns of dialogue distributed over 1,250 topics. each conversation is predicated on discussing the relevant topic in depth, with the goal of displaying expert knowledge in that subject. note that in our pipeline framework, we refer objective questions to the extractnparaphrase component, so the subjective response generation model is not required to answer factual questions with a high degree of accuracy. still, the wow dataset helps our generative model maintain a breadth of knowledge to provide pertinent answers to subjective inputs. convai2 is based off of the work of per-sonachat @xcite and was used at the neurips 2018 convai competition. this dataset is made up of 140,000 turns where gatherers are given a persona and tasked with learning about their counterpart. this helps opendomain agents ask questions, and perhaps more relevantly in our use case, respond in an engaging manner. we use the convai2 dataset to train the subjective response generation model. empathetic dialogues (rashkin et al., 2019) (ed) is a library of 50,000 turns where one speaker plays the role of sympathetic listener. these skills translate well to our needs, as the subjective model must account for previous dialogue history and attempt to match their chosen response to the appropriate tone. blended skill talk (smith et al., 2020) (bst) is a 76,000 turn compilation of the previous three datasets: wow, convai2, and ed. guided human speakers were given the option to select between outputs from models trained on each of the individual tasks, which produces data that can teach the bot when a certain class of response should be used. decode @xcite ) is a conversational dataset made up of 40,280 turns from human to human and human to bot contradictory dialogues. we use decode to train the inconsistency/toxicity detector model based off of the albert model, along with our internal media dataset. internal media dataset is composed of 100,000 movie themed turns. each turn contains a natural question without explicit reference to the movie being discussed, as well as rewritten questions that convert those references to specifics (akin to the reference resolution component of our pipeline). answer span along with web url as well as paraphrased variation that is natural and engaging is also provided. the dataset is collected using crowd-sourced annotators. the goal of the annotators is to mimic the flow of a natural human conversation while maintaining a neutral persona. the responses were validated against guidelines to be non-controversial, eliminate profanity, be neutral, engaging and concise (with an upper bound of 30 words). every conversation consists of 10 turns, and we collect 10,000 conversations. we give instructions to explicitly add anaphoric references in follow up turns. ## evaluation metrics and results we categorize our evaluation metrics based on component-wise vs end-to-end evaluation. qrecc and decode datasets are only used for taskspecific model training and are not used in establishing a chatbot's end-to-end metrics: perplexity and sensibleness and specificity average (ssa). we establish a human evaluation metric, ssa, on our internal media dataset only, due to limited human annotators. we establish the automatic evaluation metric, perplexity, on all 5 datasets: wow, convai2, ed, bt, and our internal media dataset. below we discuss the intrinsic (component-wise) and extrinsic (end-to-end) metrics used to evaluate our led framework. ## related work our work follows the objective of combining opendomain chatbot and transactional digital assistants. the factual classifier component of led serves as the gatekeeper between these two categories, sending objective asks through the extractnparaphrase model and subjective inputs through our fine-tuned open domain model. while our work broadly falls under the category of open-domain generative chatbots, because of the variety of models and their corresponding tasks, our work also covers multiple key areas in language understanding with a focus on low-resource adaptation design. prior works @xcite @xcite have shown that end-toend neural approaches, where the responses are produced in a generative fashion, can result in engaging dialogue. however, the resultant models from these approaches are huge -multiple billions of parameters -and are not on-device friendly. it has also been shown that end-to-end generative chatbots frequently generate responses with inconsistencies @xcite . it is obvious that there is need for an additional module that can correct, or at least detect, these inconsistencies. generalizing this approach where we assign a specific task to a module, modularization can lead to overall improvement in dialogue systems @xcite . we adopt the modularization approach to opendomain generative chatbot to minimize the total number of parameters while tackling some of the shortcomings in the end-to-end neural approaches. blender @xcite showed non-trivial improvement in response generation when evaluated using human side-by-side comparison. we adopt the blender model as a basis for the core response generation functionality in subjective cases. we follow the blender methodology of experimenting with multiple decoding algorithms for optimal human performance. however we also differ from blender's approach. firstly, we place a larger emphasis on model size for better on-device compatibility. secondly, we account for a wider variety of cases where we use answer extraction and paraphrasing to accurately answer factual questions. and finally, we use the reference resolution component to track dialogue state since it is helpful for multi-turn conversations @xcite , along with providing our fine-tuned model with a wider variety of training data (multi-turn conversations where questions are either rewritten or preserved). meena @xcite proposed a new metric, sensibleness and specificity average (ssa), which captures key elements of a humanlike multi-turn conversation. additionally, they also show perplexity is the best automatic metric that correlates well with human judgement. we borrow ssa to evaluate human performance. it is good for our use case, where the model is required not just to answer logically but should also be rewarded for referencing context from earlier in the conversation. one of the differences between our work and meena is we do not use evolved transformer layers, though that may be basis for future work. one difference of our work compared to both blender and meena is we follow a modularized approach, instead of a single parameter-heavy model.