Few Shot Dialogue State Tracking using Meta-learning (2021)

Abstract: Dialogue State Tracking (DST) forms a core component of automated chatbot based systems designed for specific goals like hotel, taxi reservation, tourist information etc. With the increasing need to deploy such systems in new domains, solving the problem of zero/few-shot DST has become necessary. There has been a rising trend for learning to transfer knowledge from resource-rich domains to unknown domains with minimal need for additional data. In this work, we explore the merits of meta-learning algorithms for this transfer and hence, propose a meta-learner D-REPTILE specific to the DST problem. With extensive experimentation, we provide clear evidence of benefits over conventional approaches across different domains, methods, base models and datasets with significant (5-25%) improvement over the baseline in a low-data setting. Our proposed meta-learner is agnostic of the underlying model and hence any existing state-of-the-art DST system can improve its performance on unknown domains using our training strategy.

## introduction task-oriented dialogue (tod) systems are automated conversational agents built for a specific goal (for example hotel reservation). many businesses from wide-variety of domains (like hotel, restaurant, car-rental, payments etc) have adopted these systems to cut down their cost on customer support services. almost all such systems have a dialogue state tracking (dst) module which keeps track of values for some predefined domain-specific slots (example hotel-name, restaurant-rating etc) after every turn of utterances from user and system. these values are then used by natural language generator (nlg) to generate system responses and fulfill the user goals. many of the recent works @xcite @xcite have proposed various neural models that achieve good performance for the task but are data hungry in general. therefore, adapting to a new unknown domain (target domain) requires large amounts of domain-specific annotations limiting their use. however, given a wide range of practical applications, there has been a recent interest in data-efficient approaches. @xcite , @xcite used transformer @xcite based models which significantly reduce data dependence. further, @xcite model the problem as machine reading comprehension task and benefit from its readily available external datasets and methods. @xcite were first to propose transferring knowledge from one domain to another. since, many domains like restaurant and hotel share a lot of common slots like name, area, rating, etc and hence such a transfer proved to be effective for a low-resource domain. more recently, @xcite aimed at zero-shot dst using synthetic data generation for the target domain imitating data from other domains. recent meta-learning methods like maml @xcite , reptile @xcite have proven to be very successful in efficient and fast adaptations to new tasks with very few labelled samples. these methods specifically aim at the setting where there are many similar tasks but very small amount of data for each task. agnostic of the underlying model, these meta-learning algorithms spit out initialization for its parameters which when fine-tuned using low-resource target task achieves good performance. following their widespread success in few-shot image classification, there has been a lot of recent work on their merit in natural language processing tasks. @xcite ; @xcite ; @xcite ; @xcite ; @xcite ; @xcite attempt at using meta-learning for efficient transfer of knowledge from high-resource tasks to a low-resource task. further, some of the more recent works @xcite have shown meta-learners can be used for system response generation in tod systems which is generally downstream task for our dst task. to the best of our knowledge, ours is the first work exploring meta-learning algorithms for the dst problem. while prior work focused on training their models with a mixture of data from other available domains (train domains) followed by finetuning with data from target domain, we identify that this method of transferring knowledge between domains is inefficient, particularly in very low-data setting with just 0, 1, 2, 4 annotated examples from target domain. we, on the other hand, use train domains to meta-learn the parameters of the model used to initialize the fine-tuning process. we hypothesize that though different domains share many common slots, they can have different complexities. for some of the domains, it might be easier to train the model using very few examples while others may require large number of gradient steps (based on their different data complexity and training curves with 1%, 5%, 10% data in @xcite ). meta-learning takes into account that this gradient information and share it across domains. rather than looking for an initialization that try to simultaneously minimizes joint loss over all the domains, it looks for a point from which the optimum parameters of individual domains are reachable in few (< 5) gradient steps (and hence very few training examples for these steps). then the hope is that the target domain is similar to at least one of the train domains (for example hotel & restaurant or taxi & train) and hence the learned initialization will achieve efficient fine-tuning with very few examples for the target domain as well. this direction of limited data is motivated by practical applicability, where it might be possible for any developer to manually annotate 4-8 examples before deploying the chatbot for a new domain. we highlight the main contributions of our work below (i) we are the first to explore and reason about the benefits of meta-learning algorithms for dst problem (ii) we propose a meta-learner d-reptile that is agnostic to underlying model and hence has the ability to improve the state of the art performance in zero/few-shot dst for new domains. (iii) with extensive experimentation, we provide evidence of the benefit of our approach over conventional methods. we achieve a significant 5-25% improvement over the baseline in few-shot dst that is consistent across different target domains, methods, base models and datasets. ## methodology in this work, we propose d-reptile, a metalearning algorithm specific to dst task. following what @xcite did for dialogue generation problem, we treat different domains as tasks for the meta-learning algorithm. let @xmath0, d 2 , . . . d n } (eg. {restaurant, taxi, payment, . . .}) be the set of train domains for which we have annotated data available. let p d (.) define a probability distribution over these domains. let d d 1 , d d 2 . . . d dn be the training data from each of these domains. let m be any dst model with parameters θ m . let m be the task-batch size (number of domains in a batch in our case), α, β be the inner and outer learning rate respectively, k be the number of gradient steps. let sgd(.) be the function as defined in equation 1. borrowing the meta-learning theory regarding optimizing the objective equation 3 from @xcite , we define the algorithm d-reptile in algorithm 1. the update rule for initialization (as defined in step 8) is same as that of reptile. we chose reptile over other meta-learning algorithms because of its simplicity and computational advantages. nonetheless, its straight-forward to switch any other initialization based meta-learner by changing meta-update step. the novelty of our learner lies in its definition of the meta-learning tasks that represent different domains of dst problem. this algorithm aims to find θ in it m , which we use to initialize the model for the fine-tuning stage with the target domain. algorithm 1: d-reptile: meta learner for dst input: we argue that the meta-learned initialization are better suited for fine-tuning than conventional methods. in the hope that joint optimal parameters for train domains lie close to individual domains, @xcite initialize the fine-tuning stage of the target domain from the joint minimum of the loss from data from all the train domains (called naive pre-training before fine-tuning or nft here). more formally, they chose the following initialization such an initialization tries to simultaneously minimize the loss for all the domains which might be useful if the goal was to perform well on test data coming from mixture of these domains. however, here our goal is to perform well on a single unknown target domain and no direct relation between this initialization and the optimal parameters for the target domain can be seen. further, as the number of train domains increases or training data for each domain decreases, the joint optimum can be very far-off from the individual domainoptimum parameters. therefore, these methods perform particularly bad. we show empirical evidence for this hypothesis in section 4. on the other hand, if we optimize equation 3, we will reach a point in the parameter space from where all the domain-optimum parameters are reachable in k-gradient descent steps. therefore, we can hope to reach the optimum parameters for the target domain as well efficiently. this hope is much larger for dst problem specifically because of similarities in different related domains (specifically related slots as shown in section 5). let us consider the following example, let restaurant and taxi be two of the train domains. optimizing equation 3, we might reach a point which is closer to optimum parameters of restaurant domain than taxi domain if we have have smaller gradient values for restaurant data but large for taxi. however notably, both the optimum-parameters are reachable in k-gradient steps. now if target domain is hotel (similar to restaurant domain with common slots like rating, name, etc), we will already be close to its optimum parameters. also if the target domain is bus (similar to taxi domain with common slots like time, place, etc), we will have larger gradients in fine-tuning stage and thus will reach the optimum parameters for bus as well. this might not have been possible with equation 4 as the optimum parameters for joint of restaurant and taxi data might be very far from both the individual train domains and will also have no specific gradient properties for faster adaptation for any of hotel or bus target domains. ## ablation studies to validate our various theoretical hypothesis, search for hyper-parameters, clearly identify and reason about the situations where using metalearning helps dst, we perform additional analysis as written in subsections below. ## conclusion we conclude our analysis on the merits of metalearning as compared to naive pre-training for dst problem on a very positive note. given the practical applicability of very-low data analysis, we provide enough evidence to a developer of an automated conversational system for an unknown domain that irrespective of his/her model and target domain, d-reptile can achieve significant improvement (sometimes almost double) over conventional fine-tuning methods with no additional cost. with detailed ablations, we further provide insights on which slots and domains will particularly benefit from pre-traning strategies and which of those will require additional data. being agnostic to underlying model, our proposed algorithm has capability to push state-of-the-art in zero/few-shot dst problem, giving hope for expanding the scope of similar chatbot based systems in new businesses.