Self-training Improves Pre-training for Few-shot Learning in Task-oriented Dialog Systems (2021)

Abstract: As the labeling cost for different modules in task-oriented dialog (ToD) systems is expensive, a major challenge is to train different modules with the least amount of labeled data. Recently, large-scale pre-trained language models, have shown promising results for few-shot learning in ToD. In this paper, we devise a self-training approach to utilize the abundant unlabeled dialog data to further improve state-of-the-art pre-trained models in few-shot learning scenarios for ToD systems. Specifically, we propose a self-training approach that iteratively labels the most confident unlabeled data to train a stronger Student model. Moreover, a new text augmentation technique (GradAug) is proposed to better train the Student by replacing non-crucial tokens using a masked language model. We conduct extensive experiments and present analyses on four downstream tasks in ToD, including intent classification, dialog state tracking, dialog act prediction, and response selection. Empirical results demonstrate that the proposed self-training approach consistently improves state-of-the-art pre-trained models (BERT, ToD-BERT) when only a small number of labeled data are available.

## introduction large-scale pre-trained language models, such as bert @xcite , unilm @xcite , gpt @xcite , gpt-2 @xcite , and @xcite , have shown great few-shot or zero-shot learning abilities in various nlp tasks with the help of task-agnostic language knowledge learned via pre-training tasks. in task-oriented dialog (tod) systems, the labeling cost is very high such that the size of well-labeled data is often small. therefore, few-shot learning in tod is important and valuable in many practical applications. many attempts @xcite have been proposed to leverage large-scale pre-trained language models to improve few-shot learning in tod. specifically, a model pretrained on general text corpora is further trained on public tod datasets. although the size of labeled data is often small, a practical tod system de facto has many unlabeled dialog data. therefore, utilizing unlabeled data to improve a tod system is practically important. in this paper, we take a semi-supervised self-training (st) perspective to iteratively train a better student model using unlabeled data @xcite . st has been successfully applied to a variety of tasks, including image classification @xcite @xcite , automatic speech classification @xcite @xcite , sequence generation @xcite , and natural language understanding @xcite . we are going to study this research question: can self-training provide complementary benefits on top of the strong pre-training models for few-shot learning in tod? recently, @xcite ; @xcite studied a similar question in the context of image classification, showing that st effectively refines pre-training models. @xcite also recently showed the benefit of st over pre-training for general natural language understanding. yet, their main proposal is to crawl a large amount of similar unlabeled data from the web. in this paper, we propose a self-training approach based on iterative pseudo-labeling @xcite . it first trains a teacher on the labeled samples. the teacher then iteratively generates pseudolabels for the most confident subset of unlabeled samples to train a better student. to train a more robust student during self-training, we propose a data augmentation technique called gradaug. gradaug first "masks" a fraction of tokens of a dialog input. then, it reconstructs the corrupted text with a pretrained masked language model of bert. different from @xcite , the probability of masking a token is conditioned on the gradient of the corresponding token embedding w.r.t. the downstream task. in this way, gradaug prevents replacing tokens that are critical for a downstream task. the main contribution of this paper is three-fold: recent studies have shown that large-scale pretrained language models are good few-shot learners @xcite . several studies have also confirmed these findings for tod. for the task of generating responses conditioned on a semantic representation, gpt-2 was leveraged by @xcite to improve few-shot learning. @xcite utilized gpt-2 for few-shot end-to-end response generation from dialog contexts. t5 @xcite is also recently applied to few-shot tod by @xcite for dialog state tracking and by @xcite for natural language generation. @xcite improve a bert model for few-shot learning on four downstream tasks. ## background of using pre-trained models for downstream tasks in tod in this section, we first briefly overview the pipeline of utilizing large-scale pre-trained models for four common downstream tasks (intent classification, dialog state tracking, dialog act prediction, and response selection) in tod. we denote the input and label of different downstream tasks as x and y, and a prediction model is denoted as ŷx = f (x). f can often be decomposed into two parts. the first part is a feature extractor @xmath0, and the second part is an output network for prediction. large-scale pre-trained language models serve as feature extractor a to compute a hidden representation for an input. for example, we use the [cls] embedding of bert as the hidden representation h when bert is adopted as a. different output networks are designed for different downstream tasks, and the details following tod-bert @xcite ) are described below. intent classification. this is a multi-class classification problem to predict the single intent label y of an input utterance x. the model computes the probability over i possible intents as: dialog state tracking. it is a multi-class classification problem based on a predefined ontology. unlike intent classification, the dialog history (a sequence of utterances) is used as the input x. for each (domain, slot) pair, the model predicts a score over all potential slot values. for the i-th slot value v j i of the j-th pair, the cosine similarity score compared to the input x is computed as follows: a threshold of 0.5 is applied during inference. response selection. this task predicts the most relevant system response from a candidate pool. a dual-encoder model @xcite ) is adopted to compute the similarity between the input dialog history x and the i-th candidate response c i : during training, we randomly sample 20 negative responses for each ground truth response. a crossentropy loss is applied aiming to rank the ground truth highest. ## self-training in this section, we introduce our self-training (st) algorithm. the overall st algorithm is introduced in section 4.1, and a new text augmentation method (gradaug) for st to train a more robust student is elaborated in section 4.2. ## conclusion we study using self-training to improve the strong pre-trained models for few-shot learning tasks in tod. an iterative self-training method with a new text augmentation technique (gradaug) is proposed to gradually train a stronger student model using unlabeled data. extensive empirical results on four downstream tasks in tod demonstrate the consistent improvements of self-training on top of pre-trained models. our findings on using selftraining to improve learning from limited labeled data may inspire future studies towards building more sample-efficient and scalable tod systems.