Jointly Improving Language Understanding and Generation with Quality-Weighted Weak Supervision of Automatic Labeling (2021)

Abstract: Neural natural language generation (NLG) and understanding (NLU) models are data-hungry and require massive amounts of annotated data to be competitive. Recent frameworks address this bottleneck with generative models that synthesize weak labels at scale, where a small amount of training labels are expert-curated and the rest of the data is automatically annotated. We follow that approach, by automatically constructing a large-scale weakly-labeled data with a fine-tuned GPT-2, and employ a semi-supervised framework to jointly train the NLG and NLU models. The proposed framework adapts the parameter updates to the models according to the estimated label-quality. On both the E2E and Weather benchmarks, we show that this weakly supervised training paradigm is an effective approach under low resource scenarios with as little as 10 data instances, and outperforming benchmark systems on both datasets when 100% of the training data is used.

## introduction natural language generation (nlg) is the task that transforms meaning representations (mr) into natural language descriptions @xcite ; while natural language understanding (nlu) is the opposite process where text is converted into mr @xcite . these two processes can thus constrain each other -recent exploration of the duality of neural natural language generation (nlg) and understanding (nlu) has led to successful semi-supervised learning techniques where both labeled and unlabeled data can be used for training @xcite @xcite @xcite . standard supervised learning for nlg and nlu depends on the access to labeled training data -a major bottleneck in developing new applications. in particular, neural methods require a large annotated dataset for each specific task. the collection process is often prohibitively expensive, especially when specialized domain expertise is required. on the other hand, learning with weak supervision from noisy labels offers a potential solution as it automatically builds imperfect training sets from low cost labeling rules or pretrained models @xcite @xcite . further, labeled data and large unlabeled data can be utilized in semi-supervised learning @xcite , as a way to jointly improve both nlu and nlg models. to this end, we target a weak supervision scenario (shown in figure 1 ) consisting of small, highquality expert-labeled data and a large set of unlabeled mr instances. we propose to expand the labeled data by automatically annotating the mr samples with noisy text labels. these noisy text labels are generated by a weak annotator, which is built upon recent works that directly fine-tune gpt-2 @xcite on joint meaning representation (mr) and text @xcite . then, we jointly train the nlg and nlu models in a two-step process with semisupervised learning objectives @xcite . first, we use pretrained models to estimate quality scores for each sample. then, we down-weight the loss updates in the back-propagation phase using the estimated quality scores. this way, the models are guided to avoid mistakes of the weak annotator. on two benchmarks, e2e @xcite and weather @xcite , we utilize varying amount of labeled data and show that the framework is able to successfully learn from the synthetic data generated by weak annotator, thereby allowing jointly-trained nlg and nlu models to outperform other baseline systems. this work makes the following contributions: learning with weak supervision. learning with weak supervision is a well-studied area that is popularized by the rise of data-driven neural approaches @xcite @xcite @xcite @xcite . our approach incorporates similar line of work, by providing noisy labels (text) with a fine-tuned lm which incorporates prior knowledge from general-domain text and data-text pair @xcite @xcite @xcite @xcite , and use it as the weak annotator, similar by functionality to that of fidelity-weighted learning @xcite , or data creation tool snorkel @xcite . learning with semi-supervision. work on semi-supervised learning considers settings with some labeled data and a much larger set of unlabeled data, and then leverages both labeled the unlabeled data as in machine translation @xcite , data-to-text generation @xcite or more relevantly the joint learning framework for training nlu and nlg @xcite . nonetheless, these approaches all assume that a large collection of text is available, which is an unrealistic assumption for the task due to the need for expert curation. in our work, we show that both nlu and nlg models can benefit from ( 1 ) automatically labeling mr with text, and (2) by semi-supervisedly learning from these samples while accounting for their qualities. ## approach we represent the set of meaning representation (mr) as x and the text samples as y. there are no restrictions on the format of the mr: each @xmath0, or can take the form of tree-structured semantic definitions as in @xcite . each text @xmath1. in our setting, we have (1) k labeled pairs and (2) a large quantity of unlabeled mr set x u where |x u | @xmath2. (we force @xmath3.) this is a realistic setting for novel application domains, as unlabeled mr are usually abundant and can also be easily constructed from predefined schemata. notably, we assume no access to outside resources containing in-domain text. the k annotations are all we know about in-domain text. the core of our approach consists of first labeling mr samples with text, and then training on the expanded dataset. we start with describing the process of creating weakly labeled data ( §4). next, we delve into the semi-supervised training objectives for the nlu and nlg models, which allow the models to learn from labeled and unlabeled data ( §5). lastly, we explain the training process where nlg and nlu models are jointly optimized in two steps: in step 1, we pretrain the models on the weakly-labeled corpus, then continue updating the models on the combined data consisting of the weak and real data in step 2. importantly, to account for the noise that comes with the automatic weak annotation, step 2 trains the model with quality-weighted updates ( §6). we depict this process in figure 2 . we construct synthetic data in two ways: (1) creating more mr samples (see §4.1), and (2) by creating a larger parallel set of mrs with texts (see §4.2). ## learning with weak supervision the primary challenge that arises from the synthetic data is the noise introduced during the generation process. noisy and poor quality labels tend to bring little to no improvements @xcite . to better train on the large and noisy corpus described in section §4 (size ~500k), we employ a two-step training process motivated by fidelity-weighted learning @xcite . the two-step process consists of (1) pretraining and (2) quality-weighted fine-tuning to account for the heterogenous data quality. step 1: pre-train two sets of models on weak and clean data, respectively. we train the first set of models (teacher) consisting of nlu, nlg, and autoencoder (auto) models on the clean data. the second set of models (i.e. nlu and nlg) is the student that pretrains on the weak data. step 2: fine-tune the student model parameters on the combined clean and weak datasets. we use each teacher model to determine the step size for each iteration of the stochastic gradient descent (sgd) by down-weighting the training step of the corresponding student model using the sample quality given by the teacher. data points with true labels will have high quality, and thus will be given a larger step-size when updating the parameters; conversely, we down-weight the training steps of the student for data points where the teacher is not confident. for this specific fine-tuning process, we update the parameters of the student (i.e. nlg and nlu models) at time t by training with sgd, where l(•) is the loss of predicting ŷ for an input x i when the label is ỹ. the weighted step is then c(x i , ỹi )∇l(ŷ, ỹ), where c(•) is a scoring function learned by the teacher taking as input mr x i and its noisy text label ỹi . in essence, we control the degree of parameter updates to the student based on how reliable its labels are according to the teacher. we denote c(•) as the function of the label quality based on the dual mutual information (dmi), defined as the absolute difference between mutual information (mi) foot_4 in inference directions @xmath4. @xcite shows that mi @xmath5, and vice versa. thus we expect the difference between mi @xmath6, y) to be relatively small compared to noisy samples, since the level of difficulty is largely proportional between nlu and nlg on the samplesdifficulty in inferring x from y will result in harder prediction of y from x. based on this intuition, the dmi score of the sample (x, y) is defined as: thus, c(•) defines the confidence (quality) the teacher has about the current mr-text sample. we use c(•) to scale η t . note that η t (t) does not necessarily depend on each data point, whereas c(•) does. we define c(x t , y t ) as: c(x t , y t ) = 1 -n (dmi(x t , y t )) where n (•) normalizes dmi over all samples in both clean and weak data to be in [0, 1]. 7 experiment setting data. we conduct experiments on the weather (balakrishnan et al., 2019) and e2e (novikova et al., 2017b) datasets. weather contains 25k instances of tree-structure annotations. e2e is a crowd-sourced dataset containing 50k instances in the restaurant domain. the inputs are dialogue acts consisting of three to eight slot-value pairs. configurations. both nlu and nlg models are implemented in pytorch (paszke et al., 2019) with 2 bi-lstm layers and 200-dimensional token embeddings and adam optimizer (kingma and ba, 2014) with initial learning rate at 0.0002. batch size is kept at 28 and we employ beam search with size 3 for decoding. the score is averaged over 10 random initialization runs. in our implementation, the sequence-to-sequence models are built upon the bi-directional long short-term memory (bi-lstm) @xcite . for lstm cells, both the encoder and decoder have 2 layers, amounting to 18m parameters for of labeled data from 10, 50 labeled instances to 1%, 5%, and 100% of the labeled data (dl). models that have access to unlabeled ground-truth text labels are marked with *. we provide results for the nlg and nlu models trained separately using supervised objectives alone (decoupled), our semi-supervised joint-learning model (joint), joint with all unlabeled data (joint+aug), and weakly-supervised models (step 1). step 1+2 denotes the full proposed approach. the seq2seq model. all models were trained on 1 nvidia v100 gpu (32gb and cuda version 10.2) for 10k steps. the average training time for seq2seq model was approximately 1 hour, and roughly 2 hours for the proposed semi-supervised training with 100% data. the total number of updates is set to 10k steps for all training and patience is set as 100 updates. at decoding time, sentences are generated using greedy decoding. ## results we first compare our model with other baselines on both datasets, then perform a set of ablation studies on the e2e dataset to see the effects of each component. finally, we analyze the strength of the weak annotator, and the effect of the qualityweighted weak supervision, before concluding with the analysis of dual mutual information. ## analysis error analysis. since word-level overlapping scores usually correlate rather poorly with human judgements on fluency and information accuracy @xcite , we perform human evaluation on the e2e corpus on 100 sampled generation outputs. for each mr-text pair, the annotator is instructed to evaluate the fluency (score 1-5, with 5 being most fluent), miss (count of mr slots that were missed) and wrong (count of included slots not in mr) are presented in mix y miy x 32.67°e 2e mix y miy x 33.09°w eather figure 5: visualization of dual mutual information (dmi) on both datasets where × markers are 50 random samples from annotated data and • markers are 50 random samples in the weak dataset. dotted lines are trend lines for • markers and solid black lines are diagonal reference that correspond to the perfect nlg-nlu balance where both tasks have equal difficulties. method nlg nlu bleu-4 (accuracy (%)) accuracy (%) (f1) w/ df 0.683 (77.69) 24.71 (0.6443) with df 0.703 (79.08) 27.19 (0.6840) with ws 0.733 (82.65) 30.23 (0.7028) with ws+cw 0.754 (86.44) 34.09 (0.7200) ## conclusion and future work in this paper, we show the efficacy of the framework where data is automatically labeled and both nlu and nlg models learn with quality-weighted weak supervision so as to account for the individual data quality. most importantly, we show that not only is the two-step training process useful in improving the model, it yields decent quality text. this work serves as a starting point for weaklysupervised learning in natural language generation, especially for topics related to instance-based weighting approaches. for future work, we hope to extend on the framework and propose ways with which it can be incorporated into existing text annotation systems.