D u NST : Dual Noisy Self Training for Semi-Supervised Controllable Text Generation (2023)

Abstract: Self-training (ST) has prospered again in language understanding by augmenting the fine-tuning of big pre-trained models when labeled data is insufficient. However, it remains challenging to incorporate ST into attribute-controllable language generation. Augmented only by self-generated pseudo text, generation models over-exploit the previously learned text space and fail to explore a larger one, suffering from a restricted generalization boundary and limited controllability. In this work, we propose DuNST, a novel ST framework to tackle these problems. DuNST jointly models text generation and classification as a dual process and further perturbs and escapes from the collapsed space by adding two kinds of flexible noise. In this way, our model could construct and utilize both pseudo text generated from given labels and pseudo labels predicted from available unlabeled text, which are gradually refined during the ST phase. We theoretically demonstrate that DuNST can be regarded as enhancing the exploration of the potentially larger real text space while maintaining exploitation, guaranteeing improved performance. Experiments on three controllable generation tasks show that DuNST significantly boosts control accuracy with comparable generation fluency and diversity against several strong baselines.

## introduction recently, pretrained language models (plm) @xcite @xcite have shown superiority in natural language processing (nlp). however, the ever-growing size of these models demands more training data, which destabilizes the fine-tuning of plms when labeled data is highly insufficient @xcite . in this case, selftraining (st) @xcite ; labeled data model unlabeled data pseudo data step 1: train a base model step 2: synthetic labeling step 3: select a subset top-k pseudo data step 4: update model updated model figure 1 : classic self-training (st) procedure. st trains a base classifier on a small labeled dataset, then iteratively predicts pseudo labels for unlabeled data to augment the original labeled training set, and finally fits the model to the augmented dataset. @xcite , a classical semisupervised paradigm, has come to the fore again. as depicted in fig. 1 , st produces pseudo labels for text using a classifier and then retrains the classifier with augmented data in an iterative process. by this means, st utilizes massive unlabeled text to denoise the pseudo-annotated neighbors and improve the generalization on real data @xcite , boosting various natural language understanding (nlu) tasks @xcite . nevertheless, how to apply st to natural language generation (nlg), especially the datahungry attribute-controllable nlg, remains an open question. different from typical nlu tasks like text classification, controllable nlg takes an attribute label as input to generate a textual sequence meeting the given attribute rather than predicting labels given input text. this brings two new challenges for st. challenge 1: since model inputs become discrete labels, there is no massive unlabeled data for the nlg model to extend the learned distribution boundary. challenge 2: augmented only by self-generated text, nlg models focus on exploitation and cannot explore a larger space. as a result, classic st merely works for a few nlg tasks, e.g., machine translation @xcite where adequate in-domain text exists, but fails to benefit controllable nlg. to handle these challenges, we propose a novel dual noisy self training (dunst), for semi-supervised controllable nlg. dunst jointly learns to generate text from given attribute labels and predict labels for text, characterizing these two directions as a dual variational generation process. such duality allows our model to leverage not only generated pseudo text but also pseudo labels predicted for available unlabeled text. both generation and classification would be augmented by the two kinds of pseudo data and thus gradually refined during the st process, handling challenge 1. besides, dunst incorporates two new types of flexible noise into generated pseudo text, namely softmax temperature and soft pseudo text, to further perturb and escape from the text space learned at the previous st iteration, which helps propagate local smoothness and enhance robustness @xcite , addressing challenge 2. our method can be theoretically regarded as exploring a larger potential space, thus facilitating an extended generalization boundary and improved attribute coverage, balancing exploration and exploitation better. hence, dunst could further boost controllability while maintaining comparable generation fluency and diversity. our code is available at https://github.com/peter fengyx/dunst . in summary, our contributions are as follows: self-training: recently, self-training has flourished again by iteratively generating pseudo labels and augmenting the tuning of data-hungry plms, showing great advantages in further enhancing nlu @xcite @xcite @xcite and neural machine translation (nmt) @xcite where massive unlabeled input text exists. beyond naive st, mukherjee and hassan awadallah (2020) select unlabeled instances based on the prediction uncertainty of the classifier to improve model confidence. jiao et al. (2021) also collect informative (high-uncertainty) monolingual sentences to enhance the translation quality of hard examples. relevant to our work, he et al. (2020) corrupts the pseudo target sentences in nmt by synthetic noise like token shuffle or paraphrase to propagate local smoothness. however, as mentioned in sec.1, due to challenges 1&2, it is challenging to apply these st methods to attribute-controllable nlg directly. ## end 3.3 dual noisy self-training as discussed in sec. 1, augmented only by selfgenerated text, the model would increasingly enhance the exploitation of the previously learned space but fail to explore more, resulting in constrained attribute distributions and thus marginal improvement of control accuracy (challenge 2, see to address this problem, we propose two novel and effective types of soft noise to enable safer exploration, namely high-temperature generation and soft pseudo text, in what follows. high-temperature generation (htg): we introduce temperature τ in the softmax layer: soft pseudo text (spt): htg improves the diversity of pseudo text, but also takes the risk of sampling invalid tokens and propagating errors in an autoregressive generation. moreover, htg produces discrete pseudo text (a point in text space) and thus requires numerous sampled pseudo text (points) to cover a small neighborhood (fig. 2 ). therefore, we further propose to generate soft pseudo text, where we directly store the output token distribution d and let g directly learn to reproduce d. then we replace eq.( 1 ) with: kl[p( ψ z|x, y)||q θ (z|y)], x, @xmath0. the complete algorithm is described in alg. 1. ## conclusion and future work we propose a novel dunst method to apply self-training to semi-supervised controllable nlg. dunst (1) jointly optimizes generation and classification via a dual variational learning framework to leverage both pseudo text and pseudo labels, and (2) incorporates two kinds of soft noise into st, better exploring larger potential text space and model generation ctr-pf at 20 years old, ichorous and unrepentant, ichanarito is a well-liked celebrity, but a poor quality actor, and a poor quality movie... st-(gpt2) at 20 years old, i grew up with it and had no expectations. i was a big fan of the series, i thought it was great, and i wanted to see more of it. but soon enough, i got bored and just got bored watching it. dunst at 20 years old, the poorest hour of my life had me scratching my head. this was an insult to the intelligence of mankind, and a complete disregard for anyone else's intelligence. do not watch this movie. it is a disgrace to any religion.