GAN - BERT : Generative Adversarial Learning for Robust Text Classification with a Bunch of Labeled Examples (2020)

Abstract: Recent Transformer-based architectures, e.g., BERT, provide impressive results in many Natural Language Processing tasks. However, most of the adopted benchmarks are made of (sometimes hundreds of) thousands of examples. In many real scenarios, obtaining high- quality annotated data is expensive and time consuming; in contrast, unlabeled examples characterizing the target task can be, in general, easily collected. One promising method to enable semi-supervised learning has been proposed in image processing, based on Semi- Supervised Generative Adversarial Networks. In this paper, we propose GAN-BERT that ex- tends the fine-tuning of BERT-like architectures with unlabeled data in a generative adversarial setting. Experimental results show that the requirement for annotated examples can be drastically reduced (up to only 50-100 annotated examples), still obtaining good performances in several sentence classification tasks.

## introduction in recent years, deep learning methods have become very popular in natural language processing (nlp), e.g., they reach high performances by relying on very simple input representations (for example, in @xcite @xcite ). in particular, transformer-based architectures, e.g., bert @xcite , provide representations of their inputs as a result of a pre-training stage. these are, in fact, trained over large scale corpora and then effectively finetuned over a targeted task achieving state-of-the-art results in different and heterogeneous nlp tasks. these achievements are obtained when thousands of annotated examples exist for the final tasks. as experimented in this work, the quality of bert fine-tuned over less than 200 annotated instances shows significant drops, especially in classification tasks involving many categories. unfortunately, obtaining annotated data is a time-consuming and costly process. a viable solution is adopting semisupervised methods, such as in @xcite @xcite to improve the generalization capability when few annotated data is available, while the acquisition of unlabeled sources is possible. one effective semi-supervised method is implemented within semi-supervised generative adversarial networks (ss-gans). usually, in gans @xcite a "generator" is trained to produce samples resembling some data distribution. this training process "adversarially" depends on a "discriminator", which is instead trained to distinguish samples of the generator from the real instances. ss-gans @xcite are an extension to gans where the discriminator also assigns a category to each example while discriminating whether it was automatically generated or not. in ss-gans, the labeled material is thus used to train the discriminator, while the unlabeled examples (as well as the ones automatically generated) improve its inner representations. in image processing, ss-gans have been shown to be effective: exposed to few dozens of labeled examples (but thousands of unlabeled ones), they obtain performances competitive with fully supervised settings. in this paper, we extend the bert training with unlabeled data in a generative adversarial setting. in particular, we enrich the bert fine-tuning process with an ss-gan perspective, in the so-called gan-bert 1 model. that is, a generator produces "fake" examples resembling the data distribution, while bert is used as a discriminator. in this way, we exploit both the capability of bert to produce high-quality representations of input texts and to adopt unlabeled material to help the network in generalizing its representations for the final tasks. at the best of our knowledge, using ss-gans in nlp has been investigated only by @xcite with the so-called kernel-based gan. in that work, authors extend a kernel-based deep architecture (kda, @xcite ) with an ss-gan perspective. sentences are projected into low-dimensional embeddings, which approximate the implicit space generated by using a semantic tree kernel function. however, it only marginally investigated how the gan perspective could extend deep architecture for nlp tasks. in particular, a kgan operates in a pre-computed embedding space by approximating a kernel function @xcite . while the ss-gan improves the quality of the multi-layered perceptron used in the kda, it does not affect the input representation space, which is statically derived by the kernel space approximation. in the present work, all the parameters of the network are instead considered during the training process, in line with the ss-gan approaches. we empirically demonstrate that the ss-gan schema applied over bert, i.e., gan-bert, reduces the requirement for annotated examples: even with less than 200 annotated examples it is possible to obtain results comparable with a fully supervised setting. in any case, the adopted semisupervised schema always improves the result obtained by bert. in the rest of this paper, section 2 provides an introduction to ss-gans. in sections 3 and 4, gan-bert and the experimental evaluations are presented. in section 5 conclusions are derived. more formally, let d and g denote the discriminator and generator, and p d and p g denote the real data distribution and the generated examples, respectively. in order to train a semi-supervised k-class classifier, the objective of d is extended as follows. let us define p m (ŷ = y|x, @xmath0, @xmath1, ..., k)) that x is con-sidered real, thus belonging to one of the target classes. the loss function of d is defined as: l @xmath2. + l dunsup. where: l dsup. measures the error in assigning the wrong class to a real example among the original k categories. l dunsup. measures the error in incorrectly recognizing a real (unlabeled) example as fake and not recognizing a fake example. at the same time, g is expected to generate examples that are similar to the ones sampled from the real distribution p d . as suggested in @xcite , g should generate data approximating the statistics of real data as much as possible. in other words, the average example generated in a batch by g should be similar to the real prototypical one. formally, let's f (x) denote the activation on an intermediate layer of d. the feature matching loss of g is then defined as: while ss-gans are usually used with image inputs, we will show that they can be adopted in combination with bert @xcite ) over inputs encoding linguistic information. ## gan-bert: semi-supervised bert with adversarial learning bidirectional encoder representations from transformers (bert) @xcite belongs to the family of the so-called transfer learning methods, where a model is first pre-trained on general tasks and then fine-tuned on the final target tasks. in computer vision, transfer learning has been shown beneficial in many different tasks, i.e., pre-training a neural network model on a known task, followed by a fine-tuning stage on a (different) target task (see, for example, @xcite ). bert is a very deep model that is pre-trained over large corpora of raw texts and then is fine-tuned on target annotated data. the building block of bert is the transformer @xcite , an attentionbased mechanism that learns contextual relations between words (or sub-words, i.e., word pieces, @xcite ) in a text. bert provides contextualized embeddings of the words composing a sentence as well as a sentence embedding capturing sentence-level semantics: the pre-training of bert is designed to capture such information by relying on very large corpora. after the pre-training, bert allows encoding (i) the words of a sentence, (ii) the entire sentence, and (iii) sentence pairs in dedicated embeddings. these can be used in input to further layers to solve sentence classification, sequence labeling or relational learning tasks: this is achieved by adding task-specific layers and by fine-tuning the entire architecture on annotated data. in this work, we extend bert by using ss-gans for the fine-tuning stage. we take an already pre-trained bert model and adapt the fine-tuning by adding two components: i) task-specific layers, as in the usual bert fine-tuning; ii) ss-gan layers to enable semi-supervised learning. without loss of generality, let us assume we are facing a sentence classification task over k categories. given an input sentence @xmath3, ..., t n ) bert produces in output n + 2 vector representations in r d , i.e., (h cls , h t 1 , ..., h tn , h sep ). as suggested in @xcite , we adopt the h cls representation as a sentence embedding for the target tasks. as shown in figure 1 , we add on top of bert the ss-gan architecture by introducing i) a discriminator d for classifying examples, and ii) a generator g acting adversarially. in particular, g is a multi layer perceptron (mlp) that takes in input a 100-dimensional noise vector drawn from n (µ, σ foot_1 ) and produces in output a vector h f @xmath4. the discriminator is another mlp that receives in input a vector h * ∈ r d ; h * can be either h f ake produced by the generator or h cls for unlabeled or labeled examples from the real distribution. the last layer of d is a softmax-activated layer, whose output is a k + 1 vector of logits, as discussed in section 2. during the forward step, when real instances are sampled (i.e., h * = h cls ), d should classify them in one of the k categories; when h * = h f ake , it should classify each example in the k + 1 category. as discussed in section 2, the training process tries to optimize two competing losses, i.e., l d and l g . during back-propagation, the unlabeled examples contribute only to l dunsup. , i.e., they are considered in the loss computation only if they are erroneously classified into the k + 1 category. in all other cases, their contribution to the loss is masked out. the labeled examples thus contribute to the supervised loss l dsup. . finally, the examples generated by g contribute to both l d and l g , i.e., d is penalized when not finding examples generated by g and vice-versa. when updating d, we also change the bert weights in order to fine-tune its inner representations, so considering both the labeled and the unlabeled data 2 . after training, g is discarded while retaining the rest of the original bert model for inference. this means that there is no additional cost at inference time with respect to the standard bert model. in the following, we will refer to this architecture as gan-bert. ## experimental results in this section, we assess the impact of gan-bert over sentence classification tasks characterized by different training conditions, i.e., number of examples and number of categories. we report measures of our approach to support the development of deep learning models when exposed to few labeled examples over the following tasks: topic classification over the 20 news group (20n) dataset @xcite , question classification (qc) on the uiuc dataset @xcite , sentiment analysis over the sst-5 dataset @xcite . we learning rate was set for all to 2e-5, except for 20n (5e-6). the 20n classification results are shown in figure 2a. the training and testing datasets are made of 11, 314 and 7, 531 documents classified in 20 categories foot_2 , respectively. the plot shows f1 scores of the models: when 1% of data is used (i.e., about 110 examples) bert almost diverges while gan-bert achieves more than 40% of f1. this trend is confirmed until 40% of labeled documents are used (i.e., about 5, 000 examples). in the qc task we observe similar outcomes. the training dataset is made of about 5, 400 question. in the coarse-grained setting (figure 2b ) 6 classes are involved; in the fine-grained scenario (figure 2c ) the number of classes is 50. in both cases, bert diverges when only 1% of labeled questions are used, i.e., about 50 questions. it starts to com-pensate when using about 20% of the data in the coarse setting (about 1, 000 labeled examples). in the fine-grained scenario, our approach is performing better until 50% of the labeled examples. it seems that, when a large number of categories is involved, i.e., the classification task is more complex, the semi-supervised setting is even more beneficial. the results are confirmed in sentiment analysis over the sst-5 dataset (figure 2d), i.e., sentence classification involving 5 polarity categories. also in this setting, we observe that gan-bert is beneficial when few examples are available. this is demonstrated by the difference in accuracy at 1% of the data (about 85 labeled examples), where bert accuracy is 22.2% while gan-bert reaches 30.4% in accuracy. this trend is confirmed until about 20% of labeled examples (about 1, 700), where bert achieves comparable results. finally, we report the performances on natural language inference on the mnli dataset. we observe (in figures 2e and 2f) a systematic improvement starting from 0.01% labeled examples (about 40 instances): gan-bert provides about 6 -10 additional points in f1 with respect to bert (18.09% vs. 29.19% and 18.01% vs. 31.64%, for mismatched and matched settings, respectively). this trend is confirmed until 0.5% of annotated material (about 2, 000 annotated examples): gan-bert reaches 62.67% and 60.45% while bert reaches 48.35% and 42.41%, for mismatched and matched, respectively. using more annotated data results in very similar performances with a slight advantage in using gan-bert. even if acquiring unlabeled examples for sentence pairs is not trivial, these results give a hint about the potential benefits on similar tasks (e.g., questionanswer classification). ## conclusion in this paper, we extended the limits of transformed-based architectures (i.e., bert) in poor training conditions. experiments confirm that fine-tuning such architectures with few labeled examples lead to unstable models whose performances are not acceptable. we suggest here to adopt adversarial training to enable semisupervised learning transformer-based architectures. the evaluations show that the proposed variant of bert, namely gan-bert, systematically improves the robustness of such architectures, while not introducing additional costs to the infer-ence. in fact, the generator network is only used in training, while at inference time only the discriminator is necessary. this first investigation paves the way to several extensions including adopting other architectures, such as gpt-2 @xcite or distilbert @xcite or other tasks, e.g., sequence labeling or question answering. moreover, we will investigate the potential impact of the adversarial training directly in the bert pre-training. from a linguistic perspective, it is worth investigating what the generator encodes in the produced representations.