G -Tuning: Improving Generalization of Pre-trained Language Models with Generative Adversarial Network (2023)

Abstract: The generalization ability of pre-trained language models (Plms) in downstream tasks is heavily influenced by fine-tuning. The objective of fine-tuning is to transform the latent representation of Plms from a universal space to a target space, allowing the model to be applied to downstream tasks with the capability of generalizing to unseen samples. However, the effect of Plms will be diminished when the training data coverage is insufficient, in which fine-tuning is inadequate to learn the complete mapping. In this study, we propose a new fine-tuning framework, referred to as G-Tuning, that aims to preserve the generalization ability of Plms in downstream tasks. Specifically, we integrate a generative adversarial network into the fine-tuning process to aid in the transformation of the latent representation in the entire space. Empirical evaluations on the GLUE benchmark, as well as two additional demanding scenarios involving domain and language generalization, demonstrate that G-Tuning can accurately map the universal representation to the target space, thus effectively enhancing the generalization performance of Plms across various downstream tasks.

## introduction large-scale pre-trained language models (plms) have demonstrated substantial achievements in natural language processing (nlp) recently @xcite . generally, fine-tuning plms with the task-specific training data can get significant improvements compared to training a model from scratch @xcite @xcite . fine-tuning aims at transforming the representation from plms in the universal space to the target space, thereby enabling the model to generalize to a wider range of samples @xcite @xcite . however, the generalization capability of plms is largely affected by the task-specific data when using fine-tuning to further train the model @xcite . as noted by @xcite , fine-tuning is susceptible to memorizing the training data when the capacity of the plm exceeds that of the downstream task data. furthermore, the advantage of plms over random initialization is lost when the coverage of task-specific data is low, resulting in a large gap between training and test data @xcite . for instance, in domain generalization, plms fine-tuned with in-domain training sets often fail to perform well on out-of-domain test sets, even when data from the test sets is used for pre-training @xcite . therefore, a crucial challenge for unlocking the potential of plms is how to learn a complete and accurate mapping from the universal space to the target space with limited training data. previous studies have presented some promising approaches to address this problem. @xcite proposed a self-teaching method to use a fine-tuned plm to get soft labels of the unlabeled data and train another plm by these synthetic data, which brings considerable improvements in the cross-lingual transfer scenario. @xcite presented regularized self-labeling to correct mislabeled data points and reweight less confident data points to regularize plms. @xcite proposed an ensemble learning method for domain generalization, which can dynamically dispatch proper plms to predict each test sample. @xcite proposed a noisy tuning method to add matrix-wise perturbation to different parameter matrices to overcome the outfitting problem of finetuning, which indirectly improve the generalization ability of plms. @xcite presented stochastic weight averaging to improve generalization by encouraging convergence of the model to a flatter minimum. furthermore, compared to fine-tuning, in-context learning which doesn't tune the parameter of plms, we leave this to future work @xcite @xcite . in this paper, we propose a novel fine-tuning framework (named g-tuning) to preserve the generalization capability of plms when training with task-specific data. we adopt parameter efficient fine-tuning (peft) as our backbone, which only tunes a lightweight adapter network connected behind plms @xcite 1 , and incorporate a generative adversarial network @xcite @xcite into it to learn the representation mapping. specifically, we first train a discriminator to aim of discriminating the representation that is not mapped correctly from the target space. then, besides predicting the ground-truth label, the model is seen as a generator requested to generate the representation hard to be discriminated. we conduct experiments on the glue and two more challenging scenarios, i.e., domain and language generalizations. experimental results show that g-tuning can improve generalization capability by effectively mapping the universal representation to the target space. parameter efficient fine-tuning. given a training set b from a downstream task, we can summarize the loss function of peft @xcite as: ## conclusion in this work, we elucidate a drawback of the finetuning strategy on plms, which is that the representation from plms is not fully mapped to the target space when the training data is insufficient. the generalization ability of plms in the downstream tasks will be diminished in this situation. to address this issue, we present g-tuning, a framework that aims to preserve the generalization ability of plms in the downstream tasks. the proposed g-tuning utilizes a generative adversarial network to transform the representation that is not covered by training data into the target space. extensive experiments on the glue and two additional scenarios show that g-tuning accurately maps the universal representation to the target space, getting substantial improvements in generalization performance.