Task-guided Disentangled Tuning for Pretrained Language Models (2022)

Abstract: Pretrained language models (PLMs) trained on large-scale unlabeled corpus are typically fine-tuned on task-specific downstream datasets, which have produced state-of-the-art results on various NLP tasks. However, the data discrepancy issue in domain and scale makes fine-tuning fail to efficiently capture task-specific patterns, especially in low data regime. To address this issue, we propose Task-guided Disentangled Tuning (TDT) for PLMs, which enhances the generalization of representations by disentangling task-relevant signals from the entangled representations. For a given task, we introduce a learnable confidence model to detect indicative guidance from context, and further propose a disentangled regularization to mitigate the over-reliance problem. Experimental results on GLUE and CLUE benchmarks show that TDT gives consistently better results than fine-tuning with different PLMs, and extensive analysis demonstrates the effectiveness and robustness of our method. Code is available at https://github.com/lemon0830/TDT .

## introduction in recent years, pretrained language models (plms) trained in a self-supervised manner like mask language modeling have achieved promising results on various natural language processing (nlp) tasks @xcite @xcite , which learn general linguistic and semantic knowledge from massive general corpus. to adapt plms to specific nlp tasks, a commonlyused approach is fine-tuning, where the whole or part of model parameters are tuned by task-specific objectives. despite its success, the fine-tuned models have been proven ineffective to capture task-specific patterns due to the gap between taskagnostic pre-training and the weak fine-tuning with limited task-specific data @xcite @xcite . the fine-tuned models tend to learn a simple rule that "apple" (red) indicates "tech" class while ignore the real meaning of "apples" (green) a reliable model is expected to find out truly task-specific patterns (underlined words) instead of some high frequency but insignificant words ("apple"). to address this problem, most existing methods focus on adapting plms to downstream tasks by continual pre-training on in-domain unsupervised data @xcite @xcite @xcite . for example, @xcite propose intermediate continual pre-training with a selective masking strategy, and @xcite adapt plms to in-domain tasks by domain-adaptive pretraining. although straightforward, these kinds of methods heavily rely on the selection of large-scale additional domain corpora and the design of appropriate intermediate training tasks @xcite . in this paper, we propose a task-guided disentangled tuning (tdt) for plms by automatically detecting task-specific informative inputs without the need of additional corpora and inter-mediate training. the core component of tdt is a confidence model which assigns each token a confidence score, and we construct distilled samples by retaining informative tokens with high confidence scores while perturbing the rest. the confidence model performs a "deletion game" strategy, which encourages the model to perturb inputs as much as possible and to maintain the performance of downstream tasks to the greatest extent with the distilled samples. although the informative tokens are important for downstream predictions, existing work shows that over-relying on part of these words may result in pool generalization, i.e., over-reliance problem @xcite @xcite . take the sentences in figure 1 as an example, when the context word "apple" frequently co-occurs with the label "tech", fine-tuned models may learn a spurious association by binding "apple" and "tech", leading to incorrect predictions of sentences which contain "apple" but belong to other categories. based on the observation, we further enhance our method with a disentangled regularization, aiming to distinguish task-relevant and task-irrelevant features. first, we construct two variants of the original input in a complementary view: (1) positive variant, which maintains the high-confidence keywords, and (2) negative variant, derived by a "cut-out-keyword" operation on the original input. next, we propose a "triplet-style loss", which makes predictions between the original input and the positive variant similar while the predictions between the negative variant and the other two different. to illustrate the mechanism of our disentangled regularization, we go back to figure 1 and take the sentence "jobs founded apple in 1976" as an example. under the influence of the disentangled regularization, the positive variant tends to retain clue words for predictions (i.e., "founded apple"), while the negative variant, as the complement (i.e., "jobs in 1976"), tends to be task-irrelevant. we evaluate our tdt on a wide range of neural language understanding benchmark datasets in english and chinese, i.e., glue and clue, and our tdt affords strong predictive performance compared with standard fine-tuning. moreover, we conduct extensive analysis with respect to robustness to perturbation, domain generalization, and low-resource settings, from which we conclude: ## method in this section, we begin with a brief introduction of the vanilla fine-tuning, and then introduce task-guided disentangled tuning (tdt) in detail. figure 2 shows the overall framework. tdt is composed of two parts: (1) token-level confidence model, which discovers the essential parts of inputs for the model prediction; ## related work fine-tuning large-scale plms tends to be a popular paradigm of various nlp tasks @xcite @xcite . however, the fine-tuned models fail to capture task-specific patterns due to the imbalanced nature between the large number of parameters and limited training data @xcite . to address this issue, two main research lines are proposed: (1) continual pretraining after general pre-training, (2) regularization techniques in fine-tuning. continual pretraining of plms on unlabeled data of a given downstream domain or task has been proved effective for the end-task performance @xcite , and various continual pre-training objectives designed for different downstream tasks have been proposed @xcite . for example, @xcite propose a selective masking strategy to learn task-specific patterns based on mid-scale in-domain data. however, such methods usually rely on extra indomain data and manually designed training objectives. due to the overfitting problems of fine-tuning, lots of regularization techniques have been proposed. @xcite and @xcite regularize fine-tuned weights with original pretrained weights while others design adversarial training objectives or introduce noise into the input @xcite @xcite @xcite @xcite . @xcite regularize the training by minimizing the kl-divergence between the output distributions of two sub-models sampled by dropout and @xcite only updates a sub-set of the whole network during fine-tuning by selectively masking out the gradients in both task-free and task-driven ways. @xcite handle the over-reliance problem by reconstructing keywords based on other words and making lowconfidence predictions without enough context. ## conclusion in this paper, we propose task-guided disentangled tuning for enhancing the efficiency and robustness of plms in downstream nlp tasks. our method is able to efficiently distinguish task-specific features and task-agnostic ones, and bridges the gap between pretraining and adaptation without the need of immediate continual training. experiments on glue and clue benchmarks demonstrate the effectiveness of our method, and extensive analysis shows the advantage in domain generalization and low-resource setting over fine-tuning. large, roberta-large for glue and bert-wwmbase, macbert-large, roberta-wwm-large for clue. the batch size, training steps, warmup steps, and learning rate are listed in