B i KT : Enabling Bidirectional Knowledge Transfer Between Pretrained Models and Sequential Downstream Tasks (2024)

Abstract: Adapting pretrained models to downstream tasks is important in practical applications. Existing frameworks adapt from an initial pretrained model to each downstream task directly, but ignore the sequential nature of the downstream tasks and their feedback effect on the pretrained model. In this work, we propose a new framework, called BiKT, to enable bidirectional knowledge transfer between pretrained models and downstream tasks in rounds. We model each downstream task in the current round as a target task for adaptation and treat all the tasks in the previous rounds as source tasks for feedback. We design a feedback algorithm by multi-task learning over the labeled data of the source tasks, where task-specific prompts are plugged into the backbone network for decoupling task-exclusive knowledge from task-shared knowledge. We further utilize the good initiation of the new backbone network updated in the feedback phase and the trained prompts of the source tasks for adaptation. Evaluation over 9 GLUE datasets, 6 SuperGLUE datasets, and 8 other datasets using models with different pretraining levels and different parameter scales shows remarkable improvement in full-shot and few-shot adaptation settings.

## introduction language models are initially pretrained on a diverse corpus and then adapted to various downstream tasks in practical applications. finetuning @xcite and prompt tuning @xcite are two typical and widely used adaptation methods. in particular, finetuning updates all the parameters of a pretrained language model, shifting the whole backbone network towards a specific downstream task. in contrast, prompt tuning freezes the backbone network and tunes a small number of parameters inserted to the pretrained model for each downstream task. as shown in figure 1a , the current pretrain-thentune paradigm directly adapts a pretrained model to each downstream task, which is a one-time and unidirectional knowledge transfer. multiple downstream tasks are treated independently as different ending points with the initial pretrained model as the same starting point of adaptation. nevertheless, in practice, the deployment requirements of the pretrained model in different application scenarios are raised over time rather than all at once. therefore, as depicted in figure 1b , the downstream tasks should come in a sequential way, and a certain task for adaptation has some available previous tasks that have been adapted to ahead of this task. in this work, we consider how to enable the bidirectional knowledge transfer between a pretrained model and downstream tasks. such a problem is well-motivated. on the one hand, different from pretraining over public corpus, the downstream tasks in practical applications directly serve a large scale of users and receive their feedback, thereby generating massive fresh labeled samples. these high-quality samples can be exploited to continuously improve the generalization ability of the pretrained model in the real open world. on the other hand, the pretrained model, which has been optimized over previous downstream tasks, can become a better starting point of adaptation to the current task, facilitating knowledge transfer from upstream to downstream in the sequence of tasks. to deal with the problem above, we propose a new framework, called bikt, including the feedback phase from sequential downstream tasks to pretraining and the adaptation phase from pretraining to downstream tasks, thereby boosting the generalization ability of the pretrained model and improving the adaptation performance. we first model the sequential nature of downstream tasks in different rounds. we call a certain task in the current round to be adapted to as the target task and treat the tasks ahead of the target task in the previous rounds as the source tasks for feedback. we then design a multi-task feedback algorithm over the source tasks for the pretrained model with trainable prompts in an incremental way. the newly inserted prompts are task-specific, whereas the backbone network of the pretrained model is task-shared, eliminating the interference of task-specific knowledge on the generalization of the pretrained model. after feedback, we further leverage the latest backbone network and the trained prompts of the source tasks from the feedback phase as good model initialization for adapting to target tasks. we summarize the key contributions as follows: ## related work in this section, we briefly review related work. ## problem formulation we formulate the problem of bidirectional knowledge transfer between a pretrained model and sequential downstream tasks, including the feedback link from the downstream tasks to the pretrained model and the adaptation link from the pretrained model to the downstream tasks. ## evaluation we evaluate bikt over a wide range of downstream tasks using various pretrained models. ## conclusion in this work, we have studied the relationship between pretrained models and downstream tasks. we first have modeled the sequential nature of downstream tasks and have proposed the new framework of bidirectional knowledge transfer, called bikt, including the two phases of adaptation and feedback. we have designed a multi-task feedback learning algorithm with trainable task-specific prompts as well as a model adaptation algorithm with feedback-based initialization. evaluation results over several datasets using different models with varying sizes have demonstrated the effectiveness of bikt in both full-shot and few-shot settings as well as the remarkable advantage over conventional finetuning and prompt tuning methods.