One Network, Many Masks: Towards More Parameter-Efficient Transfer Learning (2023)

Abstract: Fine-tuning pre-trained language models for multiple tasks can be expensive in terms of storage. Parameter-efficient transfer learning (PETL) methods have been proposed to address this issue, but they still require a significant number of parameters when being applied to broader ranges of tasks. To achieve even greater storage reduction, we propose ProPETL, a novel method that enables efficient sharing of a single prototype PETL network (e.g. adapter, LoRA, and prefix-tuning) across layers and tasks. We learn binary masks to select different sub-networks from the prototype network and apply them as PETL modules into different layers. We find that the binary masks can determine crucial structural information from the network, which is often ignored in previous studies. Our work can also be seen as a type of pruning method, where we find that overparameterization also exists in the seemingly small PETL modules. We evaluate ProPETL on various downstream tasks and show that it can outperform other PETL methods with around 10% parameters required by the latter.

## introduction with the release and wide application of numerous pre-trained language models (plms) @xcite , pre-training and subsequently fine-tuning them becomes prevalent in natural language processing (nlp). this yields good performance in many downstream tasks. however, such a paradigm requires the entire model to be updated and saved after fine-tuning. as plms grow in size, traditional fine-tuning becomes costly in storage, limiting the application to multi-task scenarios. in order to ameliorate this issue, many parameter-efficient transfer learning (petl) methods have been proposed @xcite @xcite . rather than fine-tuning the entire model, they introduce new parameters and only fine-tune those additional parameters on downstream tasks, which drastically reduces the storage of the parameters required by each task. however, they still require a significant number of parameters when more tasks are considered. in this paper, we continue this line of research and target using even less storage per task. we observe that recent advancements in the field of petl focus on finding better ways to apply the additional parameters, such as the adapter module after each feed-forward layer @xcite , or the low-rank matrices in the query and value projection of the self-attention networks @xcite . however, limited works examine the impact of sub-network structure and integrate pruning methods with petl methods. in fact, studies in network pruning have shown that the modeling ability of neural networks relies not only on the parameters but also on the sub-network structures that are decided by the pruning masks. for instance, @xcite discovered that the sub-network of an untrained model can yield great performance without any parameter update. in light of this, we seek to incorporate the structural information of sub-networks into petl. we believe that when enough structural information is injected into the network, we no longer need that many parameters in pelt modules and further improve the parameter efficiency. to this end, we propose a novel petl method dubbed propetl that enables efficient sharing of a single prototype adapter, prefix, or lora across layers and tasks. when sharing the prototype network, propetl learns binary masks to prune different sub-networks in different layers and tasks (figure 1 ). the connections of the prototype network pruned in one layer can be used in another with a different pruning mask. in this way, a parameter can be used multiple times across different modules, achieving higher parameter efficiency. previous methods @xcite only consider simply discarding (pruning) the useless parameters, while we focus on the structural information in masks by strategically dispatching the parameters in the single prototype network to different modules. we evaluate propetl on various downstream tasks, including glue @xcite , xsum @xcite , and wmt16 ro-en @xcite . experiments show that propetl achieves better performance than other petl methods while using significantly fewer parameters. our contributions are summarized as follows: ## related work in this section, we briefly survey ideas that are related to our work from three fields: parameterefficient transfer learning, pruning methods, and multi-task learning. ## methods in this section, we first give an introduction to parameter-efficient transfer learning (petl). we then present our method propetl, depicted in figure 2 , which combines the techniques of parametersharing and pruning to further improve the parameter efficiency compared to existing petl methods. ## experimental setup we briefly summarize the experimental setup in this section. more details can be found in appendix c. ## discussion how does propetl scale differently to adapter? figure 3 presents the results when we adjust the size of the adapter and propetl adapter on three different datasets. despite the difference in tasks and methods, we discover that adapter and propetl adapter show similar scaling trends on all three datasets when we increase the proportion of the task-specific bit-level storage. their performance increases linearly with respect to the log scale of the extra storage in the beginning. when the adapter and propetl adapter reach close to the performance of the fully fine-tuned model, their performance gradually converges. even though propetl adapter and adapter tuning can slightly exceed the fully fine-tuned performance on some datasets, their performance is still bounded to the same level and cannot outperform the fully finetuned model by a large margin. for instance, the performance of both adapter and propetl starts to drop when the task-specific storage exceed 0.4% per task on glue (figure 3 (c )). however, propetl adapter is able to reach the fully finetuned level much earlier in the scaling curves. given a fixed amount of task-specific storage, propetl adapter is also able to achieve better results than the adapter. these results indicate that our share-and-mask method is overall more efficient than the adapter across almost all scales. which is more important, sharing or masking? in this section, we discuss the effect of masking and sharing in the prototype network by comparing propetl with a random mask baseline and two alternative settings (only mask and only share). for the random mask setting, we randomly select the sub-network of the prototype module during the forward process rather than relying on the updated mask score. only mask involves not sharing the module across layers but only learning masks to prune different petl modules into sparse subnetworks. only share shares a single petl module among layers without using masks for pruning. masking without sharing will drastically increase the number of fine-tunable parameters if we keep the same bottleneck dimension and sparsity ratio. to keep the number of fine-tunable parameters on the same level, we either keep the bottleneck dimension the same and reduce the sparsity ratio k% or use the same sparsity ratio with reduced bottleneck dimension for the only mask setting. we also slightly increase the bottleneck dimension of the only share set up to compensate for the parameters of masks. our results, as presented in ## conclusion and future work in this paper, we introduce propetl, a method for sharing prototype petl modules across different layers and tasks. our method significantly improves the parameter efficiency by utilizing the prototype network and maintaining a binary mask for each layer and task. extensive experiments show that our method achieves comparable performance to fully fine-tuned models and conventional petl methods with a much smaller fraction of storage. for future works, we aim to study the interpretability of the masks in different layers and explore their potential relationships. we also intend to apply our method to the pre-training process of large language models to reduce the overall number of parameters.