Assessing the Portability of Parameter Matrices Trained by Parameter-Efficient Finetuning Methods (2024)

Abstract: As the cost of training ever larger language models has grown, so has the interest in reusing previously learnt knowledge. Transfer learning methods have shown how reusing non-task-specific knowledge can help in subsequent task-specific learning.In this paper, we investigate the inverse: porting whole functional modules that encode task-specific knowledge from one model to another. We designed a study comprising 1,440 training/testing runs to test the portability of modules trained by parameter-efficient finetuning (PEFT) techniques, using sentiment analysis as an example task. We test portability in a wide range of scenarios, involving different PEFT techniques and different pretrained host models, among other dimensions. We compare the performance of ported modules with that of equivalent modules trained (i) from scratch, and (ii) from parameters sampled from the same distribution as the ported module.We find that the ported modules far outperform the two alternatives tested, but that there are interesting differences between the four PEFT techniques tested.We conclude that task-specific knowledge in the form of structurally modular sets of parameters as produced by PEFT techniques is highly portable, but that degree of success depends on type of PEFT and on differences between originating and receiving pretrained models.

## introduction and related work given the increasing costs of training and running neural models @xcite , the interest in finding methods to reduce these costs is growing. reusability of previously learned knowledge is one very promising avenue to pursue, in particular if this were possible in plug-and-playable form. methods that come under the broad heading of transfer learning have shown for some time that general, non-task-specific knowledge transferred from one learning scenario to another can help speed up task-specific learning in the latter. well established techniques such as word and wordsequence embeddings, and pretraining plus finetuning are examples, as is adaptation from one domain to another @xcite , one language to another @xcite , or one task to another @xcite . what these approaches have in common is that they aim to extract general, or at least non-task-specific, knowledge while discarding the task-specific knowledge. reusability could be radically extended if it were possible to reuse both generic and different types of task-specific knowledge, especially if these could be recombined with some degree of freedom. for this to be possible, the knowledge would have to be contained in structurally and functionally modular, or portable, (sub)networks. some research has explored model compression @xcite which can be seen as attempting to extract modules with desired functionality. other work has looked at identifying subnetworks with given functionality @xcite , but none has to our knowledge successfully demonstrated portability of task-specific modules. parameter efficient finetuning (peft) techniques such as adapters @xcite , prefix tuning (li and liang, 2021), compacters @xcite , and lora @xcite , train sets of parameters that have been shown to be structurally modular @xcite , in the sense that they form separate parameter sets that interact with their host model via dedicated interfaces. however, it is currently unclear if peft modules are also functionally modular. one important marker of functional modularity is encapsulation, i.e. the degree to which a (structural) module performs dedicated functions that are separate from functionality elsewhere in the system. encapsulation is a precondition for portability which would be an important step in the direction of plug-and-playable neural components, potentially capable of achieving substantial reductions in training time and resources, and increased reusability in neural system development @xcite @xcite . modularity (without porting) has been explored in the context of adapters for multi-task crosslingual transfer @xcite . cross-task transferability (in unchanged peft-tuned models) has also started to be explored very recently, e.g. in conjunction with prompt tuning @xcite . @xcite extended this to other peft techniques, showing that peft-tuned models maintain performance on closely related tasks, but not on less closely related tasks. in this focused contribution, we assess something more challenging: whether peft techniques, specifically, create modules that encode task-specific knowledge that is portable to new models. we start with an overview of our study (section 2) and the experimental set-up (section 3). we then present the results (section 4), and conclude with discussion and findings (section 5). ## study overview our goal in the present study is to investigate the degree to which the knowledge encoded in the parameter matrices that result from peft tuning (which we call peft modules) is portable. more specifically, the degree to which such knowledge is portable between different models under different conditions. the study is designed to test the portability of modules trained by different peft techniques from an originating model (in conjunction with which the module was trained), to a different receiving model; moreover to test it under different conditions, including different types and combinations of originating and receiving models, different numbers of learning steps during module training at the originating model end, and (b) module training at the receiving model end, as described in more detail in the next section. peft archit. repeats insertion workspace (mlp) prefix tun. non-lin. all layers parallel attn keys/values lora linear all layers parallel attn query/val. adapter non-lin. all layers sequential ffn, attn block compacter non-lin. all layers sequential ffn, attn block ## experimental set-up put simply, if the knowledge encapsulated in peft modules is portable to new models, then plugging a pretrained peft module into a new model will result in superior performance for the same number of post-porting learning steps than a randomly initialised peft module. more strictly, if it really is the knowledge encapsulated by the pretrained peft module that leads to the superior performance rather than simply starting training off in a statistically advantageous point in the search space, then initialisation with parameters sampled from the same distribution (with the same mean and variance) will result in worse performance. to establish whether these are the case is the purpose of the present study. in it we performed experiments as per the following experimental grid: (i) four combinations of originating and receiving models, (ii) sentiment classification as the example nlp task, (iii) four peft techniques, (iv) same vs. different datasets on originating and receiving sides, (v) two importing scenarios (exact parameters vs. sampled from same distribution), (vi) two different numbers of learning steps in the pre-porting training of modules, (vii) three different numbers of learning steps in post-porting training (module adaptation to the receiving model environment), and (viii) three different random seeds. this grid corresponds to 1,152 experiments; we added 288 experiments for training from scratch without importing the pretrained peft module (where there is no pre-porting training, and no importing scenarios), making it a total of 1,440 experiments. pretrained models used (i above): we selected four different versions of the open-source t5 model @xcite , as shown in and their instruction-tuned flan equivalents. 3 this selection gives us good coverage in terms of model size and types of knowledge in pretrained models (raw language model vs. instruction tuned). datasets (ii): our example nlp task is sentiment analysis, and we used two english datasets, namely sst-2 foot_3 and rotten tomatoes, foot_4 with accuracy as the performance measure. the task was construed as sequence prediction, i.e. the input is provided as the prompt directly without task descriptions or prefixes, and the sequence continuation generated by the model should be the desired output (here, the sentiment label: 'great' or 'terrible'). we opted for this setup to ensure a level playing field for raw and instruction-tuned models. it avoids granting the latter an unfair advantage that could result from explicit task descriptions. the (raw) t5 v1 models ( combinations of pre-porting and postporting datasets (iv): the pre-porting dataset is the one used to peft-tune the module (i.e. before it is exported). the post-porting dataset is the one used in further tuning an imported peft module within its new environment. we compare (a) using the same dataset (rotten tomatoes) in post-porting tuning and testing as was used in pre-porting tuning, and (b) using different datasets (rotten tomatoes on the pre-porting side, and sst-2 on the postporting side). importing scenarios (v, additionally fromscratch tuning): in this experimental dimension we tested three alternatives, namely (i) importing peft module parameters exactly as they are at the end of (pre-porting) peft-tuning, (ii) sampling new parameters from the same (normal) distribution, i.e. with the same mean and variance, and (iii) initialising parameters randomly using their peft default initialisation techniques foot_7 . pre-porting and post-porting learning steps (vi, vii): we tested two different numbers of learning steps for pre-porting peft tuning: 5k and 10k. on the post-porting side, we tested three different numbers of learning steps: 0.5k, 1k and 3k. for details of the hyperparameters we used with the different methods, see appendix a.2. ## results the first two sets of results we present (in tables 3a and 3b ) are the mean and variance of accuracy scores over all of the following experimental dimensions: i (four pairs of originating and receiving models), vi (two different numbers of pre-porting learning steps), vii (three different numbers of postporting learning steps), and viii (three different random seeds). this provides a high-level perspective on the extent to which knowledge has been successfully ported on average for each of the four types of peft module, as compared to the corresponding sampled and from-scratch parameters. we can very clearly see the substantial advantage that importing a pretrained peft module brings for all four peft techniques. performance increases are similar across peft techniques and same/different datasets, but compacter benefits the most, followed by adapter, lora and prefix-tuning. as indicated in figure 1 shows more finegrained results, for same datasets at the top (a and b), and different datasets at the bottom (c and d). each half of the figure is further divided into porting from raw to instruction-tuned host models (left) and vice versa (right). more information in figure caption. accuracy is remarkably similar for same vs. different pre-porting and post-porting datasets across the different scenarios. this implies that the knowledge acquired is dataset-agnostic. it is also very stable across 5k vs 10k peft-tuning steps on the originating side. the porting direction makes a big difference. when porting from a raw host model to an instruction-tuned one (left side of figure 1 ), we see the following pattern. remarkably, all peft techniques exhibit some degree of zero-shot portability, with ported modules achieving up to around 0.7 accuracy straight out of the box, compared to 0 for both sampling and random parameters. from 500 post-porting learning steps onwards, performance evens out between ported, sampled and random parameters, and also plateaus out, for adapter, lora and prefix-tuning. for compacter, this happens at 1,000 steps. when porting from an instruction-tuned host model to a raw one (right side of figure 1 ), we see different patterns. only adapters exhibit any zero-shot portability in this porting direction, albeit at much reduced accuracy levels. however, here the performance with imported modules remains much higher than with sampled and random parameters across all learning steps; this is the case for all peft techniques except compacters. in terms of overall best performance, only adapters match the corresponding best performance in the other porting direction (raw to instruction-tuned) by 3,000 learning steps. lora and compacter perform much less well overall than adapter and prefix-tuning in this porting direction. the differences between the two porting directions may be in part due to differences in knowledge encoded in raw and instruction-tuned models. a peft module trained with a raw model as host has to acquire all task-specific knowledge (because a straightforward language model has none), making the knowledge encapsulated in the peft module more task-specific and more self-contained, explaining the good zero-shot post-porting performance observed. at the same time, the receiving host model, because instruction-tuned, already has relevant task-specific knowledge, explaining why ported, sampled and random variants perform on a par from 500 (1,000 for compacter) post-porting learning steps onward. conversely, a peft module trained with an instruction-tuned model as host only has to acquire task-specific knowledge not already present in the host, making the knowledge encapsulated in the resulting peft module less task-specific and less self-contained, explaining the mostly absent zeroshot post-porting performance observed. at the same time, the partial task-specific knowledge encoded in the imported parameter still bestows a substantial boost in a situation where the receiving host model is a raw model with no task-specific knowledge, explaining why the ported modules outperform alternatives in all scenarios except for compacters with different datasets. the results reported here are for a comparatively easy task. in appendix b, we report preliminary results for similar experiments involving natural language inference, a much more complex task, with the aim of confirming generalisation to more complex tasks. ## conclusion our study shows, for the first time, that peft modules are structurally and functionally sufficiently modular to be portable from one host model to another. remarkably, we observed pronounced zero-shot portability (with no post-porting adaptation tuning at all) for the best peft techniques. the performance that can be achieved in the model being ported to depends on the porting direction and peft technique used. adapters appear to deliver the highest degree of portability overall across both directions. given the structural differences between the types of peft modules tested, our results point in an exciting direction: it may be possible to extrapolate from such results to design new peft techniques specifically optimised for portability. the structural properties of current peft techniques impose limits on the reusability of ported modules, e.g. requiring the receiving model to have the same hidden dimension and number of layers as the originating model. addressing these limitations could pave the way for more versatile and widely portable peft modules. we are currently epxloring these aspects further in extended portability tests, initially for a wider range of different tasks, and subsequently for other models and task construals. a particular focus in future work will be the efficiency savings that can be achieved through portable modules, including computational budgets required for different peft techniques to achieve satisfactory performance in ported modules.