Jointly Reparametrized Multi-Layer Adaptation for Efficient and Private Tuning (2023)

Abstract: Efficient finetuning of pretrained language transformers is becoming increasingly prevalent for solving natural language processing tasks. While effective, it can still require a large number of tunable parameters. This can be a drawback for low-resource applications and training with differential-privacy constraints, where excessive noise may be introduced during finetuning. To this end, we propose a novel language transformer finetuning strategy that introduces task-specific parameters in multiple transformer layers. These parameters are derived from fixed random projections of a single trainable vector, enabling finetuning with significantly fewer parameters while maintaining performance. We achieve within 5% of full finetuning performance on GLUE tasks with as few as 4,100 parameters per task, outperforming other parameter-efficient finetuning approaches that use a similar number of per-task parameters. Besides, the random projections can be precomputed at inference, avoiding additional computational latency. All these make our method particularly appealing for low-resource applications. Finally, our method achieves the best or comparable utility compared to several recent finetuning methods when training with the same privacy constraints, underscoring its effectiveness and potential real-world impact.

## introduction transformer-based bidirectional language models (lms), pretrained on a sizeable text corpus and finetuned on task-specific objectives, outperform models trained from scratch by large margins @xcite . the straightforward approach to finetune a language model is to initialize with pretrained parameters and train the model on the downstream task. however, it is inefficient to finetune language models for each task as it requires training and storing a massive number of parameters per task (roughly the same as the size of language models) @xcite ; devlin et al., 2019). these inefficiencies are exacerbated in resource-constrained settings, such as personal devices with limited or federated learning scenarios where the costs of communicating parameter updates may limit the scope of applications @xcite . the shortcomings of naive finetuning methods have motivated research into approaches that identify and train fewer task-specific parameters @xcite . those parameter-efficient finetuning methods work by introducing taskspecific trainable layers while freezing most of the pretrained language model parameters (e.g., adapter @xcite , @xcite ) or by introducing taskspecific trainable prompts or inputs (e.g., prompttuning based warp @xcite , prefix-tuning @xcite ). we summarize the key properties of prominent efficient finetuning methods in warp increases the effective sequence length, and adapter inserts task-specific layers, incurring additional computational overhead. in contrast, our method is efficient in memory usage and run-time during training. further, task-specific parameters learned by our approach can be fused with lm during inference, leading to no additional latency during inference, making it especially appealing for resource-constrained applications. besides computational efficiency, our approach's parameter efficiency makes it an excellent private learner. our approach's utility is competitive or outperforms the best differential private finetuning results @xcite when training for similar levels of privacy. ## method model. figure 2 summarizes our model, highlighting task-specific parameters with colored fonts. specifically, we consider a trainable vector @xmath0. we do so by projecting z with random but fixed matrices w l to obtain shift vectors z l for the l th transformer block (z @xmath1, w @xmath2, and @xmath3. . . l}). z l is added to the output activations of the respective transformer block, as shown in figure 2 . z l is of the same dimensionality as the activations of the output feedforward layer in the l th transformer block (d ′ l ), and z is shared between all the blocks. hence, we call our approach shared layer shift or slash. the random projection matrices, w l , are not trainable and are fixed throughout the training. we initialize w l and z with zero-centered gaussian or uniform distribution for our experiments (see appendix b.2 for ablations on initialization choices). slash is akin to training only bias parameters of the output feed-forward layers. however, the projection step decouples the dimensions of z and activations, providing the flexibility to change the number of trainable parameters and control the complexity of the model by varying d irrespective of the activation dimensions. our choice of adding z l to only output activations is inspired by @xcite , who use a similar setup to learn sentence representations. we also consider adding the shifts to other activations, such as intermediate activations or activations after the self-attention layer in appendix b.1. in particular, adding shifts to output activations performs similarly or better than other choices. adding shifts to intermediate layers performs similarly to adding shifts to the output layer. however, the dimensionality of intermediate activations is usually more than that of output activations which would increase the size of projection matrices, making it an inferior choice. classification head. we experiment with token classification and sequence classification tasks with bert-like models. to this end, we remove the decoder layer of the pretrained lm and attach a taskspecific linear layer (classifier) to predict the output from text representations. verbalizers @xcite can also be used. ## experiments we evaluate our approach for sequence classification tasks in section 3.1 with the general language understanding evaluation (glue) benchmark @xcite and token classification tasks with named entity recognition (ner) on conll 2003 dataset @xcite in section 3.2. we report memory and training time requirements to quantify the computational efficiency in section 3.3. finally, we demonstrate the utility of our approach for differential private finetuning of lms in section 3.4. 1 baselines. we compare against full-finetuning and several prominent parameter-efficient finetuning techniques. specifically, we compare with adapter @xcite adapter introduces task-specific feed-forward layers in each transformer block. adapter typically trains down-project and up-project feed-forward layers in pairs for each transformer block. the dimensions of the down-projection (denoted as m) govern the per-task trainable parameters. low-rank adaptation, or lora learns the change in the pretrained weights, i.e., ∆w , for 2022 ). 3 the standard deviations are computed over 5 training runs with different seeds. due to computational limitations, we report error bars for our methods only. the downstream tasks. ∆w is parameterized as the product of low-rank matrices, which requires much fewer parameters than full-finetuning. the rank of the matrices determines per-task parameters. warp n introduces n learnable input tokens by adding trainable embeddings to the input. it is the continuous version of prompt-tuning and a special case of prefixtuning @xcite , with prefixes introduced only in the embedding layer. the learned tokens do not necessarily correspond to an existing token from the vocabulary. finally, we compare with bitfit, which finetunes only all the bias parameters. indeed, bitfit finetunes a superset of parameters considered by our approach. further, slash shares trainable parameters across all the blocks, which is more efficient. ## warp0 feeds [mask] representations to the classifier head, whereas the linear classifier uses [cls]representations. results. tables 2 and 3 summarize the results of finetuning with different methods using pretrained roberta models. parameter-efficient finetuning approaches aim to achieve performance at par with full finetuning while using fewer parameters. to this end, figure 1 provides a visual summary of the parameter vs. performance trade-offs. slash's average performance is already within 4 points of full finetuning for both roberta-base and -large models with @xmath4,024. this gap is further reduced by increasing the dimension of the z vector. even though the best models from our approach do not match the full-finetuning performance overall, for smaller datasets such as sts-b, mrpc, and rte, slash is competitive with fullfinetuning. in the case of roberta-large, we have 92.4 vs. 91.5 for sts-b, 90.9 vs. 91.3 for mrpc, and 86.6 vs. 84.1 for rte with finetuning and slash, respectively. 4 the parameter sharing reduces the per-task parameters considerably (4 orders of magnitude less) and is faster and more efficient to train (section 3.3). all these make our approach suitable for low-resource, low-data applications such as training on edge devices or learning personalized models. most efficient tuning techniques tune a few hundred thousand parameters, except for warp. it adds trainable parameters around input embeddings, which facilitates training with a few thou-sand parameters and is most comparable to our approach in per-task parameters. our approach with @xmath5,048 (i.e., 5.1k parameters) outperforms warp with 25k parameters on all datasets with less than 10k training samples. further, slash outperforms the best results of warp while using less than 60% of parameters (13k vs. 25k). these observations do not change even with warp pretraining on the mnli task to improve the performance on smaller datasets (warp mnli ). we do not require this supervised pretraining trick. these results validate the intuition that instead of introducing task parameters closer to the input layer as in warp, it may be more effective to introduce the parameters throughout the layers as in slash. armed with this intuition, we improve warp's performance by introducing prompts in all transformer blocks derived from a single vector (jr-warp). on average, it underperforms slash, and the variance among different training runs is higher. nevertheless, jr-warp performs comparably to warp 20 (84.4 vs. 84.8) while using fewer parameters (13k vs. 25k), suggesting that reusing parameters across layers improves parameter efficiency but does not deteriorate performance. next, we compare with lora and adapter, arguably the most prominent language transformer finetuning approaches. we note that the adapter (@xmath6. slash performs comparably to these methods for smaller datasets, using 5x fewer parameters and being roughly 2x faster to train for roberta-base and 7x fewer parameters and roughly 1.25x faster to train for roberta-large (tables 2, 3 and 5). for example, in the case of roberta-base, we have 91.0 vs. 89.9 for sts-b, 92.3 vs. 90.7 for mrpc, and 78.3 vs. 76.7 for rte with adapter and slash, respectively. finally, slash performs comparably to bitfit while tuning much fewer parameters. as with the other baselines, it is only for the larger datasets that bitfit considerably outperforms slash. further, we observe that tuning only output activation's biases, which used fewer than 15% of bitfit's parameters, performs comparably to bitfit on average (last row of tables 2 and 3 ). ## related work prompt tuning and task-specific finetuning are standard ways to prime lms for downstream tasks @xcite . prompt tuning inserts task-specific information or parameters around the input. various versions exist, such as manual prompt-tuning, discrete prompt search @xcite , and continuous search @xcite . prompt tuning is highly parameter efficient but is generally only effective for larger lms @xcite . due to joint reparametrization, our method uses a similar number of parameters as prompt-tuning methods but outperforms them. several parameter-efficient lm finetuning methods have been proposed, such as adapter @xcite , lora @xcite , prefix-tuning @xcite , and parallel adapters @xcite . further improvements try to maintain the utility while reducing the parameters such as compacter @xcite that parameterizes weight matrices via the sum of kronecker products, pruning adapter layers @xcite and gating mechanisms to choose the best modules @xcite . these methods outperform prompt tuning but use more parameters. in contrast, we outperform prompt tuning while using similar number of parameters and are competitive with other finetuning approaches. our approach could be of independent interest for understanding intriguing properties of pretrained language models, the role of different parameters, and sharing parameters across layers. ben @xcite have shown that pretrained models can be finetuned by only updating the bias parameters, but unlike us, they do not share parameters. @xcite finetune only cross attention layers for machine translation. @xcite share only output layers across tasks, but parameters across different layers are not shared. @xcite have shown that task embeddings can be derived from task-specific finetuned parameters. the z in our approach can also be helpful as a task-embedding. parameters derived by fixed random transformations a few parameters have previously been used to study the task's intrinsic dimensionality @xcite . those works focus on weight matrices. while insightful, these are cumbersome to train for real-world deployment. instead, we focus on bias or embeddings, providing a tractable operationalization for regular training and finetuning while using similar order of parameter count. for example, @xcite show that the intrinsic dimension of the qqp dataset with roberta-large is 774, i.e., at least 774 parameters are required to achieve within 90% of full finetuning performance. slash achieves an f1-score of 83.2, more than 90% of full finetuning performance on qqp with 4.1k parameters (92.2 × 0.9 = 83.0). ## conclusion we introduce a multilayer lm finetuning technique where task-specific parameters are derived from a single vector. we show two instantiations of this technique -slash and jr-warp. slash introduced shifts in the output activation of each transformer block, whereas jr-warp inserted prompts in each transformer block. these methods require only a tiny fraction of the original language model parameters (similar to prompt-tuning) and outperform previous methods that use a similar number of per-task parameters. despite the drastic reduction in the number of parameters, we demonstrate that these perform just as well as full finetuning for sentence and token classification tasks (only at max a 5% difference in performance). the high parameter efficiency leads to better training speed and resource utilization and improves private training.