IT -Tuning : Parameter Efficient Information Token Tuning for Language Model (2024)

Abstract: Recently, language models have demonstrated exceptional performance compared to their predecessors. In this context, attention mechanisms and pre-training significantly contribute to the enhanced performance of modern language models. Additionally, a continuously increasing number of parameters plays a crucial role in these advancements . However, an increase in the number of parameters significantly increases the GPU memory and training time required during fine-tuning of language models, this makes fine-tuning infeasible in environments with limited computing resources. Furthermore, after fine-tuning, the storage space required for deployment increases proportionally with the number of tasks, making it challenging to deploy devices with limited storage capacities. In this study, we propose IT-Tuning, a Parameter Efficient Fine-Tuning method that introduces a new concept called information tokens to address these issues.

## introduction since the introduction of transformer @xcite and bert @xcite , recent transformer based pre-trained language models have achieved unprecedented performance, coinciding with an increase in the number of parameters. @xcite @xcite consequently, research on various applications, such as sentiment analysis, question answering, sentence classification, summarization, and machine translation, has been actively conducted. although pretrained language models can be utilized in various tasks using only prompts without fine-tuning, as demonstrated by approaches such as chain-ofthought prompting @xcite or in-context learning @xcite , fine-tuning often leads to better performance. however, recently introduced language models have billions to tens of billions of parameters @xcite , resulting in increased gpu memory and extended training time requirements during fine-tuning. in addition, deploying these models after fine-tuning across various tasks requires significant storage space proportional to the size of the model and the number of tasks, which poses a challenge. in this context, we introduce a new concept called "information token" to address this issue. the information token attends to all tokens within the input sentence during the attention mechanism process, selectively condensing the information of the sentence according to the task and delivering it to the input and output sentences, enabling efficient fine-tuning. based on these information tokens, we propose a new parameter efficient fine-tuning (peft) method called it-tuning, the contributions of which are as follows: ## related works recently, extensive research has been conducted on peft, achieving a performance equivalent to that of full fine-tuning using only 0.1% of the total model parameters. @xcite @xcite this approach significantly reduces the size of the gpu memory and training time required. furthermore, because peft requires only a few additional parameters to be stored when a single model is used for various tasks, it saves storage space. in this section, we describe the research on peft conducted to date. adapter it is a method of inserting multiple adapter modules (mlp modules) into the layers of a language model. research has continued extensively after the adapter @xcite , and recent studies such as adapterbias @xcite and adamix @xcite have significantly improved performance by modifying the structure of the adapter module or proposing the mixtureof-adaptation method. furthermore, adapter-based peft methods can adjust the number of parameters used in training by altering the number of parameters in the adapter module, making them applicable to both nlu and nlg tasks. trainable prompt this method involves adding trainable tokens to sentences, with prominent examples of p-tuning v1 and v2 @xcite , prefix tuning(li and liang, 2021), and prompt tuning @xcite . these studies aimed to address the performance gap observed when using sentence-form prompts (discrete prompts) compared with fine-tuned language models by incorporating trainable tokens through fine-tuning rather than nontrainable sentence-form prompts. these studies have demonstrated results achieving equivalent performance to full fine-tuning in nlg(li and liang, 2021) tasks and nlu @xcite @xcite tasks. low-rank peft methodologies based on lowrank, such as lora @xcite and hiwi @xcite , are gaining attention owing to their robust performance and capability to mitigate the issue of increased inference times associated with additional parameters. however, one of the significant advantages of peft is its ability to switch tasks in a multitasking environment, which is compromised by combining model parameters and lowrank parameters to mitigate the increase in inference time. additionally, there are studies such as (ia) 3 @xcite ) that achieve better performance with fewer parameters than lora in some tasks. nonetheless, lora has proven to be a robust method that is effective in both nlu and nlg tasks. therefore, we conducted experiments using the lora as the baseline. ## experiment in this section, experiments are conducted on both nlu and nlg tasks to demonstrate the effectiveness of it-tuning. in nlu tasks, we selected the [cls] token as the information token, while in nlg tasks, we inserted real words such as "sum-marize", "transformation", "text", "table", etc., between input and output sentences and selected them as information tokens for experimentation. we conducted experiments using these datasets, excluding the wnli. we used the roberta-large model @xcite for experimentation, with both rank and α set to 1. additionally, although the roberta paper suggests using a model pretrained on the mnli dataset as the initial model when experimenting with small datasets such as rte, mrpc, and sts-b, we chose not to employ this approach. this is because finetuning a pretrained language model on the mnli dataset is not parameter efficient. ## conclusion in this study, we propose information tokens to efficiently learn various tasks such as prediction, classification, and generation by selectively condensing the information of input sentences according to the task and conveying the condensed information to both input and output sentences. we enabled information tokens to selectively attend to the tokens of input sentences through direct updates of the query vectors of the information tokens in each layer of the model. in addition, we enabled bidirectional operations for information tokens in the attention process in unidirectional language models, such as gpt @xcite , allowing information tokens to fulfill their roles even in unidirectional language models. furthermore, we enhanced the performance by proposing an efficient structure that combines scaling and shifting within the layers to update the hidden state of the information tokens according to the task requirements. ultimately, when conducting experiments using it-tuning, we surpassed both full fine-tuning and lora @xcite on the glue benchmark @xcite using only 0.14m parameters, which is five times fewer. furthermore, unlike existing methods, such as bitfit @xcite , pasta @xcite , and (ia) 3 @xcite , where the increase or decrease in the number of parameters used for training is fixed and cannot be adjusted, making them applicable only to specific tasks (e.g., nlu tasks), it-tuning enables the adjustment of the number of parameters used for training through information tokens. owing to the scalability of it-tuning, when applied to nlg tasks, we achieved a performance surpassing both full fine-tuning and prefix tuning(li and liang, 2021), as well as lora. this demonstrates the wide applicability of it-tuning for various tasks.