Learning Global Controller in Latent Space for Parameter-Efficient Fine-Tuning (2024)

Abstract: While large language models (LLMs) have showcased remarkable prowess in various natural language processing tasks, their training costs are exorbitant. Consequently, a plethora of parameter-efficient fine-tuning methods have emerged to tailor large models for downstream tasks, including low-rank training. Recent approaches either amalgamate existing fine-tuning methods or dynamically adjust rank allocation. Nonetheless, these methods continue to grapple with issues like local optimization, inability to train with full rank and lack of focus on specific tasks. In this paper, we introduce an innovative parameter-efficient method for exploring optimal solutions within latent space. More specifically, we introduce a set of latent units designed to iteratively extract input representations from LLMs, continuously refining informative features that enhance downstream task performance. Due to the small and independent nature of the latent units in relation to input size, this significantly reduces training memory requirements. Additionally, we employ an asymmetric attention mechanism to facilitate bidirectional interaction between latent units and freezed LLM representations, thereby mitigating issues associated with non-full-rank training. Furthermore, we apply distillation over hidden states during the interaction, which guarantees a trimmed number of trainable parameters.Experimental results demonstrate that our approach achieves state-of-the-art performance on a range of natural language understanding, generation and reasoning tasks.

## introduction large language models (llms) @xcite @xcite @xcite , exemplified by chatgpt, have garnered substantial attention within the scholarly and industrial realms owing to their remarkable efficacy in a plethora of natural language processing undertakings. however, full training of llm is timeconsuming and labor-intensive. besides the high training overhead, maintaining a replica for each task introduces significant storage redundancy. to address these issues, researchers either add extra neural modules @xcite @xcite or model incremental updates @xcite @xcite . more recently, researchers either merge existing fine-tuning methods @xcite or dynamically adjust rank allocation @xcite . @xcite presents a unified framework and enables an efficient combination of existing fine-tuning methods. @xcite dynamically allocates the parameter budget among weight matrices. however, these methods still have some flaws. first, local optimization is a notorious problem. as shown in figure 1 (a), lora @xcite approximates the update of each weight matrix with a pair of a i and b i . these low rank-based methods use numerous pairs of low rank matrices to fit local parameters and neglect global control, resulting in suboptimal performance. @xcite ; @xcite remains with this problem even though it makes the unalterable rank in lora to be adaptive. second, these methods mostly suffer from inability to train with full rank. @xcite finds that this problem manifests itself even more severely in ffns, and therefore assigns more parameters to ffns than to attention modules to alleviate this problem. however, as in previous approaches @xcite , they still takes a bitwise summing approach to varying the output probabilities, limiting the expressive power of the model. third, these methods focus only on approximate updates to llms and do not adequately consider task-specific relevant features @xcite . modeling the topics or labels of specific tasks will result in performance gains. to address these issues, we propose to learn a global controller (gloc) for parameter-efficient training in latent space, in which we use a set of latent units to iteratively distill information features from llm. as shown in figure 1 (b), we treats llm as a feature extractor and uses a set of latent units to perform the exchange of information with llm. by the nature of the small and independent nature of the latent units relative to the size of the input, this greatly reduces the requirement for training memory. we consider this set of latent units as a global controller that runs through all layers of the large language model, fully exploiting the capabilities of llm. in addition, we employ asymmetric attentional mechanisms to facilitate bidirectional interactions between latent units and freezed representations, hence mitigating the problems associated with non-full-rank training. further, we apply a distillation technique to the hidden states during the interaction to compress the hidden state size to a very small scale, which ensures fewer trainable parameters. we also find in our experiments that these latent units learn task-specific relevant features that show strong statistical correlation with task labels. our main contributions are as follows: ## experiments to demonstrate the effectiveness of the proposed global controller method (gloc), we conduct extensive experiments on a range of natural language understanding, generation, and reasoning tasks. ## conclusion in this paper, we present a novel global controller (gloc) approach for parameter-efficient finetuning. based on a small set of latent units, we harness the large language model from a global perspective to seek optimal performance. we design asymmetric attention mechanisms and distillation compression modules during interaction to reduce training memory while mitigating the problem of non-full-rank training. extensive experiments on a range of natural language understanding, generation, and reasnoning tasks show that our model reaches the state-of-the-art and significantly outperforms a range of robust baselines.