K now LA : Enhancing Parameter-efficient Finetuning with Knowledgeable Adaptation (2024)

Abstract: Parameter-efficient finetuning (PEFT) is a key technique for adapting large language models (LLMs) to downstream tasks. In this paper, we study leveraging knowledge graph embeddings to improve the effectiveness of PEFT. We propose a knowledgeable adaptation method called KnowLA. It inserts an adaptation layer into an LLM to integrate the embeddings of entities appearing in the input text. The adaptation layer is trained in combination with LoRA on instruction data. Experiments on six benchmarks with two popular LLMs and three knowledge graphs demonstrate the effectiveness and robustness of KnowLA. We show that KnowLA can help activate the relevant parameterized knowledge in an LLM to answer a question without changing its parameters or input prompts.

## introduction in the era of large language models (llms) with billions and possibly trillions of parameters @xcite @xcite , parameter-efficient finetuning (peft) stands out as a crucial technique enabling the necessary adaptation of llms to downstream tasks. it freezes most or even all parameters of llms and only finetunes a small number of parameters using limited instruction data. lora @xcite is a widely-used peft method that trains small low-rank adapters to approximate the large layers in llms. followup work improves the efficiency of lora by using quantized weights @xcite . in this work, we seek to improve the effectiveness of lora while preserving comparable efficiency. inspired by knowledge-injected pre-trained language models (plms), e.g., ernie @xcite , we explore knowledge graphs (kgs) to enhance the peft of llms with lora. a kg is a large-scale structured knowledge base containing a massive amount of trustworthy knowledge. the typical way of injecting kgs into plms in the past few years is incorporating pre-trained entity embeddings at the input layer of a plm and finetuning the full model on nlp tasks @xcite @xcite @xcite @xcite . knowledge injection has improved many plms, e.g., bert @xcite and roberta @xcite . however, previous knowledge injection methods require fully tuning plms, which is inapplicable to llms. furthermore, these methods are founded on the encoder-based architecture of plms, and their effectiveness for recent decoder-based llms remains unknown. the following questions thereby arise: can knowledge injection still enhance the peft of llms? also, how can knowledge injection be used to enhance peft? to answer these questions, in this paper, we propose a knowledgeable adaptation method for peft, particularly for lora, called knowla. it inserts an adaptation layer into a pre-trained llm. the layer integrates external kg embeddings of entities appearing in the input text of the llm. entity embeddings and parameters of the llm are frozen in peft. the proposed adaptation layer is trained combined with lora on instruction data. the parameters in our adaptation layer are significantly fewer than those in the llm and even fewer than those in lora. thus, our knowla is also a parameter-efficient method without changing the original parameters of the llm. we evaluate knowla on six datasets, including commonsense reasoning on commonsenseqa @xcite , social interaction reasoning on siqa @xcite and @xcite , single-hop reasoning of kbqa on webquestionsp @xcite , and closebook qa on triviaqa @xcite and truthfulqa @xcite . experimental results show that knowla can enhance the effectiveness of lora at the expense of a limited number of additional parameters. even when compared to al-paca2 @xcite , which has a larger lora with a similar number of parameters, knowla with a smaller lora achieves better results. we assess the robustness of knowla with two popular foundation models (i.e., llama 1 @xcite and llama 2 @xcite ), different instruction data (i.e., instructionfollowing demonstrations in alpaca2 and vicuna2 @xcite ), various kgs (i.e., wordnet @xcite , conceptnet @xcite , and wikidata (vrandecic and krötzsch, 2014)), and typical embedding learning models (i.e., rescal @xcite , transe @xcite , and rotate ( sun et al., 2019) ), combined with two peft methods (i.e., lora @xcite and adalora @xcite ). experiments show that knowla can offer stable improvements. to understand how knowla changes the output of an llm, we analyze the results from two perspectives, which show several interesting findings: (i) knowla with lora can align the space of the llm with the space of kg embeddings, and (ii) knowla can activate the parameterized potential knowledge that originally exists in the llm, even though the used kg does not contain such knowledge. according to our findings, in some cases, the llm outputs incorrect answers not because it does not know the answers, but because its relevant knowledge is not activated by the input prompts. knowla can help activate its relevant knowledge without changing its parameters or input prompts. ## knowla considering that the hidden states in transformer layers encapsulate the parameterized knowledge of an llm @xcite , we propose fusing entity embeddings in a kg with the hidden states of an llm during peft. knowla inserts an adaptation layer into an llm, as shown in figure 1 . given a kg, we adopt a representation learning model, e.g., transe @xcite , to train its entity embeddings. the pre-trained embedding of entity e is denoted by e. for an input question @xmath0, each token t i may be linked to a set of entities e(t i ) in the kg. our key idea is to enhance peft by injecting the embedding e i for each e @xmath1. this method can be divided into three modules: (i) entity linking, which links the tokens in a question to entities in the kg. (ii) knowledge mapping and injection, which maps the kg embedding space to the llm's representation space and infuses the entity embeddings corresponding to a specific token in the question. (iii) knowledge fusion, which integrates each token representation with its entity embedding. given the powerful abilities, popularity, and open-source nature of the llama family @xcite , we consider it the foundation to build our knowla.