Revisiting the Knowledge Injection Frameworks (2023)

Abstract: In recent years, large language models (LLMs), such as GPTs, have attained great impact worldwide. However, how to adapt these LLMs to better suit the vertical domain-specific tasks by utilizing external knowledge remains not completely solved. Indeed, there have emerged a few works on this line where most of them rely on an alignment heuristic that is built to inject the corresponding knowledge tuple into the associated text sample. However, despite the promise, we identify a pivotal problem in this work ubiquitously. Simply put, we find that injecting unaligned (i.e., random) knowledge tuple into the LLMs achieves comparable (and sometimes better) results than the aligned knowledge being injected. We therefore take a thorough investigation of this frustrating finding on a variety of related prior work and further provide a chain of potential interpretations for the phenomenon. Based on all that, we offer a simple remediated technique. Briefly, the core of this technique roots in an ideological emphasis on the pruning and purification of the external knowledge base to be injected into LLMs. At last, we show that by integrating this technique into most (if not all) knowledge injection frameworks and recent LLMs, it manages to overcome the aforementioned sanity problem and further pushes the boundary of the performance of the domain-adaptive LLMs.

## introduction the large language models (llms) 1 -like bert @xcite , roberta @xcite , gpt-3 @xcite , chatgpt, gpt-4, etc. -truly have brought gigantic waves worldwide. while these llms have evidently extended the frontier of nlp, the prior works (ji et al., * "*" means both peng fu and yiming zhang contributed equally to this work. 1 llms are commonly referred to as pre-trained language models (plms). 2022; @xcite have also pointed out that when lacking domain-specific knowledge, llms are more prone to hallucinate in downstream tasks. notably, a relatively lightweight but promising means to tackle this is through knowledge injection such as ernie @xcite , knowbert @xcite and k-bert @xcite where an external knowledge graph is adopted as shown in figure 1 . despite the plethora of work on this line being proposed in the past, we present a pivotal problem in this work via comprehensive scrutiny. generally, this line of work relies on an alignment module where one can automatically associate a given text sample with a knowledge tuple that is extracted from an external knowledge base. this aligned knowledge tuple is then facilitated to influence the downstream task, which manifests a hybrid mixing in the input text @xcite , positional embedding @xcite or the upperlevel embedding space @xcite . our findings: in brief words, for most, (if not all) of the prior work, injecting misaligned, randomized, or (intentionally) irrelevant knowledge tuples yields comparable (and sometimes better) results than the aligned knowledge being injected. more specifically, this ablation protocol indicates a replacement of the matched (aligned) knowledge in figure 1 by a randomly drafted knowledge. these results are validated both quantitatively and qualitatively on a variety of prominent knowledge injection frameworks across 12 popular datasets. we further note that we dedicate this work to the spectrum of fine-tuning stages thanks to its lightweight nature and arguably wider real-world deployment. nevertheless, our work does not mean that knowledge injection is unfeasible as a whole. rather, the similar mechanism applied in the pretraining stage did have some successes @xcite , in spite of the forbidden computational cost incurred. to this end, we believe that there are two prioritized prospects that for an input text, the injection method first aligns the entities in the input text (as mentions m) with its corresponding entities in the external knowledge base (as entity k e ). afterward, it retrieves the external knowledge that needs to be injected through entity k e . finally, the injection model injects the input text together with the knowledge into the pre-trained model. at last, rather than composing a complete methodological solution to this newly found problem, we intend to emphasize the importance of injected item itself. in particular, we construct a new conceptual knowledge graph that is purified and pruned from other knowledge base's taxonomy, similar to @xcite . by injecting this knowledge graph into the aforementioned llm frameworks, the llms work just as expected and manage to overcome the previous sanity-checking experiments. in virtue of this workflow, we posit that our hypothesis is further strengthened and validated. we prove that this remediated technique can seamlessly be consolidated with all prior knowledge injection frameworks, and also recent llms such as chatgpt. ## preliminaries in this section, we present some preliminary concepts related to knowledge injection. and these introductions serve as a foundation for the subsequent chapters. ## random v.s. aligned in this section, we address a research question -does the performance improvement of existing injection algorithms truly attribute to the injected knowledge? to solve this issue, we design a series of ablation experiments with rigorously controlled variables to investigate the practical impact of knowledge information. the experiment results across 12 pertinent datasets demonstrate that aligned knowledge injection is not superior to random knowledge injection. in the remainder of this section, we will provide a comprehensive description of the experimental setup and present the corresponding results in detail. ## more does not mean better prior results have pointed to a devastating conclusion -the knowledge injection frameworks are not generally grounded in the knowledge injected in the fine-tuning stage. llms are more likely to treat knowledge injection in a way resemblance to white noise injection. in this section, to answer the question why the injected llms figure 3 : the relationship between the number of injected triplets and the injection effect. the vertical axis represents the f1 deviation between knowledge injection and random injection on the test set. the horizontal axis represents the number of triples extracted from each mention of the input text (for example, if there are 4 mentions in a text, a horizontal axis of 1 means that 4 triples are injected into the text). and 0.1 means that there is a probability of 10% to inject a triplet for each mention. as we posit before, excessive and overly complex injecting knowledge may be partially responsible for this issue. to validate our hypothesis, we conducted further experimental explorations by increasing the quantity of relevant knowledge injected at once. an excessive amount of knowledge may not necessarily indicate better performance, and in some cases, it could lead to a performance drop, as per figure 3 . ## a remedy by a simple method in this section, we provide an (embarrassingly) simple fix (only in fine-tuning stage) that succeeds in all the aforementioned ablation tests. to alleviate the problem of knowledge injection failure, we introduce conceptual knowledge, which may be more clean and abstract, as a remedy. to validate the effectiveness of the remedy, we devise the last piece of our protocol. injection details. in particular, we propose to alter the injected knowledge with a much cleaner and more concise one: @xmath0, k c ), where k c is the conceptual knowledge 3.3 we construct. in ## conclusion in this article, we present a comprehensive empirical study of current knowledge injection frameworks. unfortunately, with a series of testing and ablation protocols we propose, most, if not all, prior knowledge injection methods perform erroneously. we then provide an interpretation from the similarity of noise injection. we finally provide a (very) simple remediation method that may remedy the issues. with this work, we wholeheartedly encourage the community towards (i)-further checking the knowledge injection methods; (ii)-focusing a bit more on the side of the knowledge itself, rather than the entire dedication to the knowledge injec-tion mechanism or the neural architectures. at last, we humbly hope that the set of our protocols can be adapted for sanity-check in future research on this line, together with our simple remediation method applied as an additional baseline.