Revisiting the Knowledge Injection Frameworks
Authors: Peng Fu
Yiming Zhang
Haobo Wang
Weikang Qiu
Junbo Zhao
Venue: EMNLP 2023 Main (2023)
Keywords: language models
knowledge pruning
large language models for downstream task

Abstract: In recent years, large language models (LLMs), such as GPTs, have attained great impact worldwide.
However, how to adapt these LLMs to better suit the vertical domain-specific tasks by utilizing external knowledge remains not completely solved.
Indeed, there have emerged a few works on this line where most of them rely on an alignment heuristic that is built to inject the corresponding knowledge tuple into the associated text sample.

However, despite the promise, we identify a pivotal problem in this work ubiquitously.
Simply put, we find that injecting unaligned (i.e., random) knowledge tuple into the LLMs achieves comparable (and sometimes better) results than the aligned knowledge being injected.
We therefore take a thorough investigation of this frustrating finding on a variety of related prior work and further provide a chain of potential interpretations for the phenomenon.
Based on all that, we offer a simple remediated technique.
Briefly, the core of this technique roots in an ideological emphasis on the pruning and purification of the external knowledge base to be injected into LLMs.
At last, we show that by integrating this technique into most (if not all) knowledge injection frameworks and recent LLMs, it manages to overcome the aforementioned sanity problem and further pushes the boundary of the performance of the domain-adaptive LLMs.

Revisiting the Knowledge Injection Frameworks
Peng Fu'l, Yiming Zhang Haobo Wang', Weikang Qiu', Junbo Zhao'
"Zhejiang University. Hangzhou, China Yale University, New Haven, America {p_fu, wanghaobo, j. Jezja weikang. qiu@yale edu
Abstract
2022: Bang et al., 2023) have also pointed out that when lacking domain-specinn knowledge, LLMs are more prone to hallucinate in downstream tasks. Notably, a relatively lightweight but promising means to tackle this is through knowledge injection such as ERNIE (Zhang et al., 2019), Know (Pe- ters et al., 2019) and K-BERT (Liu et al., 2020) where an external knowledge graph is adopted as shown in Figure 1. Despite the plethora of work on this line being proposed in the past, we present pivotal problem in this work via comprehensive scrutiny. Generally, this line of work relies on alignment module where one can automatically sociate a given text sample with a knowledge tuple that extracted from an external knowledge base. This aligned knowledge tuple is then facilitated to influence the downstream task, which manifests a hybrid mixing in the input text (Liu et al., 2020), positional embedding (He et al., 2020) or the upper- level embedding space (Zhang et al., 2019).
In recent years, large language models (LLMs), such as GPTs, have attained great impact world- wide. However, how to adapt these LLMs to better suit the vertical domain-specine tasks by utilizing external knowledge remains not completely solved. Indeed, there have emerged a few works on this line where most of them rely on an alignment heuristic that is built to inject the corresponding knowledge tuple into the associated text sample.
However, despite the promise, we identify piv- otal problem this work ubiquitously. Simply put, find that injecting unaligned (i.e., ran- dom) knowledge tuple into the LLMs achieves comparable (and sometimes better) results than the aligned knowledge being injected. We therefore take a thorough investigation of this frustrating finding on variety related prior work and further provide a chain of potential interpretations for the phenomenon, Based on all that, offer simple remediated technique. Briefly, the core of this technique is rooted in an ideological emphasis on the pruning and purification of the external knowledge base to be injected into LLMs. At last, we show that by integrating this technique into most (if not all) knowledge injection frameworks and recent LLMs, manages to overcome the aforemen- tioned sanity problem and further pushes the boundary of the performance of the domain- adaptive L.MSs.
Our findings: In brief words, for most, (if not all) of the prior work, injecting misaligned, random- ized, or (intentionally) irrelevant knowledge tuples yields comparable (and sometimes better) results than the aligned knowledge being injected. More specifically, this ablation protocol indicates a re- placement of the matched (aligned) knowledge in Figure 1 by randomly drafted knowledge. These results are validated both quantitatively and qual- itatively on variety of prominent knowledge in- jection frameworks across 12 popular datasets. We further note that we dedicate this work to the spec- trum of fine-tuning stages thanks to its lightweight nature and arguably wider real-world deployment. Nevertheless, our work does not mean that knowledge injection is unfeasible as whole. Rather, the similar mechanism applied in the pre- training stage did have some successes (Ye et al., 2022: Wang et al., 2021b), in spite of the forbid- den computational cost incurred. To this end, we believe that there are two prioritized prospects that
1 Introduction
The large language models (LLMs)) like BERT (Devlin al., 2019), RoBERTa (Liu al., 2019), GPT-3 (Brown al., 2020), ChatGPT, GPT- 4, etc. - truly have brought gigantic waves world- wide. While these LLMS have evidently extended the frontier of NLP, the prior works (JI et al., Yiming Zhang contributed ML. are commonly referred to as pre-trained language models (PLMs).
**" means both Peng equally to this work.

---

For input text: WorldCom's rival, AT & announced a similar restructuring plan.
Text-GG Alignment
aligned
entity
mentions(In)
WorldCom's rival, AT & announced similar restructuring plan.
01881291
Knowledde injection
knowlege embedding
knowledge
6,009884355161155
triples
00,001044
LLMs
or
Vertron (Q867752))
Injecting Knowledger
The Process of Common Knowledge Injection. For an input text, the injection method first aligns the entities in the input text (as mentions m) with its corresponding entities the external knowledge base (as entity ke). Afterward, it retrieves the external knowledge that needs to be injected through entity kg. Finally, the injection model injects the input text together with the knowledge into the pre-trained model.
Figure I:
ought to be explained studied: (i)-to revisit the tuning dynamism2 by injected knowledge and (ii)- derive fix the problem.
knowledge graph into the aforementioned LLM frameworks, the LLMs work just as expected and manage to overcome the previous sanity-checking experiments. virtue of this workflow, we posit that our hypothesis is further strengthened and vali- dated. We prove that this remediated technique can seamlessly be consolidated with all prior knowl- edge injection frameworks, and also recent LLMs such as ChatGPT.
To do that, we first supplement a set of addi- tional experiments by injecting Gaussian noise as a replacement for injected knowledge. Perhaps with surprises, this set of injection flow winds up with indiscernible results with either randomized or aligned knowledge injection. The major question we come up with this far therefore directed to why the LLMs treat the aligned knowledge similarly to noise. In pursuit of the answer to this question, we cast our hypothesis: Within the fine-tuning scheme, the LLMs fail to adequately disentangle the intricacy possessed in the external knowledge base so as to treat the injected item ubiquitously as noise. For instance, ERNIE (Zhang et al., 2019) intends to integrate wiki knowledge graph that is composed of a vast of more than 5 mil- lion entity nodes. We thereby vaguely connect this hypothesis - together with the prior empirical con- clusion - to data augmentation that explains why both randomized knowledge and noise injection still renders some performance gain.
Related Works
2.1 Knowledge injection for LLMs
Recently, the emergence of large pre-trained lan- guage models, such as ChatGPT and GPT-4, has at- tracted great attention from the community and the public, due to their emergent abilities demonstrated in many tasks. Although ChatGPT includes lot of knowledge through pre-training, the knowledge injection method is still necessary because Chat- GPT cannot fully solve problems in professional fields, such as healthcare (Wu et al., 2023; Liu al., 2023). For this problem, LLMs can pre-train on professional field corpora or retrieve documents (like New Bing) to obtain that knowledge. How- ever, these methods may incur substantial costs and pose a challenge wherein the obtained knowl- edge may not align seamlessly with the internal knowledge of the models. We aim integrate the external structured knowledge sources in a more concise and convenient way, rather than updating the internal parameters of LLMs..
At last, rather than composing a complete methodological solution to this newly found prob- lem, we intend to emphasize the importance of construct injected item itself. In particular, new conceptual knowledge graph that is purified and pruned from other knowledge base's taxonomy, similar to McCrae et (2019). By injecting this
"This refers to the fine-tuning mechanism.

---

2.2 Sowwledge-Enhnneee Models
Since the large-scale application of pre-trained models in the NLP field, many works expect to improve the downstream tasks' performance by integrating external knowledge. Among those knowledge-enhanced models, many works use knowledge representation--bseed methods to incor- porate factual knowledge (Zhang et al., 2019; Su et al., 2021; Ye et al., 2022; Peters et al., 2019; Wang al., 2021b; Yamada et al., 2020; Sun et al., 2020; He et al., 2020; Yuan al., 2021). Other models use other forms integrate knowledge into the model (Liu et al., 2020; Wang et al., 202 Meng et al., 2021; Hosseini et al., 2022,?; Ke et al., 2020; Lu et al., 2022).
Internediste Layer
Altention
coaaderd
diectonii
w:lle gpines
Figure 2: Word And Knowledge Embedding Fusion Pro- cess Diagram. Word embedding (blue) and knowledge embedding (green) usually infused in the intermedi- ate layer, and after that, the related tokens may contain some knowledge-rees information, as shown in the iconic framework ERNIE (Zhang et al., 2019).
Among those works, some achieved eye- catching performances different downstream tasks. To name few, ERNIE (Zhang et al., 2019) integrates entities' knowledge aligned with the mentions of the input text in the pre-training and fine-tuning stage. LUKE (Yamada et al., 2020) proposes an aware self-attention mechanism and forms multi-way injection summarizing both words and entities. KnowBert (Peters et al., 2019) incorporates additional entity disambiguation module towards improving the entity linker and recombines knowledge features for injection. K- BERT (Liu et al., 2020) converts the relation triples with the context into the sentence tree, then encodes them assisted by a novel soft positional encoding method. Although they designed various injection mechanisms, they do not discuss and analyze the research questions in depth. To some extent, this makes their works lack interpretability.
Regardless, we believe proper study on the trans- parency of knowledge injection is somewhat criti- cal. This line of work is still at its early stage and is often neglected deprioritized by prior works. With the study of this work, may humbly alert the community by showcasing some negative re- sults yielded by our proposed protocol.
3 Preliminaries
In this section, we present some preliminary con- cepts related to knowledge injection. And these introductions serve foundation for the subse- quent chapters.
3.1 Text-KG Alignment
As prerequisite step, needs align the knowl- edge graph its subgraph the input text. For a standard method, uses the entity alignment tool such as TagMe (Ferragina and Scaiella, 2010a) to detect KG's entities mentioned the input text and link them to the correct KG entry, then tuple them together (Broscheit, 2019). Specifically, given knowledge graph G and sentence æ, this process can be defined as
2.3 Interpretable Analysis In LLMs
The closest to this work is the transparency and interpretability analysis of knowledge injection frameworks. While there has been much work covering it, as we go deep into the literature: Pe- ters et (2019): Jiang et al. (2020); Cao et al. (2021) have proved that a pre-trained language model can acquire substantial factual knowledge via pre-training on large-scale unlabeled data. Li et al. (2022) analyzes the capacity of LLMs from the aspect of capturing factual knowledge. Zhang al. (2021) exhibits that injecting redundant and ir- relevant knowledge causes an efficiency drop. Hou al. (2022) shows there is no positive relation- ship between knowledge injection corpus size and knowledge injection quality.
(1) where denotes entities mentioned in (mention entity or mentions), ke denotes the linked entities (a kind of factual knowledge) in G, and means the entity linking or alignment tool.
m, he h(x, G).
3.2 Knowledge Injection Methods
From a high-level standpoint, the mission of knowl- edge injection methods aims to inject an exter- nal source of knowledge into the language mod-

---

MedicalNER
MedicalNER
MedicalNER
FinancialNER
FinancialNER
+ HowNet FI
Setup
MedicalKG
HowNet
CuDbpedia F1
CnDbpedia F1
BERT (w/ KI)
92.5
86.1
K-BERT K-BERT-Aligned
94.2
93.3
93.8
87.3
87.4
94.00+0.118
93.51+0.15
93.48+0.26
87.49-0.08 07.35=0.19
87.37+0.19
K-B K-BERT-Rale MERT-Random
93.891:0.3
93.52.10.22
0.255:10.25
0.14
(a) Results of K K-BERT. K-BERT-A K-BERR-Aligned and K- K-BERT-Randon correspond setup 1, 2 respectively. The results BERT and K-BERT come from Liu al. (2020).
BCSchem F1
BC5dis F1
NCBI F1
BC2GM F1
JNLPBA FI
Setup
BioBERT (w/o KT)
92.9
84.7
89.1
83.8
79.4
KeBioLM
93.3
86.1
89.1
85.1
82.0
M-BigoMM-lligee
93.24-00.71
87.96=1.055
08.4
83,99:10.22
78.81+22.51
KeBioLMM-andonm
93.06-10.69
88.57
08.9
83.25
78.81122488
(b) Results of MeBioLM. KeBioLM-Alignn and KeBioLM-Randee Random correspond setup 1, respectively. The results of BioBERT and MeBioLM come from Yuan al. (2021) and BioBERT is LLM pre-trained on biomedical corpora.
Table 1: Results of Named Entity Recognition Task. All these experiments are run 5 times with varying random seeds.
Open Entity R
TACRED
Setup
R
F1
BERT (w/o KI)
76.37
70.96
73.56
67.23
64.81
66,00
ERNIE ERNIE-Aligned
78.42
72.90
75.56
69.97
66.08
67.97
1.08
72.15=0.92
75.33=00.41
1.60
58.15-5.88
63.79:33.00
ERNIF-Random
1..8
73.12=1.07
75.37-00.11
68.2945.911
58.08-5.85
63.73++.644
Table 2: Results of ERNIE on Open Entity and TACRED. ERNIE ERNIE-Aligned and ERNIE- Random correspond setup 5 times with varying random seeds. The results of BERT and ERNIE come from Zhang et al. (2019). The drop in performance of ERNIE on TACRED may be attributed to the data quality issues inherent in the dataset itself (Alt et al., 2020; Stoica al., 2021).
1, 2 respectively and all these experiments
els, with an ultimate goal of better suiting the models to downstream tasks, particularly the low- source domains. Throughout the literature, there has emerged few separate branches shedding light on different paradigms.
may use the simplest form to represent the knowl- edge injection process, as follows:
y f(x,k),
(2)
where denotes the input text, k denotes the in- jected knowledge regardless of its instantiated form, y denotes the corresponded gold label and f indi- cates a trainable neural network. Notice, this investigative work, we cover many instantiations of f and k. including not only ERNIE, KnowBert, and other models that integrate external knowledge, but also ChatGPT, GPT-4, etc., mainly for knowledge injection during the fine-tuning stage to achieve optimal empirical transparency.
To begin with, the major division of this line can be categorized by injection during the pre-training stage or the fine-tuning stage. Hereby, we use the iconic framework, ERNIE (Zhang et al., 2019), for demonstration. On one hand, in the pre-training stage of knowledge injection, ERNIE forms sepa- rate masked language modeling objective. Specifi- cally, it randomly masks off the linked entities and has additional softmax head to recover it. On the other hand, during the fine-tuning stage, ERNIE fuses the text and aligned knowledge in the vecto- rial representation space, as shown in Figure 2.
3.3 The Different Injected Knowledge
To explore the above questions, design a set of ablation experiments with strictly controlled vari- ables. In those ablation experiments, we follow the previous protocol with the origin knowledge- injected models, only changing the knowledge they inject. That knowledge includes:
Notice, the purpose of this paper is not to pro- pose a novel knowledge injection scheme, nor to promote any existing method. Therefore, abstract- ing away from one specific showcasing method, we

---

SQuAD
Setup
Dev Acc
Dev
WiC
Sctup
Dev Acc
BERT-large (w/o KI)
84.1
90.9
LUKE LUKE--ligned
$6.1
92.3
AnowBerr-Alignee
69.53:11.24 69.25+
86.22-0.337
92.34+00.09
KnowBert -aneet
LUKE-Random
86.15=0.155
9.11
Table Results of KnowBert, KnowBert-Aligned and KnowBeri-Randood correspond setup 1, 2 spectively and all these experiments are run 5 times with varying random seeds.
Table 3: Results of LUKE, LUKE-Aligned and LUKE- Random correspond setup 1, respectively and all these experiments are run 5 times with varying random seeds. The results of BERT-large LUKE come from Lan et al. (2019) and http://gttub.cco//suuiioooosssss
Aligned Knowledge: refers to the retrieved knowledge that is injected into the model, as done by all prior work, described as k. The Knowledgw Graph alignment process is often conducted beforehand.
of knowledge information. The experiment re- sults across 12 pertinent datasets demonstrate that aligned knowledge injection is not superior to random knowledge injection. In the remainder of this section, we will provide a comprehensive description of the experimental setup and present the corresponding results in detail.
Random Knowledge: refers to the random se- lection of a knowledge point from an external knowledge base and using it in the same form as aligned knowledge, denoted as random- No alignment process is conducted.
A Ablation Protocol. In hindsight, the effect of knowledge injection can be decomposed into two parts: (i)-the knowledge injection mechanism as to how to inject it and (ii)-the knowledge itself as to what to inject. The very majority, if not all, of the prior work is dedicated to the (i) and uses final performance as the sole metric check if the injection works. Nevertheless, further enhance the transparency of the system, we wholeheartedly believe that both conditions shall be studied and met. In that regard, to complete the picture, we focus majorly on (ii).
Wiki Triples Knowledge: refers to the triples extracted from WikiData5M (Wang et al., 2021b), where the knowledge graph only com- poses entity id from the linked ones ke. The triples are deseribed as k1, ..,..., ......,,,.... kn where represents the length of a triplet matched by an entity ID. Entity-linkkin is conducted necessarily before retrieving triples.
Briefly, we intend to substitute the previously- added knowledge with the random knowledge 3.3, and assess the performance of the original injection (with aligned knowledge 3.3) in comparison. particular, we adopt the following settings: 1. knowledge injection refers to injecting aligned knowledge in the training and testing process, which can be described in Equation 2;
Conceptual Knowledge: refers to the knowl- edge extracted from Wikidata and Wordnet, denoted as kic Specifically, for an entity id in ke, we extract its title and type from Wiki- data and find the corresponding concept from Wordnet. Finally, combine title, type, and concept into triplet such (title, type, con- cept), as the conceptual knowledge of the cor- responding entity. The entity-linking process is also necessary to obtain conceptual knowl- edge.
2. random injection refers to injecting random knowledge in the training and testing process. Other experimental settings, like baseline and consistent with the knowledge injection. can formally be defined as y f(z krandom;;
fine-tuning configurations,
4 Random v.s. Aligned
In this section, address research question Does the performance improvement of existing in- jection algorithms truly attribute to the injected knowledge? To solve this issue, we design a se- ries ablation experiments with rigorously con- trolled variables to investigate the practical impact
3. noise injection refers to injecting randomized Gaussian white noise in the training and test- ing process, as f(z e).
Backbones and Datasets. Indeed, different downstream tasks may require different knowledge

---

MedicalNER
MedicalNER
MedicalNER
FinancialNER
FinancialNER
Setup
+
MedicalKG
HowNet
CaDbpedia
HowNet
CuDbpedia
BERT (w/o KI)
92.5
86.1 87.35+0..15 87.15=0.111
K-RERT-Random K-BERT-Noise
93.89+0..27 93.79:+0.32
93.52+0.18 93.73:20113
03.55+0.20 93.80:10113
87.46-0.09 87.10=0.11
(a) Results of noise injecting to K-BERT for NER datasets. The results of BERT come from Liu et al. (2020)
Open Entity R
SQuAD 1.1
Setup
Sctup
P
Dev Ace
Dev Fl
BERT (w/o 76.37
70.96 72.15-0.92 72.98-0.42
73.56 75.33-10.41 75.07:10.06
LUKE-Rane AUKE-Random
06.32
0.09 02.33=00004
MAAI--Raddom 78.81
LUKE-NON Noise
86.09=t0.48
ERNIE-Noise
77.28=10.54
(b) Results noise injecting to LUKE. The results of BERT-large come from Lan et al. (2019)
(c) Results of noise injecting to ERNIF. The results of BERT come from Zhang al. (2019).
Table 5: Results of Gaussian Noise, All these experiments are run times with varying random seeds.
types, scales, or quantities. Distinctive knowledge injection methods and model backbones for differ- ent NLP applications may also vary widely. To take a comprehensive revisit of the knowledge- enhanced models, we choose the most advanced as well as the best performing knowledge-injectee LLMs as our baselines, in correspondence to the different benchmarks. Following the aforemen- tioned principles, we primarily choose LUKE (Ya- mada et al., 2020), ERNIE (Zhang al., 2019), KnowBERT (Peters et al., 2019), K-BERT (Liu al., 2020) and KeBioLM (Yuan al., 2021) as the major backbones/methods for the purpose of the study. The methods can be primarily classified into two categories: text- bext- methods, exemplified by K-BRERT, and embedding- based methods, such as KnowBert, ERNIE, LUKE, and KcBioLM. In the meantime, we cover most of the major datasets. The details and stats of them provided in Ap- pendix A. and A.2. And for the information knowledge graphs, please refer Appendix A.3.
tions, we have reason to believe that the model may treat knowledge injection in way resemblance white noise injection.
To further explore the differ- ence between knowledge injection and random in- jection, we compare their similarity the encoder and the output of the classifier in Open Entity and TACRED and find the differences are also small. For the details of the further exploration, please refer to Appendix B.
Further Analysis.
Takeaways @. Through these experiments, we discover that the previous approaches of knowl- edge injection, random injection, and even noise injection do not produce notable distinctions. It renders us that they may not be regarded as favor- able choices. Drawing from previous analyses, we observe that prior works (Zhang et al., 2019; Ya- mada et al., 2020) tend to emphasize the injection method itself rather than considering the model's 's ability to accurately perceive and comprehend the injected knowledge. This could be identified the underlying cause of the problem.
Main Results And Discussion. Exhibited in Ta- ble 1 to 5c, we conclude that: (i) the knowledge injection is not superior random injection. The differences between them generally within 1.0, and some are even lower than 0.1; (ii) the difference between random injection and noise injection is also much neglectable, ranging by no more than 0.3 by F1.
5 More Does Not Mean Better
Prior results have pointed to a devastating conclu- sion - the knowledge injection frameworks are not generally grounded in the knowledge injected in the fine-tuning stage. LLMs are more likely to treat knowledge injection in a way resemblance white noise injection. In this section, to answer the question why the injected L.MS treat the in- jected knowledge as noise during the fine-tuning stage, we begin to analyze the reasons behind this
These phenomenons can be further inferred that the knowledge-injected models do not adequately make use of the knowledge injected in the fine- tuming stage, which may be a fatal problem for those injection models. Upon those closer examina-

---

The absolute deviation
The absolute
0.80
300
2.77
0.70
0.67
2:50
0.60
2.16
Deviston
0.50
Deviation
2.00 2.11
0.40
0.38
1.71
161
0.30
0.29
1.50
1,25
0.20
0.1
0.16
0.16
1.00
0.00
0.0
0.1
0.2
0.5
max
0.1
0.
0.5
2
max
Triples
Triples
(a) The deviation variation Open Entity.
(b) The deviation variation FewRel.
The relationship between the number injected triplets and the injection effect. The vertical axis represents the Fl deviation between knowledge injection and random injection the represents the number of triples extracted from each mention of the input text (for example. if there are 4 mentions in text, horizontal axis of means triples are injected into the text). And 0.1 means that there is probability of 10% to inject triplet for each mention.
Figure 3:
The horizontal axis
phenomenon.
the amount of injecting knowledge we limited and (ki, h2,..., , kin refer to wiki triples knowledge 3.3. To strictly control the variables and correspond to the previous experimental analysis, we use BERT- base the baseline and keep the same fine-tuning settings as ERNIE.
As we posit before, excessive and overly com- plex injecting knowledge may be partially respon- sible for this issue. To validate our hypothesis, we conducted further experimental explorations by increasing the quantity relevant knowledge in- jected at once. An excessive amount of knowledge may not necessarily indicate better performance, and in some cases, it could lead to performance drop, as per Figure 3.
Analysis. Figure shows the performance dif- ference change between knowledge injection and random injection on Open Entity and FewRel in the text-based method experiment, With the increase of injected knowledge. We could observe that the number of injected triples increases, the disparity between knowledge injection and random injection diminishes on the whole.
Experiment Details. In previous ablation exper- iments, there are mainly two types of knowledge injection in the fine-tuning stage, text-based and embedding-based It is hard to design related ex- periments in embedding- based methods, for adding different knowledge embed- dings at single point may lose a lot of information from those knowledge embeddings. However, text- based noowledge-ijjecter methods like K-BERT, which are designed for mass knowledge injection (such as the design of soft-position embedding and seeing layer), are unfit for this kind of experiment.
wewledge-injectern
Takeaways 2. More does indeed not mean bet- ter without controlling knowledge purity. Conse- quently, it is imperative direct our focus from injecting more knowledge injecting more refined and targeted knowledge.
A Remedy by a Simple Method
In this section, we provide an (embarrassingly) sim- ple fix (only in fine-tuning stage) that succeeds in all the aforementioned ablation tests. To alleviate the problem of knowledge injection failure, we in- troduce conceptual knowledge, which may be more clean and abstract, as a remedy. To validate the ef- fectiveness of the remedy, we devise the last piece of our protocol.
Based on these, we design a straightforward text- based injection method. This method involves utilizing the corresponding title of the mentions to label the mentioned entity within the text and appending the corresponding triplet of the men- tions at the text's conclusion. To give concrete example, given the original input text, Grumpy Cat, the internet's most famous cat, died at 7 years old. is transferred be: *Grumpy Cat* Grumpy Cat, the internet's most famous cat, died at years old. (Grumpy Cat type cat). can be de- fined as y = f(x, (Ri, k2...., ......,,, kn)), where n is
Injection Details. In particular, propose alter the injected knowledge with a much cleaner and more concise one: = f(z ke), where ke is the conceptual knowledge 3.3 we construct.

---

Open Entity
Setup
P
BERT (wlo KT)
76.37
70.96
73.56
ConceptualKI-Randoos (Ours)
77.18:100.7
73.11=:0.61
75.09=00.53
ConceptualKI-Alige Alligned (Ours)
1.75
73.54+0.99
75.47+0044
Table Results of Our method on Open Entity. ConceptullKI-Aligeee (Ours) and ConceptualKI- Random (Ours) correspond setup 1, 2 respectively and all these experiments run 5 times with varying random seeds. The results of BERT come from Zhang et al. (2019).
FewRel R
TACRED R
Setup
P
F1
P
F1
BERT (w/o KI)
85.05
85.11
84.89
67 .3
64.81
66.00
ConceptualKI-Raan RRandom (Ours)
83.54=00.61
83,60=0.661
83.43=+0.62
70.87=0.77
54.60- 12.14
61.04+88.84
ConceptualKl-AAiggan (Ours)
87.47-10.06
87.41-0.005
87.34-10.06
70.81
62. 62.80-12.30
66.5411..36
Table 7: Results of Our method FewRel and TACRED. ConceptualKI- (Ours) and Conceptual/I Randomm (Oury) correspond setup I, 2 respectively and all these experiments are run 5 times with varying random seeds. The results of BERT come from Zhang et al. (2019).
contrast to the factual knowledge base Wikidata (including more than 80 million entities), the con- ceptual knowledge base Wordnet is significantly smaller, consisting of only 117 thousand concepts. And as the conceptual network (structure of Word- net) deepens, the conceptual knowledge base can be refined and pruned to greater extent. With the conceptual knowledge, the example section is transferred to be: *Grumpy Cat Grumpy Cat, the internet's most famous cat, died at 7 years old. (Grumpy Cat cat animal).
"Injection Details", without triples in the end;
Group retains and injects all the wiki triple knowledge (K. ju).. as = f(r, (ki, .2,....,k,)). kk)))
Group injects conceptual knowledge kes ex- actly as y - f(x kc).
The results demonstrate that both Group 1 and Group 2 exhibit an accuracy level of 88%. Con- versely, Group 3, which incorporates conceptual knowledge, achieves higher accuracy rate of 92 92,, by an absolute 4% enhancement. It implies that concept injection may exert a discernible impact on ChatGPT. For more experimental information. please refer to Appendix D.1.
We choose BERT-basc as backbone and baseline, and follow the previous protocol. Notice that, among the comparisons, all setups are kept identical except for the different forms of injected knowledge. As shown Table and 7, we draw the following observations: (i)- the performance difference between correct knowl- edge injection and random injection has been appar- ently enlarged compared to previous sections, e.g. +3.91 Fl on the two (ii)-this difference on Open Entity remains rela- tively smaller (0.32 F1), but still better than previous ablation experiment results (50.06 FI). We speculate that this might be caused by the small scale (only 2000 samples for training and testing each) of the dataset.
Main Results.
Takeaways @ Pruning the knowledge source is essential for successful knowledge injection into language models.
Conclusion
datasets;
In this article, we present a comprehensive empir- ical study of current knowledge injection frame- works. Unfortunately, with series of testing and ablation protocols we propose, most, not all, prior knowledge injection methods perform erroneously. We then provide an interpretation from the similar- ity of noise injection. We finally provide (very) simple remediation method that may remedy the issues. With this work, wholeheartedly encour- age the community towards (i)-further checking the knowledge injection methods; (ii)-focusing a bit more on the side of the knowledge itself, rather than the entire dedication to the knowledge injec-
Experiment in ChatGPT To test the practical effectiveness of concept injection in ChatGPT, extracted some data from TACRED. This experi- ment was divided into three groups: 1. Group adopts the text format of paragraph

---

tion mechanism or the neural architectures. At last, we humbly hope that the set of our protocols can be adapted for sanity-check in future research on this line, together with our simple remediation method applied as an additional baseline.
and inspire future research render the purity of knowledge.
References
Christoph Alt, Aleksandra Gabryszak, and Leonhard Hennig. 2020. Tacred revisited: A thorough eval- uation of the tacred relation extraction task. arXiv preprint arXiv:2004. 14855.
Acknowledgement
This work is majorly supported by the NSFC under Grants (No. 62206247), and in part by the National Key Research and Development Program of China (No. 2022YFB3304101) JZ also thanks the spon- sorship by the Fundamental Research Funds for the Central Universities (No. 226 226-2022-00028).
Yejin Bang. Samuel Cahyawijaya, Nayeon Lee, Wen- liang Dai, Dan Su, Bryan Wilie, Holy Lovenia, Ziwei Ji, Tiezheng Yu, Willy Chung, et al. 2023. A multi- task, multilingual, multimodal evaluation of chatgpt reasoning, hallucination, and interactivity. arXiv preprint arXiv:2302.044:
Limitations
Olivier Bodenreider. 2004. The unified medical lan- guage system (umls): integrating biomedical termi- nology. Nucleic acids research, 32(suppl_ (xssupp_1):D267.. D270.
We present our limitations of this work in this sec- tion. First, we only pick the most influential, repre- sentative and iconic knowledge-injection methods and datasets to form the main body of investigation. Admittedly, there are other works proposed in re- cent years (introduced in Section 2.2), but that is perhaps beyond the scope of this paper.
Samuel Broscheit. 2019. Investigating entity knowledge in bert with simple neural end-to-end entity linking. In Proceedings of the 23rd Conference on Computa- tional Natural Language Learning (CoNLL), pages 677-685.
Second, we primarily dedicate our extensive study to the knowledge injection performed dur- ing tne-ttunng. The reasons are three-fold: (i)- knowledge injected within the fine-tuning stage is the most dominant paradigm in a real-world appli- cation, compared to the prompting schemes with pre-training which is significantly more unstable; (ii) kil-llowed injection in the fine-tuning stage ex- tracts much less computational and carbon cost, so most of the research groups worldwide can freely reproduce our results; (iii)-if we extrapolate into the future of the LLMs, it is trendy that these mod- els' sizes may keep growing. At that point, we believe the portions of the model (say, the first cou- ple layers, some intermediate layers, or the penul- timate layers, respectively) can still be fine-tuned and manageable. By contrast, pre-training a whole large LLM with external knowledge-incorpoaated and/or prompted data would become exponentially harder.
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert- Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Ma- teusz Litwin, Scott Gray. Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, 1lya Sutskever, and Dario Amodei. 2020. Language models are few-shot learners. In Ad- vances in Neural Information Processing Systems, volume 33, pages 1877-1901. Curran Associates, Inc.
Boxi Cao. Hongyu Lin, Xianpei Han, Le Sun, Lingy- ong Yan, Meng Liao, Tong Xue, and Jin Xu. 2021. Knowledgeable or educated guess? revisiting lan- guage models as knowledge bases. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the I 1tth International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 1860-1874.
Last but perhaps not least, due to computational limitations, we conduct the relevant experiments only on BERT and RoBERTa models. However, We also provide primary investigation on LLMs, such as ChatGPT. The work's primary objective is explore and contribute integrating external knowledge into Language Model architectures. We aim to provide a reference and assistance for fu- ture research in knowledge data-centric approaches
Eunsol Choi, Omer Levy. Yejin Choi, and Luke Zettle- moyer. 2018 Ultra-fine entity typing. In Proceed- ings the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).
pages 87-96.
Nigel Collier and Jin Lin-Dong Kim. 2004. Introduction to the bio-entity recognition task at jnlpba. In Proceed- ings of the International Joint Workshop on Natural Language Processing Biomedicine and Appli- cations (NLPBA/BioNLL)) pages 73-78.

---

Jacob Devlin, Ming-Wor Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language under- standing. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguisties: Human Language Tech- nologies, Volume 1 (Lon and Short Papers), pages 4171-4186, Minneapolis, Minnesota. Association for Computational Linguistics.
Zhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel, Piyush Sharma, and Radu Soricut. 2019. Albert: A lite bert for self-supervised learn- ing of language representations. arXiv preprint wwXiv:/909.111
Jiao Li, Yueping Sun, Robin J Johnson, Daniela Sci- aky, Chih-Hsuan Wei, Robert Leaman, Allan Peter Davis, Carolyn Mattingly, Thomas Wiegers, and Zhiyong Lu. 2016. Biocrcative cdr task corpus: resource for chemical disease relation extraction. Database, 2016.
Rezarta Islamaj Dogan, Robert Leaman, and Zhiyong Lu. 2014. Nebi disease corpus: a resource for dis- ease name recognition and concept normalization. Journal of hiomedical informatics, 47:1-10.
Shaobo Li, Xiaoguang Li, Lifeng Shang. Zhenhua Dong, Cheng-Jie Sun, Bingquan Liu, Zhenzhou Ji, Xin Jiang, and Qun Liu. 2022. How pre-trained language models capture factual knowledge? causal-inspired analysis. In Findings of the Association for Compu- tational Linguistics: ACL 2022, pages 1720-1732.
Zhendong Dong and Qiang Dong. Hownet and the computation of meaning: (with cd-rom).
P. Ferragina and U. Scaiella. 2010a. Tagme: on-the- fly annotation short text fragments (by wikipedia entities). In Proceedings of the 19th ACM Confer- ence Information and Knowledge Management, CIKM 2010, Toronto, Ontario, Canada, October 26- 30, 2010.
Junling Liu, Peilin Zhou, Yining Hua, Dading Chong, Zhongyu Tian, Andrew Liu, Helin Wang. Chenyu You, Zhenhua Guo, Lei Zhu, and Michael Lingzhi Li. 2023. Benchmarking large language models on emexam comprehensive chinese medical exam dataset.
Paolo Ferragina and U Scaiella. 2010b. Tagme: on-the- fly annotation short text fragments (by wikipedia entities). In ACM Conference on Information and Knowledge Management (CIKM), pages 1625-1628. ACM Press.
Weijie Liu, Peng Zhou, Zhe Zhao, Zhiruo Wang. Qi Ju, Haotang Deng, and Ping Wang. 2020. K-bert: En- abling language representation with knowledge graph. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 2901-2908.
Bin He, Di Zhou, Jinghui Xiao, Xin Jiang, Qun Liu, Nicholas Jing Yuan, and Tong Xu. 2020. Bert-mk: Integrating graph contextualized knowledge into pre- trained language models. In Findings of the Associ- ation for Computational Linguistics: EMNLP 2020, pages 2291.2290.
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Man- dar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining ap- proach. arXiv preprint arXiv: 1907.1 11622.
Pedram Hosseini, David A Broniatowski, and Mona T Diab. 2022. Knowledge-augnen augmented language models for cause-effect relation classification. In ACL. 2022 Workshop on Commonsense Representation and Rea- soning.
Yinquan Lu, Haonan Lu, Guirong Fu, and Qun Liu. 2022. Kelm:
Knowledge enhanced pre-trainet lan- guage representations with message passing on hier- archical relational graphs. In ICIR 2022 Workshop on Deep Learning on Graphs for Natural Language Processing.
Yifan Hou, Guoji Fu, and Mrinmaya Sachan. 2022. What has been enhanced in my knowledge-enhaneed language model? arXiv preprint arXiv:2202.0096.
John P McCrae, Alexandre Rademaker, Francis Bond, Ewa Rudnicka, and Christiane Fellbaum. 2019. En- glish wordnet 2019-an open-source wordnet for glish. Wrectaw (Poland), page 245.
ZIWEI JI. NAYEON LEE, RITA FRIESKE, TIEZHENG YU, DAN SU, YAN XU, and ETSUKO ISHII. 2022. Survey of hallucination in natural language generation. ACM Comput. Surv, 1(1)
Zaiqiao Meng, Fangyu Liu, Thomas Clark, Ehsan Shareghi. and Nigel Collier. 2021. Mixture-of- partitions: Infusing large biomedical knowledge graphs into bert. Proceedings of the 2021 Con- ference on Empirical Methods in Natural Language Processing, pages
Zhengbao Jiang, Frank F Xu, Jun Araki, and Graham know what language
Neubig. 2020. How models know? Transactions of the Association for Computational Linguistics, 8:423-438.
Matthew E. Peters, Mark Neumann, Robert L Lo- gan, Roy Schwartz, Vidur Joshi, Sameer Singh, and Noah A. Smith. 2019. Knowledge enhanced contex- tual word representations. In EMNLP.
Pei Kc, Haozhe Ji. Siyang Liu, Xiaoyan Zhu, and Minlic Huang. 2020. Sentilare: Sentiment aware language representation learning with linguistic knowledge. In Proceedings of the 2020 Conference Empirical Methods in Natural Language Processing (EMNLP), pages 6975-6988.
Mohammad Taher Pilchvar and Jose Camacho-Collados. 2019. Wie: the word-in-context dataset for evaluat- ing context-sensitive meaning representations. In

---

Proceedings of the 2019 Conference of the North American Chapter of the Association for Computa- tional Linguistics: Human Longuage Technologies, Volume (Lngg and Short Papers), pages 1267-1273.
Ikuya Yamada, Akari Asai, Hiroyuki Shindo, Hideaki Takeda, and Yuji Matsumoto. 2020. LUKE: Deep contextualized entity representations with entity- aware self-attention. In Proceedings of the 2020 Con- ference on Empirical Methods in Natural Language Processing (EMNLP). Association for Computational Linguisties.
Pranav Rajpurkar, Jiam Zhang, Konstantin Lopyrev, and Percy Liang. 2016. Squad: 100,000+ questions for machine comprehension of text. In Proceedings of the 2016 Conference on Empirical Methods in Natu- val Language Processing, pages 2383-233 23922
Deming Ye, Yankai Lin, Peng Li, Maosong Sun, and Zhiyuan Liu, 2022. A simple but effective pluggable entity lookup table for pre-trained language models. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 523-529
Larry Smith, Lorraine K Tanabe, Cheng-Ju Kuo, I Chung, Chun-Nan Hsu, Yu-Shi Lin, Roman Klinger, Christoph M Friedrich, Kuzman Ganchev, Manabu Torii, et al. 2008, Overview of biocreative ii gene mention recognition. Genome biology, 9(2):1-19.
Zheng Yuan, Yijia Liu, Chuanqi Tan, Songfang Huang, and Fei Huang. 2021. Improving biomedical pre- trained language models with knowledge. Pro- ceedings of the 20th Workshop on Biomedical Lan- guage Processing, pages 190, Online. Associa- tion for Computational Linguistics.
George Stoica, Emmanouil Antonios Platanios, and Barnahás Póczos. 2021. Re-tacred: Addressing short- comings of the tacred dataset. In Proceedings of the AAAI Conference on Artificial Intelligence, vol- ume 35, pages 13843-13850.
Ningyu Zhang, Shumin Deng, Xu Cheng, Xi Chen, Yichi Zhang, Wei Zhang, Huajun Chen, and Hangzhou Innovation Center. 2021. Drop redundant, shrink irrelevant: Selective knowledge injection for language pretraining. In IJCAI, pages 4007-4014
Yusheng Su, Xu Han, Zhengyan Zhang, Yankai Lin, Peng Li, Zhiyuan Liu, Jie Zhou, and Maosong Sun. 2021. Cokebert: Contextual knowledge selection and embedding towards enhanced pre- me-ttrined language models. Al Open. 2:127-134.
Yuhao Zhang. Victor Zhong, Danqi Chen, Gabor Angeli, and Christopher D. Manning. 2017. Position-aware attention and supervised data improve slot filling. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing (EMNLP 2017), pages 35
Tianxiang Sun, Yunfan Shao, Xipeng Qiu, Qipeng Guo. Yaru Hu, Xuan-Jing Huang, and Zheng Zhang, 2020. Colake: Contextualized language and knowl- edge embedding. In Proceedings of the 28th Inter- national Conference on Computational Linguistics, pages 3660-3670.
Zhengyan Zhang, Xu Han, Zhiyuan Liu, Xin Jiang, Maosong Sun, and Qun Liu. 2019. ERNIE: En- hanced language representation with informative en- tities. In Proceedings of ACL2019.
Ruize Wang, Duyu Tang. Nan Duan, Zhongyu Wei, Xuan-Jing Huang. Jianshu Ji, Guihong Cao, Daxin Jiang, and Ming Zhou. 2021a. K-adapter: Infusing knowledge into tre-trained models with adapters. In Findings of the Association for Computational Lin- guistics: ACL-IJCNLP 2021, pages 1405-1418.
A Experiment Details
A.1 Baseline Details
LUKE (Yamada et al., 2020) chooses to en- hance RoBERTa with knowledge from Wikipedia. It uses new pre-training task that involves pre- dicting randomly masked words and entities in large entity- annotaed corpus retrieved from Wikipedia. At the same time, LUKE also inputs wikipedia entities into the model which are based on the sentences in finetuning for the question- answering dataset SQUADI.1. In addition to inject- ing knowledge, LUKE proposes an entity-aware self-attention mechanism and considers the types of tokens (words or entities) when computing at- tention scores (Yamada et al., 2020).
Xiaozhi Wang, Tianyu Gao, Zhaocheng Zhu, Zhengyan Zhang, Zhiyuan Liu, Juanzi Li, and Jian Tang. 2021b. Kepler: unified model for knowledge embedding and pre-trained language representation. Transac- tions of the Association for Computational Linguis- tics, 9:94
Chaoyi Wu, Xiaoman Zhang, Ya Zhang, Yanfeng Wang. and Weidi Xie. 2023. Pmc-Ilama: Further finetuning llama medical papers.
Bo Xu, Yong Xu, Jiaqing Liang, Chenhao Xie, Bin Liang, Wanyun Cui, and Yanghua Xiao. 2017. Cn- dbpedia: A never-ending chinese knowledge extrac- tion system. In Advances in Artificial Intelligence: From Theory to Practice: 30th International Confer- ence on Industrial Engineering and Other Applica- tions of Applied Intelligent Systems, IEA/AIE 2017, Arras, France, June 27-30, 2017, Proceedings, Part I1, pages 428-438. Springer.
ERNIE (Zhang et al., 2019) injects entity knowl- edge from Wikipedia into BERT in pre-training and finetuning. ERNIE first uses TAGME (Ferragina and Scaiella, 2010b) to link entities mentioned in

---

context to their corresponding entities in KG, then injects the corresponding entities embedding into language models. Embeddings of the correspond- ing entities are trained on triples from WikiData via TransE (Zhang et al., 2019).
BC5-chem & BC5-disease (Li et al., 2016) con- tain 1500 PubMed abstracts that extract chemical and disease entities respectively.
NCBI-disease (Doğan et al., 2014) includes 793 PubMed abstracts that had been detected disease entities.
KnowBERT (Peters et al., 2019) integrates knowledge from WordNet and Wikipedia into BERT and demonstrates improved perplexity and ability to recall facts. KnowBERT first trains an integrated entity linker to retrieve relevant entity embeddings, which is used for entity disambigua- tion. Then, the model uses Knowledge Attention and Recontextualization (KAR) mechanism to com- bine the knowledge representation and contextual word representations.
BC2GM (Smith et al., 2008) is dataset includ- ing 20K PubMed sentences extracting gene entities. JNLPBA (Collier and Kim, 2004) is a dataset including 2,000 PubMed abstracts that have been identified as molecular biology-relates entities.
Entity Typing is the task to find the correct type of the corresponding label entities in giving sen- tence.
Open Entity (Choi et al., 2018), commonly used in Knowledge-enhanced LLMs, has about 6000 sen- tences with six entity types. Each sentence has five entity labels on average.
K-BERT (Liu et al., 2020) choose CN-DBpedia, HowNet, and MedicalKG as external knowledge bases. K-BERT is devised to feed structural tree that is decoded from the sentence into pre-trained language model. The construction of the structural tree is driven by both the sentence itself together with an external knowledge graph. However, inevitably brings the problem of knowledge noise. To solve this problem, K-BERT proposed to special a seeing layer, which makes the injected triples can only affect their corresponding subject.
Relation Classification is the task of identify- ing the relation between label entities in given sentence.
TACRED (Zhang al., 2017), is relation ex- traction dataset with 106,264 examples. Examples in TACRED cover 42 relation types.
Question Answering is the task of answering questions such as reading comprehension ques- tions.
KeBioLM (Yuan et al., 2021) injects entity
knowledge from UMLS (Bodenreider, 2004) by fusing the entities in the knowledge base and men- tions in the text in the middle layer. Firstly, it uses function to recognize if a span is an entity mentioned. then, it links to set of the mention's k-nearest entities and integrates the entity embed- ding and the word embedding in the hidden layer, as the input of the model.
SQuADI.1 (Rajpurkar et al., 2016), is read- ing comprehension dataset, consisting of questions from Wikipedia articles. SQuAD 1.1 contains 107,785 question-answer pairs 536 articles.
Word Sense Disambiguation is the task to let the model find label words' most suitable entry in the sense inventory.
WiC (Pilehvar and Camacho-Collados 2019), is benchmark that is used for evaluating context- sensitive word embeddings. Each instance in WiC has target word, and the task is to identify if the occurrences of the target word in the two contexts correspond the same meaning not.
A.2 Downstream tasks and Dataset Details
Named Entity Recognition (NER) is the task of finding the corresponding span of the named entity in the given sentence.
Finance NER includes 3000 financial news ar- ticles manually labeled, which contain over 65,000 name entities.
Commonsense Causal Reasoning is the task of finding corresponding options through the causal dependencies.
Medicine NER is the Clinical Named En- tity Decoonition(CNNER task that was released in CCKS 2017. The dataset mainly extracts medical-related entity names from electronic medi- cal records.
A.3 Knowledge graph Details
CN-DBpedia (Xu et al., 2017) is a large-scale open-domain encyclopedic Chinese knowledge graph developed by the Knowledge Work Lab of
hhnps://embeeddingggiiiiie http:///wwwiiieiiiooooo///o///ne hhtp:://hiiaaaa...mmmmmpmoooooor

---

FewRel
Setup
R
BERT (w/o KT)
85.05
85.11
84.89
ERNIE
88.49
88.44
88.32
F.RNIF-Aligned
87.98+0.32
87.97+0.32
87.87+0.33
ERNIE-Random
85.75::0.26
85.73+:0.25
85.62::0.26
Table 8: Results of ERNIE on FewRel. ERNIE-Aligned and FRRNIE-Randend correspond setup 1, respectively and all these experiments run 5 times with varying random seeds. The results of BERT and ERRIIE come from Zhang et al. (2019).
Fudan University, covering tens of millions of enti- ties and hundreds of millions of relationships. The CN-DBpedia used in the paper includes 5.17 mil- lion triples.
122%
100%
monton
HowNet (Dong and Dong) is large language knowledge base of Chinese vocabulary and con- cepts, including semantic annotations of Chinese words. The HowNetused in the paper includes 52576 million triples.
60%
45%
MedicalKG is the Chinese medical concept knowledge graph, which contains four types of pseudonyms (symptoms, diseases, parts, and treat- ments). MedicalKG contains a total of 13864 triples and is an open-source part of K-BERT.
Figure Schematic diagram of embedding similarity change. In this experiment, inject aligned and ran- dom entities into ERNIE on the Open Entity test dataset. We count the cosine similarity of [cls], mentions and entities embedding in the hidden layers. The similarity figure the absolute average of 1000 samples.
UMLS (Bodenreider, 2004) is a compendium of many controlled vocabularies in the biomedi- cal sciences. It provides mapping structure be- tween these vocabularies, containing over million biomedical concepts and million concept names.
prior work mostly injects knowledge into hidden layers of the encoder the representation space. In that regard, we propose to compare the hidden states' similarities. To design this part of the proto- col, we prepare well-trained knowledge-enhanced model, then respectively load the aligned and ran- dom knowledge data feeding through it. In what fol- lows, we calculate the cosine similarity between the two chosen hidden states, specifically. the values of the chosen token's 's position, from a certain layer, yielded by feeding aligned v.s. random knowledge injection. We mainly focus on the similarities [cls] and mentions' (m defined by Section 1) em- beddings, because these embeddings are primarily served as the input gate to our downstream tasks. We choose ERNIE and 2 datasets (Open Entity and TACRED) for this of experiments.
Wiki graph The knowledge base is Wiki- Data5M (Wang al., 2021b), which consists of 3085345 entities and 822 relation types.
A.4 The Results of ERNIE FewRel
Different from ERNIE's performance on Open En tity and TACRED, the result of ERNIE's s ablation experiment on FewRel is about 2.2. However, this result is risky. Because the mention of FewRel is consistent with the task label entity, and the task la- bel is also consistent with the relationship informa- tion in Wikidata, the injected knowledge may con- tain label information (just like the training logic of TransE, the difference between the two entity em- beddings is equal to the relationship embedding).
Experiment Setup. We first load the aligned and random entities data on the trained knowledge- enhanced models and print their output at each hidden layer of the encoder. Then, we compare the similarity among these outputs. We adopt co-
B Further Explanation
To dive further into these counter-intuitive results, we propose to track down the path the injected knowledge. As we mentioned Figure 2, the

---

Entity and TACRED using ERNIE, which loads the test data while injecting aligned and random entities. Accordingly,
Supput dffeeen surpue
12000
10000
9630
fairly straightforward to find that it is probably over for the model to have identical predictive results when injected with different forms of knowledge. On TACRED, this portion even exceeds 99.6% %. Simply put, those findings may microscopically explain the reason for the inconspieuous results in previous ablation experiments, that the knowledge-injectee models have failed to leverage the injected knowledge, nor to recognize the relevance of the knowledge and the text input.
0000
34
TACRED
Figure 5: The output stats of injecting aligned and ran- dom knowledge. With two inference passes performed, count the number of samples with the same and dif- ferent outputs, respectively. Notably, the "same output" (orange bins) indicates that altering knowledge form under our protocol does not change the model's ' predic- tion, while "different output" (blue bins) indicates the opposite.
C Performance Gain Explained By Data Augmentation
Indeed, given all the aforementioned empirical re- sults, it is still undeniable that knowledge-injected frameworks have a positive outcome from the per- spective of eventual performance. To answer re- search question 1, we hereby cast a hypothesis, perhaps wild, that the injected knowledge is picked by the model as a data augmentation module.
sine similarity as a measure of variation in word embeddings and entity embeddings.
Among these similarities, we only keep similari- ties of [cls]. word embeddings related to knowl- edge, and knowledge embeddings. After this, output the predictions of the same sentence with different knowledge injected. The result validates the previous inference fine-grained dimensions.
Experiment setting. To entertain this possibility, we conduct the following two gauges: we meter the degree of overfitting during training that is proxi- mally calculated by the loss gap between the train- ing and validation set. The rest of experimental setup kept identical to setup 2.
Analysis. Our analytical experiments find that word embeddings injected with different knowl- edge are highly similar. Figure 4 shows the simi- larity change of word and entity embeddings from layer to layer 12. From layer layer 5, there is no interaction between the entity and word embed- ding, so the similaritics did not change. Starting from layer 6, the entity and word embeddings be- gin to fuse, and the corresponding similarity begins to change, but the [cls] embedding changes are always small. After layer 12, [cls] embedding inputs into the linear layer and outputs logits.
Discussion. In what follows, on the experiments on Open Entity dataset with baseline ERNIE, we curate and report both the loss gap between the train and dev set. From an ordinary machine learn- ing perspective, the larger this gap being revealed, the more overfitted the model gets. The result is summarized in the text as follows: (i)-inecctnng aligned, unaligned (randomized) white noise all manage to decrease the loss gap to control overfit- ting. (ii)-through manipulating the magnitude of the knowledge vector (from 0.173 to 0.170), we can see this gap becomes smaller (but perhaps hurt the overall performance). From an empirical point of view, we may also postulate that these patterns all conform to the data augmentation, such as the regularization effect, the larger scale of augmen- tation the stronger regularization, etc. At last, an empirical study, do not intend to make de- terministic conclusion. The hypothesis we cast that the previous knowledge injections may act as a imperfect data augmentation module - is based
can be found that the similarities of [cls] embeddings are very high in the whole process, generally above 98%. In this case, difficult for the model to find the difference between the three sets of inputs. This result shows that the model hardly obtains valuable information from the knowledge representation.
Since cosine similarity may ignore differences in some dimensions, classifiers may be able to differentiate those differences by eliminating the dominant dimension. So based on the previous experiments, we output the prediction results of the model. Figure 5 shows the results of Open

---

their similar performance pattern and perhaps is only one among many other possibilities. We hope to use this finding to motivate the community to provide more theoretical and comprehensive evi- dence.
D Details of Conceptual Knowledge Injection
D.1 Details of ChatGPT Experiment
In this experiment, we choose 50 data from TA- CRED, and add the words "Question: Is there relationship between A and B? If is, what is the relationship between them?" after each text. At last, Group and correctly answer 44 questions, and Group gets 46 correct answers.
E
Dataset License
We only find three dataset licenses, which are as follows:
SQuAD: CC-BY-SA 4.0
WiC: CC BY-NC 4.0
СОРА: BSD 2-Clause


Reviews:
[{"review_id": "lFc9KgnsPz", "reviewer": "Reviewer_Lzwd", "summary": "", "questions": "1) How many epochs did you train for?\n2) How did you carry out your hyperparameter optimization?\n3) Does the number of dimensions of the embddings impact the performance increase/decrease?\n4) Do you results suggest that the current addition of \"noise\" is basically a regularizer that improves the generalization abilities of the models? ", "limitations": "", "rating": 4, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Q1: How many epochs did you train for?**\n\nA: We are sorry for missing this in our paper.\n\nThe train epochs are shown as follows:\n|       dataset       | train epochs |\n| :-----------------: | :----------: |\n| Open Entity, FewRel |      10      |\n|        SQuAD        |      2       |\n|       Others        |      5       |\n\nPresumably, we follow the prior related work’s setting for making the empirical verification fair.\n\n**Q2: How did you carry out your hyperparameter optimization?**\n\nA: Well, the answer is *no*.\n\nWe basically follow the prior work’s setting (including the hyperparatmers, architectures, etc.) . \nWe didn’t do any extra tuning work on any of the dataset we presented in the experiment section.\n\n**Q3: Does the number of dimensions of the embddings impact the performance increase/decrease?**\n\nA: This is an interesting question.  \n\nWe actually didn’t know about the answer to this.\nThis is because --- as we humbly wrote previously --- that we only adopt the same hyperparamter/architectural-configiguration as the compared method. \nSo the dimensions of the embeddings are identical to whichever approach we intended to compared with. However, we intend to supplement some more expeirments in our revision due to the interesting insight this question might bring about.\n\n**Q4: Do you results suggest that the current addition of \"noise\" is basically a regularizer that improves the generalization abilities of the models?**\n\nA: Yes, we agree with the this might be the cause to explain our findings of this paper. \nIn the Appendix C, we put some more analysis intending to interpret this.\n\nNoted, this suspicious high coorelation of the training dynamics --- between prior knowledge injection approach and data regularization --- is also recently supported by [1].\nDespite of our preliminary conjecture and analysis, we believe there are further room to dig especially from a theoretical standpoint. However, that may have gone beyond the scope of this paper. We hope to leave it to future work.\n\n*References*:   \n[1] Arora et al. Metadata Shaping: A Simple Approach for Knowledge-Enhanced Language Models. 2022", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- Very relevant to the community\n- Interesting results \n- Simple remedy", "reasons_to_reject": "- Small number of reference language models\n- Results might be due to other factors, e.g., the learning parameters", "questions_to_authors": "1) How many epochs did you train for?\n2) How did you carry out your hyperparameter optimization?\n3) Does the number of dimensions of the embddings impact the performance increase/decrease?\n4) Do you results suggest that the current addition of \"noise\" is basically a regularizer that improves the generalization abilities of the models? ", "paper_topic_and_main_contributions": "The paper suggests that current knowledge graph injection approaches are indistinguishable from white-noise injection approaches and propose an \"embarassingly\" simple solution: refining the knowledge to inject before injecting it. The paper suggests that less is more in this context, meaning that the injection of small amounts of knowledge leads to a treatment different from white-noise-like treatment and a generally better performance of the algorithms. ", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "AN1FVX0bWz", "reviewer": "Reviewer_v97Y", "summary": "", "questions": "1. This paper seems to center on BERT-based models. I'm curious about how the findings apply to decoder-only models, such as GPT-2 or Llama?", "limitations": "", "rating": 3, "confidence": 3, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Q1: The paper is hard to follow, particularly the introduction. The use of numerous terms, such as \"tuning dynamism\" makes comprehension difficult. Attention to detail is needed. For instance, Line 76 has a typo: \"successes(Ye et al.,)\" – there should be a space after the \"successes\". Such issues are prevalent throughout the paper.**\n\nA: We apologize for all of those oversights. \nWe promise we’ll fix them in our revision as soon as possible.\n\n**Q2: Referring to models like BERT and RoBERTa as \"large language models\" may be misleading. Typically, LLMs denote language models with over a billion parameters, implying that perhaps BERT and RoBERTa might not be apt descriptors any longer.**\n\nA: We thank the reviewer for pointing this out. \nIn recent days particular, words like Pre-trained Language Models (PLMs), Masked Language Models (MLM) and Large Language Models (LLMs) have been the buzzwords everywhere. \nWhile we’d like to side with the reviewer on this matter, we did actually scrutinize the past published top-tier papers for these word-usages. \nHowever, we realized there didn’t seem to have a consensus on naming these models. \nWe again are sorry for this confusion, and we will add a note in our revisions.\n\n**Q3: This paper seems to center on BERT-based models. I'm curious about how the findings apply to decoder-only models, such as GPT-2 or Llama?**\n\nA: Yes, this is a great question. \n\nOur response is three-fold:\n(i)-Indeed, we did conducted experiments on ChatGPT using the GPT-3.5-turbo API.\nYou may have missed it because we didn’t make a table (sorry!). \nPlease checkout Line 494 – 512. The conclusion taken away from this experimetn conforms to the others.  \n(ii)-As the reviewer said, we primarily picked BERT, or ROBERTA, as our major backbones. \nThis is because most --- if not all --- of the knowledge injection papers [e.g. 1,2,3] live upon them. \nWe basically follow the same protocol and setup --- as a commonground --- to make the comparison fair and grouded.  \n(iii)-Yes, to have the similar experiemnts launched on GPT-2 or LlaMA would be nice. \nHowever, here we’d like to ask for grace from the reviewer. As an academical group with limited computing resources, launching these models would be (much) costly. \nThere also hasn’t been any highlighted published work studing knowledge injection into these models. \nThus, on this matter we lacked any contexturelized information including hyperparameter, architecutres and etc. \nTo make a full-cycled experiments (explorations of KI on these new backbones, self-run baselines, ablations, our remeditated methods) on these models were unfortunately beyond our resource limit.\n\n*References*:  \n[1]Yamada et al. LUKE: Deep Contextualized Entity Representations with Entity-aware Self-attention. 2020  \n[2]Zhengyan Zhang et al. ERNIE: Enhanced Language Representation with Informative Entities. 2019  \n[3]Matthew E et al. Knowledge Enhanced Contextual Word Representations. 2019", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The paper addresses an intriguing topic: the exploration of knowledge injection in language models.\n2. The experimental design is robust, providing substantial support for the paper's conclusions.", "reasons_to_reject": "1. The paper is hard to follow, particularly the introduction. The use of numerous terms, such as \"tuning dynamism\" makes comprehension difficult.\n2. Referring to models like BERT and RoBERTa as \"large language models\" may be misleading. Typically, LLMs denote language models with over a billion parameters, implying that perhaps BERT and RoBERTa might not be apt descriptors any longer.\n3. Attention to detail is needed. For instance, Line 76 has a typo: \"successes(Ye et al.,)\" – there should be a space after the \"successes\". Such issues are prevalent throughout the paper.", "questions_to_authors": "1. This paper seems to center on BERT-based models. I'm curious about how the findings apply to decoder-only models, such as GPT-2 or Llama?", "paper_topic_and_main_contributions": "In this work, the authors delve deeply into the adaptation of large language models (LLMs) like GPTs with external knowledge. A thorough empirical study reveals that, under rigorous testing, most contemporary knowledge injection frameworks perform erroneously. This issue is likened to the effects of noise injection. Surprisingly, it's observed that unaligned knowledge can sometimes outperform its aligned counterpart. The paper suggests a straightforward remedy, which emphasizes purifying external knowledge before integrating it.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "588Qt7B7Vl", "reviewer": "Reviewer_jdMq", "summary": "", "questions": "1. What is the specific implementation of random injection and noise injection?\n\n2. Are there statistics on the number of injected knowledge tuples, including the original and the remediated methods? \n\n3. Can the phenomenon that injecting unaligned knowledge tuples achieves comparable results with the aligned knowledge be attributed to the fact that the aligned knowledge contains too much useless or even noise information?", "limitations": "", "rating": 3, "confidence": 3, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Q1:The contributions of the authors may be limited. They conduct experiments empirically on current knowledge injection frameworks. However, there is not enough research on the underlying reasons behind their findings.**\n\nA: We are sorry that the reviewer may feel this way. \nWe’d like to humbly defend our paper.\n\nFirst, we may argue, devling further into this question would likely to require an establishment of the theoretical scheme of knowledge injection, which is by far unseen from any of preprinted nor published papers. \n\nIn current spectrum of the research around knowledge injection, we believe our paper has by far provided one of the most in-depth analysis and dense empirical veirifcation in the community. \nCompared to most of the validation protocol from prior published work, we addtionally offer (i)-a new ablation verification scheme; \n(ii)-a remeditated approach with a new and purifed external KG data/benchmark and \n(iii)-a (potentially) plausible interpretation of drawing similarity to data augmetation, in Appendix C (we are sorry we leave this section to the appendix, we kindly refer the reviewer to it for further proof-read).\n\nGiven this context, together with the scope of the paper, we believe it is more suitable for future study.\n\n\n\n**Q2: What is the specific implementation of random injection and noise injection?**\n\nA: Sorry for the confusion. \nFigure 1 in the paper outlines the knowledge injection process that it consists of three steps: text knowledge alignment, knowledge retrieval, and knowledge injection. \n\nDuring the text knowledge alignment phase, random injection involves substituting the aligned knowledge with random information (usually process during data collation or preprocessing prior to data loading). \n\nIn the knowledge retrieval phase, noise injection entails replacing the acquired knowledge embeddings with random white noises (typically perform after input tokenization and before feeding it into the model).\n\n**Q3: Are there statistics on the number of injected knowledge tuples, including the original and the remediated methods?**\n\nA: Appendix A.3 provides the statistical information for several injected knowledge graphs. \nThe information as fllowing: \n\n|        KG        |  Entities   | Relations |    Triples    |\n| :--------------: | :---------: | :-------: | :-----------: |\n|    CN-DBpedia    |      -      |     -     | 5.17 million  |\n|      HowNet      |      -      |     -     | 52576 million |\n|       UMLS       |  6 million  |     -     |       -       |\n|    Wiki graph    |  3 millon   |    822    |       -       |\n| Wikidata (ERNIE) | 500 million |     -     |       -       |\n| Wikidata (LUKE)  | 50 million  |     -     |       -       |\n| Wordnet (remedy) |   117,000   |     -     |       -       |\n\n**Q4: Can the phenomenon that injecting unaligned knowledge tuples achieves comparable results with the aligned knowledge be attributed to the fact that the aligned knowledge contains too much useless or even noise information?**\n\nA: Yes, as we posited in the previous response to the Q1, we believe this conjecture very much aligns with ours. \nAgain, we may kindly refer the reviewer to the Appendix C.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The findings of the authors are meaningful. They find that injecting unaligned (i.e., random) knowledge tuples into the LLMs achieves comparable (and sometimes better) results than the aligned knowledge being injected, which is previously undiscovered.\n\n2. Their findings can encourage the community to further check the knowledge injection methods more from the side of the knowledge itself.\n\n3. The experiments are adequate and detailed, and the experimental results can support the authors’ findings and inferences", "reasons_to_reject": "1. The contributions of the authors may be limited. They conduct experiments empirically on current knowledge injection frameworks. However, there is not enough research on the underlying reasons behind their findings.\n\n2. Some implementation details are unclear, like the difference in implementing knowledge injection, random injection, and noise injection.", "questions_to_authors": "1. What is the specific implementation of random injection and noise injection?\n\n2. Are there statistics on the number of injected knowledge tuples, including the original and the remediated methods? \n\n3. Can the phenomenon that injecting unaligned knowledge tuples achieves comparable results with the aligned knowledge be attributed to the fact that the aligned knowledge contains too much useless or even noise information?", "paper_topic_and_main_contributions": "In this paper, the authors present a comprehensive empirical study of current knowledge injection frameworks. They find that most, if not all, prior knowledge injection methods perform erroneously. Furthermore, the authors provide an interpretation from the similarity of noise injection. Finally, they offer a simple remediation method to overcome the aforementioned sanity problem and further push the boundary of the performance of the domain-adaptive LLMs.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "duVw78B659", "reviewer": "Reviewer_ic1H", "summary": "", "questions": "Does the analysis in the appendix try to say that knowledge injection is hard to work when the injection is done in the hidden layer?\nAnd if so, what's the most effective knowledge injection method? (In text?)\nAnd if not, how to make knowledge injection in the hidden layer work?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Q1: There is no clear illustration of the injection method in section 4 (Text-based or Embedding-based?)**\n\nA: We apologize for this oversight. \n\nThe experiments provided in Section 4 comprise both Text-based methods, specifically K-BRERT, and Embedding-based methods such as KnowBert, ERNIE, Luke, and KeBioLM, closely following prior related work. Some more information can be found in Appendix A.1. \nWe ensure this piece of information will be incorporated in our next revision.\n\n**Q2: The remedy method is verified in a text-based injection setting, would it work in embedding-based injection?**\n\nA: Yes, this is actually a great question, thank you!\n\nWe primarily validate our remeditated approach in the text-based injection schemes for its simplicity. \nWe supplement a set of expeirments for testing the compatibility of our method with embedding-based injection, as follows.\n\n\n|    FewRel    |     F1     |\n|:------------:|:----------:|\n|     BERT     |    84.89   |\n|  Ours-Random | 85.19±0.24 |\n| Ours-Aligned | 86.03±0.18 |\n\nOn FewRel, our remeditated approach makes knowledge injection work properly, by +1.14 over the baseline and +0.84 over the random ablation setup. \nThis exhibits similar pattern to the text-based injection shown in Table 7.\n\nWe further pose another note tagged with this set of results. \nAs we wrote, perhaps the external KG the community has been relying on (all these years) can be the root cause for the problematic performance displayed in the paper. \nOur remeditated approach intends to re-establish it by purifying the KG. \nAs per the prior embedding-based injection on this line, this requires us to perform a new round of TransE embedding learning through pre-training. \n\nHere, we believe (i)-there is still much room for improvement and future study here, because the conventional TransE learning may not be optimal for the purified KG; \n(ii)-this new round of training contributes to the final performance comparison as an additional factor, and this is why we left out this verification in the submitted version.\n\nFor the time limit, we will complete the same experiments on other datasets in our next revision.\n\n\n**Q3: Does the analysis in the appendix try to say that knowledge injection is hard to work when the injection is done in the hidden layer? And if so, what's the most effective knowledge injection method? (In text?) And if not, how to make knowledge injection in the hidden layer work?**\n\nA: Another great question, thank you. \n\nTo the best of our knowledge, there has been few (or none) prior work discussing this issue, about injecting knowledge into the hidden layers. \nThe methods in-so-far are mainly categorized into text-based or embedding-based (at the input level). \nOn the line of embedding-based injection, we believe that getting the knowledge into the hidden layers properly is far more challenging due to the uncertainty of the positional encoding, higher abstraction-level in the representation, and etc.\n\nFrom this work, it seems that even injecting knowledge from the input-level embedding is by far problematic, not to mention the more challenging setup (into the hidden layers). \nThat said, we believe the investigation of this question is more suitable for a separate future study.\n\nOnce again, by doing this work of ours, rather than proposing a new neural network for knowledge injection, we hope to direct the related community to the injected knowledge itself, e.g. a random, unaligned, aligned, or puried knowledge tuple, etc. \nThe results from our work encourages a refurbish on current KB/KG adopted in these common benchmarks, and we further provided a remedy to it.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The investigated problem is important and sheds light on the task of knowledge injection.\n2. The writing of this paper is good, making it easy to follow.\n3. The experiments are extensive and the conclusion is impressive.", "reasons_to_reject": "1. There is no clear illustration of the injection method in section 4 (Text-based or Embedding-based?)\n2. The remedy method is verified in a text-based injection setting, would it work in embedding-based injection?", "questions_to_authors": "Does the analysis in the appendix try to say that knowledge injection is hard to work when the injection is done in the hidden layer?\nAnd if so, what's the most effective knowledge injection method? (In text?)\nAnd if not, how to make knowledge injection in the hidden layer work?", "paper_topic_and_main_contributions": "This paper investigates whether the previous knowledge injection methods are useful. Through extensive experiments, the author finds that the knowledge injected performs even worse than random knowledge or noise. To deal with the problem, the authors propose a simple remedy method and proven to be helpful.", "excitement": 4, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper studies how knowledge is fused in models, performing an empirical study and comparison of the existing approaches. Furthermore, experiments with randomly injected knowledge demonstrate that these models aren't behaving well as expected and that the misaligned information may result in improvements. Highlighting limitations in this approach for modelling that the community can rally around and address.

The reviews for the paper are generally strong and consistent. Weaknesses identified by the reviewers identify that the paper only contains empirical findings rather than analytical findings and regarding missing information. The missing information is adequately added in the rebuttal and response from the authors and I find the empirical contributions to be sufficient.
Decision: Accept-Main