KAPALM: Knowledge grAPh enhAnced Language Models for Fake News Detection
Authors: Jing Ma
Chen Chen
Chunyan Hou
Xiaojie Yuan
Venue: EMNLP 2023 Findings (2023)
Keywords: Knowledge Graph; Fake news detection
TL;DR: KAPALM: Knowledge grAPh enhAnced Language Models for Fake News Detection

Abstract: Social media has not only facilitated news consumption, but also led to the wide spread of fake news. Because news articles in social media is usually condensed and full of knowledge entities, existing methods of fake news detection use external entity knowledge. However, majority of these methods focus on news entity information and ignore the structured knowledge among news entities. To address this issue, in this work, we propose a Knowledge grAPh enhAnced Language Model (KAPALM) which is a novel model that fuses coarse- and fine-grained representations of entity knowledge from Knowledge Graphs (KGs). Firstly, we identify entities in news content and link them to entities in KGs. Then, a subgraph of KGs is extracted to provide structured knowledge of entities in KGs and fed into a graph neural network to obtain the coarse-grained knowledge representation. This subgraph is pruned to provide fine-grained knowledge and fed into the attentive graph and graph pooling layer. Finally, we integrate the coarse- and fine-grained entity knowledge representations with the textual representation for fake news detection. The experimental results on two benchmark datasets show that our method is superior to state-of-the-art baselines. In addition, it is competitive in the few-shot scenario.

KAPALM: Knowledge grAPh enhAnced Language Model for Fake News
Detection
Jing Ma 12 Chen Chen Chunyan Hou Xiaojie Yuan 1.2
College of Computer Science, Nankai University, Tianjin, China
2 MoE Key Lab of DISSec, Nankai University, Tianjin, China
3 School of CSE, Tianjin University of Technology, Tianjin, China majing@mail. nankai nankaiiiii..n edu. cn, {nkchenchen, yuanxj Jennniiinnaaii... edu. cn, houchunyanetjut..c edu. cn
Abstract
Tena of Thoueande Or Seicntiate Declare
University Cambridge
biologist
Social media has not only facilitated news con- sumption, but also led the wide spread of fake news. Because news articles in social me- dia are usually condensed and full of knowl- edge entities, existing methods of fake news detection use external entity knowledge to im- prove the effectiveness. However, the majority of these methods focus on news entity informa- tion and ignore the structured relation knowl- edge among news entities. To address this issue, in this work, we propose a Knowledge grAPh enhAn Language Model (KAPALM) which is novel model that fuses coarse- and fine- grained representations of entity knowledge from Knowledge Graphs (KGs). Firstly, we identify entities in news content and link them to entities in KGs. Then, a subgraph of KGs is extracted to provide structured relation knowl- edge of entities in KGs and fed into graph neural network to obtain the coarse- grained knowledge representation. This subgraph is pruned to provide fine-graid fine-grained knowledge and fed into the attentive graph pooling layer. Fi- nally, we integrate the coarse- and fine-grained grained entity knowledge representations with the rep- resentation of news content for fake news de- tection. The experimental results on two bench- mark datasets show that our method is superior statc-of-the the- baselines in the full-scale set- ting. In addition, our model is competitive in the few-shot setting.
Climate Change A Hoax
Dannia Bray and Hana von Etorch foard livd a azigefizaal
human
number sclentists were
Hana van Storch
skcpdcal of the ability cxisting gloce tlmate models
climat- ologlet
metso- raleggat
accuratety prodict gooal tomporaturos.
Universlity Haanburg
Figure 1: A news article on PolitiFact and its entity graph.
Thus, the research on automatic fake news detec- tion is desirable.
Prior researches are based on traditional machine learning methods and hand-crafted features based on news content to combat fake news on social media (Castillo et al., 201 1:Kwon et al., 2013;Zu- biaga et al., 2017:Pr 2020:). To avoid man- ual features, deep neural networks models such Convolutional Neural Networks (CNN) and Recur- rent Neural Network (RNN), have been used learn the high-level feature representations auto- matically and achieve great performance in detect- ing fake news (Ma et al., 2016; Wang, 2017; Shu et al., 2019). For the past few years, Pre-trained language models (PLMs), such as BERT (Devlin et al.. 2018) and RoBERTa (Liu et al., 2019), have been the mainstream approaches which provide the contextualized representation of textual content in natural language processing field and also achieve the competitive performance for fake news detec- tion (Pelrine et al., 2021; Sheng et al., 2021). How- ever, these works fail to consider the knowledge entities in news articles. Although news content is condensed and full of knowledge entities by which people usually verify the veracity of news content, PLMs are not effective in capturing the knowledge- level relationship between entities. As shown in the left of Figure 1, is hard to detect the veracity of news article exactly without the entity knowledge about Dennis Bray and Hans von Storch.
1 Introduction
recent years, with the development of the In- ternet, social media such as Facebook and Twitter have become the main platforms for people to con- sume news due to their real-time and casy access. However, social media is double-edged sword and it enables people to be exposed to fake news. The wide spread of fake news can misguide public opin- ion, threaten people's health, and even cause dev- astating effects on society (Vosoughi al., 2018).
*Corresponding author.

---

Large Knowledge Graphs (KGs), such Free- base (Bollacker et al., 2008) and Wikidata (Vran- decic and Krötzsch, 2014), contain large number of structured triplets, which can serve as the entity knowledge base for detecting fake news. Exist- ing rescarches have demonstrated the significant role of KGs (Pan et al., 2018: Dun et al., 2021; Jiang et al., 2022). Specifically, Dun et al. (2021) proposed knowledge attention networks to mea- sure the importance of knowledge and incorporate both semantic-level and knowledge-level represen- tations of news content. Jiang et al. (2022) pro- posed a knowledge prompt learning method which incorporated KGs into the prompt representation to make it more expressive for verbal word prediction. Hu et al. (Hu et al., 2021) explored the structural entity embedding and compared contextual entity representations with the corresponding entity repre- sentations the knowledge graph. However, they focus on the textual representation of entities in KGs and have not explored the in-depth structured knowledge associated with the new article. As shown in Figure 1, we construct a subgraph of KGs for the news article which is named as entity graph throughout this paper, and this entity graph is help- ful to detect fake news with the knowledge that "Dennis Bray a biologist and Hans von Storch is a climatologist and meteorologist".
We propose a Knowledge grAPh en- hAnced Language Model (KAPALM) integrate the coarse- and fine-grained knowl- edge representations with the representation of news content for fake news detection.
We compare our model with the state-of-the- art methods on two benchmark datasets few-shot and full-scale settings. The exper- imental results demonstrate that our model outperforms the state-of-the-art methods in the full- scale setting. Moreover, model is competitive in the few-shot setting.
Related Work
2.1 Fake News Detection
In this section, we will briefly introduce the related work fake news detection. This work (Zhou and Zafarani, 2021) outlines early methods which design handcrafted features and utilize statistical machine learning methods to detect the authentic- ity of a given news. These work (Castillo et al.. 201 Pryybbya,, 2020;Kwon et al., 2013gaubaaga et al., 2017) proposed to utilize statistical infor- mation to leverage textual and social features to detect fake news. Also, some works (Ajao et al., 201 2019;Zhang et al., 2021) focus on capturing senti- ment features better detect fake news. Next, we will explore relevant research on the application of deep learning techniques in the task of detecting fake news. With the development of deep learning techniques, some models such as CNN, RNN, and Transformer(Kim) 2014:Liu et al., 2016; Vaswani et al., 2017) are being used for fake news detec- tion(Dun et al., 2021:Ma et al., 2016:Samadi et al.. 2021;Shu et al., 2019;Jiang et al., 2022). Dun et al.(1) proposed a approach which applied knowledge attention network to incorporate ex- ternal knowledge based on Transformer. Jiang et al. a2022) proposed KPL which incorporated prompt learning on the PLMs for the first time and enriched prompt template representations with en- tity knowledge and achieved state-of-ther thetheearttttt art perfor- mance two benchmark datasets, PolitiFact(Shn et al.. 2020) and Gossipcop(Shu et al., 2017).
In this paper, we proposed a Knowledge grAPh enhAnced Language Model (KAPALM) which is enabled to explore both coarse- and fine-grained knowledge in KGs for fake news detection. Firstly, we identify entities from news content and link them the large-scale KGs. An entity graph con- structed by extracting the first-order neighbors of identified entities, and fed into a Graph Neural Net- work (GNN) to obtain coarse-grained structured knowledge representations. Then, we prune the en- tity graph and feed it into the attentive graph pool- ing layer for fine-grained knowledge representa- tions. Finally, we integrate coarse- and fine-grat fine-grained knowledge representations with the representation of news content encoded by PLMs for fake news detection. The primary contributions of this paper can be concluded as follows:
We propose method to construct entity graph for each news article. Our method is enabled extract coarse-grained structured knowledge from the entity graph and provide fine- -ree-graiedd knowledge by the pruned entity graph.
2.2 Knowledge Graphs
External knowledge can provide necessary supple- mentary information for detecting fake news. It is stored in a knowledge graph format, which con- tains information about entities in the real world,

---

[CLS] Dennis Bray and
Hans Storch, found
sigrifiann number
soientsts were
Bert with Adaptor
akeprical the ability
Tecual anooding
ovisting global climata
Leyer
models to accurately
Concatenate
prodict gicbal
[SEP)
Interaction Node
male
Attentive Graph Pooling
Denni s Bray
MLP
KC
retrieval
number
Firegrained
Prediction
Graph pruning
Stageh
Interaction Nods
Denni Bray
Interaction
counting
biotogiat
Layer
Node
number
quanbtt
University of
Camoridge
antiyy knnowl
Figure 2: The overview of proposed model KAPALM
such as people, places, etc. Nowadays, knowl- edge graphs are used in various natural language processing applications, such as news recommen- dation(Wang et al., 2018), fact verification(Zhong et al., 2020;Zhou al., 2019), and fake news de- tection(Dun al., 20ai;Jiang al., 2022). Among them, Dun et al.(2021) designed two attention mechanisms of news towards entities and news towards entities and entity context to capture the different importance of entities in detecting fake news. Jiang et al. (2022) utilized prompt template and integrated entities extracted from news articles into the prompt representation for fake news detec- tion. Hu et al. (Hu al., 2021) proposed novel graph neural model, which compared the news to the knowledge graph through entities for fake news detection. The limitations of the these methods are that they all focus on news entity information and ignore the structured information of KGs associ- ated with the news article. Our proposed model can learn knowledge from large-scale PLMs, acquire entity and topological structure information from external KGs, and improve the performance of fake news detection.
these entities and their corresponding first-order neighbors in the knowledge graph to construct the coarse-grusd -ranseegreieddd entity graph of the news. Entity ex- traction and entity graph construction are described in the knowledge extraction section. Then, feed the constructed entity graph into graph attention network (Velickovic et al., 2017) generate a new entity graph that incorporates both the knowledge graph and entity information. We use the interac- tion node in the entity graph to represent coarse- grained entity knowledge. Furthermore, to obtain fine-grained entity knowledge representation, we prune the constructed entity graph and then attention mechanism for graph pooling. Finally, we concatenate the representations of the news content, coarse- and fine ane-grained entity knowledge repre- sentation, and feed them into fully connected neural network to obtain the prediction for fake news detection.
3.1 Text Encoder
This module is to generate the representation of the news content. We utilize a pre-trained language model with Adapter (Pfeiffer et al., 2020) to en- code the news content for capturing the semantic information contained in the news article. The pre- trained language model is usually pre-trained on large amount of textual corpus, and is able to capture the semantics of the news content. How- ever, when the pre-training task is not similar to the downstream task, fine-tuning pre-trained language model required achieve the performance on the downstream task. In addition, the large number parameters pre-trained lan- guage model can lead to the high time and space
3 Model
We illustrate the framework of KAPALM in Figure 2. The input of KAPALM consists of the news con- tent and the subgraph constructed from KGs. The output of the model is the label for binary classifica- tion of fake news detection. First, for each piece of news, we use pre-trained language model with the adapter layer (Pfeiffer et al., 2020) encode the text news content. Next, entity linking is used to identify entities in new content and we extract

---

costs during the full fine-tuning. Thus, adapter turning is used as a lightweight fine-tuning strat- egy that achieves competitive performance to full fine-tuning on downstream tasks. Adapters add a small set of additional parameters at every layer of Transformer (Vaswani et al., 2017). The parame- ters of pre-trained language model are kept frozen, and the parameters of adapters are trained during fine-tuning. The adapter layer can be represented as the following equation:
we use PLMs to initialize the representations of nodes. While the representation of the interaction node is initialized with the embedding of [CLS] to- ken for the news content, the initialization of other entity nodes is provided by encoding the text of the entity name. As shown in the bottom of Figure 2, the initialized entity graph is fed into a Graph Atten- tion Network (Velickovic et al., 2017) to aggregate the information among all entities. The represen- tation of interaction node enabled to integrate entity knowledge with the contextual representa- tion of news content. Because the entity graph includes many noisy entities, we input only the representation of interaction node which is named the coarse- sranneegrinedd knowledge representation. The hidden representation of coarse-grained knowledge is calculated as follows:
h = h + W RU (oddoo)))))) down)
where Wap and W down are parameter matrixes.
3.2 Knowledge Encoder
3.2.1 Entity Graph Construction and Pruning This module is used extract relevant entities from the knowledge graph and construct two subgraphs. Figure illustrates the pipeline process which in- cludes entity linking, entity graph construction, and entity graph pruning. Firstly, we use entity linking (Milne and Witten, 2008;Sil and Yates, 2013) to extract the entities mentions from the news content, align them with entities in the knowledge graph, and obtain their first-order neighbors in the knowl- edge graph. As a result, we obtain the entity set {e1, (2,,,, } and its first-order neighbor set N = UnEenNe; where Ne; is the first-order neighbor set of Gg Secondly, we construct the en- tity graph G. In order to better aggregate entity graph information, we create an interaction node that is connected to every e; E E, and then connect each e; to its corresponding first-order neighbors in Nog-
a = InteraationnGAAT((
(2)
where G denotes the entity graph and function Interaction returns the representation of the inter- action node.
3.2.3 Fine-grained Knowledge
After pruning the entity graph, we feed the pruned graph into the attentive graph pooling layer to extract the fine-grained entity knowledge repre- sentation. Entities in the entity graph, especially those first-order neighbor entities, do not have the same role in detecting the veracity of news articles. Therefore, we propose to utilize the attentive graph pooling layer which is based on the multi-head at- tention mechanism measure the importance of different entities to fake news detection. The output of the attentive graph pooling layer is called the fine-grained knowledge representation.
Due to the large number of entities in the con- structed entity graph G, the information associ- ated with relevant entities in news content cannot be effectively captured. Therefore, we prune the entity graph to retain only the neighbors on one path of pair of entities. In other words, we re- move the first-order neighbors with degree of one in the entity graph. In this case, we obtain the prumed entity graph G', which consists of all entities in and their remained first-order neigh- bors N' = Ue N' where Ny oe is remained first- order neighbors of ej.
Attentive Graph Pooling Layer As shown in Fig- ure 2, the query is the hidden state of the news article encoded by PLMs, while both the key and value have the same representation which is de- rived from the hidden states of e, E EUN N' where E and N' denote the entity set and the remained neighbour set respectively. Each entity assigned a corresponding weight by calculating the similar- ity between the news and this entity. The attention formula is shown as follows:
Q WWh, K WKC, Wye
(3)
3.2.2 Coarse-grained Knowledge
In this module, we aim to obtain coarse-grainee grained knowledge representations for entities in the entity graph. After the construction of the entity graph,
QKT
Attention (Q, K, K,V) = -Softmar
)V
Vdly
(4)

---

A 2008 survey by two German scienlists, Dennis Bray and Hans significant number scientists skeplical of the ability of existing global climate models to accurately predict global temperatures, precipitation, sea level decade; they
'Dennis Brann--Denni Bray "Hans Hans von Storch 'number'-nnn number
Storch, found
'precipitation" -precipitation "sea level" level
changes, extreme weather events were far more skeptical as the time horizon increased.
'extreme weather -etrrme weather
A Piece of News
(b) Entity Linking
global warming
greenhouse
Hans von
gas
male
male
Hans von Storch
extreme weather
University
human
Cambridge
human
weather
weather
Dennis Bray
Interaction Node
weather
cloud
Bray
Interaction Node
biologist
precipitation
precipitation
Willbrrg
number
number
counting
level
zero level elevation
umbrella
level
quantity
WikipediaVVital
Internat- lonality
Wikipedia:Vitll
articiesiLeveli
the Coast
Entity graph
(d) Pruned entity graph
Figure 3: The process of knowledge extraction.
where h and denote the hidden states of a news article and an entity in EUN N' respectively, and Wo. WK and Wy are parameter matrices.
Experiments
4.1 Datasets
To evaluate the proposed model, we conduct exper- iments two datasets PolitiFact and Gossipcop that are included in benchmark datasets called FakeNewsNet (Shu al., 2020). The detail of these news datasets are shown Table 1. We study our proposed model both few-shot and full-scale settings.
3.3 Knowledge Fusion
3.3.1 Deep Neural Network Classifier
We concatenate the representations of the news con- tent h, granied knowledge representation and fine-grained knowledge representation tain the final representation z. Then, is fed into fully connected layer followed by a softmax func- tion to predict the probability distribution P of the news article's label.
ob-
Few-shot settings For the purpose of replicating low-resource situations in real-world scenarios. we randomly select k E(2, 4, 8, 16. 100) news articles as the training set and create the validation set of the same size. The other news articles used as the test set. We follow (Jiang al., 2022), and sample the few-shot data by 10 random seeds and use the average value calculated after deleting the (6) maximum and minimum scores as the final score. Full-scale settings For a dataset, we reserve 10% of the news articles as the validation set, and 5- fold cross validation is conducted on the other new articles. Finally, the average is reported.
Concat(h, a,
(5)
P Softmax Wa br)
The model is trained minimize the following cross-entropy loss function.
J - Yylog((P.) + 11011
4.2 Implementation Details
ECT
TagMe (Ferragina and Scaiella, 2010) is adopted extract knowledge mentions from news articles. We utilize the BERT-base version with adapter as the pre-trained language model (Pfeiffer et al., 2020) to extract text features, which based on the HuggingFace Transformer Library (Wolf et al.,
where T refers to the training dataset, P; and Y; denote the distributions of the prediction and true label of the sample denotes the coefficient of L2 regularization, and è denotes the model param- eters.

---

Statistics
Politifact
Gossipcop
observe that network architecture to better capture the tem- poral dependencies in text data, We utilize it to judege news's authenticity.
# True news
442
9714
# Fake news
371
4415
# Total news
813
14129
avg. # words/news
1449
576
(6) KCNN (Wang et al., 2018); KCNN a CNN- based model which concatenates news em- bedding and knowledge entities embedding to learn the representation of news.
avg. # entities/news
55
29
Table 1:
Statisties of the new datasets. "#" and "avg.#" denote "the number of" and "the average number of".
(7) KLSTM (Liu et al., 2016:Wang et al.. 2018) (): Similar to KCNN, KLSTM change the CNN module to BiLSTM, which has achieved comptitive results in fake news detection task.
2020). For the coarse-grained entity knowledge, we use GAT (Velickovic et al., 2017) as our graph neural network model. In the attentive graph pool- ing layer, set the attention head to 2. The size of hidden layer is set to 200 and the dropout rate to 0.2 in the MLP layer. Adam is used optimize the model's parameters in the training, The learning rate le-5 and the batch size is 10 for training model.
(8) FB (Peters et al., 2018): FB denotes the feature- baseee-bbbddddd method of utilizing the pretrain language model for feature extraction. BERT base version used in the experiments.
FFT (Devlin al., 2018): FFT is the full fine-tune approach based on BERT base.
Because our aim focuses on detecting fake news, fake news articles are regarded positive examples and FI F1-score (F1) is used the evaluation metric measure the classification performance.
(10) KAN (Dun al., 2021): KAN a knowledge- aware attention network which incorporates external knowledge entities through attention mechanisms to predict the veracity of news articles.
4.3 Baselines
We compare our proposed model with the follow- ing baselines:
(11) KPL (Jiang et al., 2022): KPL is the state-of- art model which designs one prompt template and incorporates external knowledge entities into the prompt representation.
(1) DTC (Castillo et al., 1): DTC is the deci- sion tree model, which detects the authenticity of news by utilizing hand-crafted features.
(2) RFC (Kwon et al., 2013); RFC is the random forest classifier based on hand-crafted features to detect whether news is true or fake.
4.4 Experimental Results
Experiments are conducted in both few-shot and full-scale settings on two datasets. Baseline mod- els are divided into five categories: traditional sta- tistical methods (i.c., DTC, RFC, SVM), neural network methods without external knowledge (i.e.. TextCNN, BiLSTM), neural network methods with external knowledge (i.e., KCNN, KLSTM, KAN), pre-trained language model methods without ex- ternal knowledge (i.e., FB, FFT), and pre-trained language model methods with external knowledge (i.e., KPL).
(3) SVM (Yang et al., 2012): SVM denotes a classification model that uses a hyperplane to separate news into true and fake news in high dimensional feature space.
(4) TextCNN (Kim, 2014): TexICNN is a pop- ular deep learning model for text classifica- tion, which applies convolutional filters with various window sizes to extract text features. These features are fed into pooling layer to furtherly capture the most salient features to judge whether news is true or fake.
The experimental results are presented in Table 2. We draw some conclusions in the few-shot setting. First, although the performance is not stable when the number of training data varies, the increase of training data usually gives rise the improvement. Second, the deep learning meth- ods without external knowledge are usually worse than the statistical methods. The cause may be that
(5) BiLSTM (Bahad et al., 201::Hochreiter and Schmidhuber, 1997): BiL.STM denotes the bidirectional long short-term memory which introduces a two-directional recurrent netural

---

Data
Method
Few shot
Full scale
2
4
8
16
100
DTC
43.16
50.89
57.31
53.42
74.58
73.44
RFC
30.43
48.37
38.28
69.66
82.73
82.24
SVM
24.63
47.20
51.13
60.91
84.90
86.29
TexICNN
20.91
19.88
31.91
51.44
75.72
84.34
BiLSTM
30.36
29.78
45.75
62.11
76.11
85.96
KCNN
20.89
16.24
31.43
55.71
76.80
85.40
PolitiFact
KLSTM
29.43
20.44
34.96
65.48
76.60
86.41
KAN
34.05
38.05
47.94
66.27
76.91
87.28
FB
33.26
39.65
48.34
35.96
75.93
61.02
FFT
38.88
20.77
47.04
39.83
84.40
87.35
KPL
61.25
63.92
68.34
74.60
83.51
89.52
KAPALM(OUrs)
52.78
52.66
55.49
71.74
85.77
91.34
DTC
25.62
32.40
33.02
34.35
40.90
50.82
RFC
33.88
30.49
34.04
37.93
49.28
43.73
SVM
33.69
38.34
40.20
44.66
54.36
65.03
TexICNN
23.68
21.22
19.69
16.84
36.26
66.82
BiLSTM
26.06
30.76
37.58
33.36
41.43
66.78
KCNN
23.31
19.75
19.12
15.84
38.26
66.51
Gossipcop
KLSTM
26.54
27.76
20.45
20.84
42.25
65.94
KAN
29.30
31.98
33.56
35.94
43.57
67.35
FB
32.96
37.16
34.40
33.23
52.98
58.81
FFT
34.92
40.05
39.05
30.67
58.79
70.71
KPL.
37.80
38.78
40.20
41.63
51.72
69.20
KAPALM(Ours)
42.26
44.58
44.69
44.51
60.03
71.68
Table 2: Comparsion with existing models. These scores refer to the F1 scores (%) of fake news.
Data
Method
Full scale
Data
Method
Full scale
Ours
91.34
FB+Graph
87.27
PolitiFact
w/o GP
90.05
PolitiFact
FFT+Graph
90.05
w/o IN
89.72
Ours(AT+Graph)
91.34
Ours
71.68
FB+Graph 50.99
Gossipcop
w/o GP 70.99
Gossipcop
FFT+Graph 71.53
w/o IN
71.02
Ours(AT+Graph) 71.68
Table 3: Ablation experimental results. "w/o GP" means removing attentive graph pooling layer. "w/o -IN" means that we remove interaction node part.
Table 4: Comparison with different fine-tuning methods. "FB+Graph" means that training our model by keep- ing parameters of pre- train language model frozen and updating other parameters only. "FFT+Graph means training our model by full fine-tuning all parameters of the model. "AT+Graph" means training our model by adapter tuning just small amount of additional param- eters.
deep learning methods have more parameters than statistical methods and are easy to be over-fitted by the lack of training data. Third, neural net- work methods with external knowledge are better than those without external knowledge. The results demonstrate that knowledge integration allevi- ate the over-fitting problem some extent. Four, the pre-trained language models can improve the effectiveness of fake news detection generally. Our model is better than KPL most cases, but wrose
than KPL in the few-shot setting on the FolitiFact dataset. In the full-scale setting, our method outperforms all baselines on two datasets and achieves the highest Fl score. Specifically, KPL is the recent state-of-art model that incorporates large-scale pre-

---

Data
Method
Full scale
Tuning (FFT) or Feature-Based (FBB, our model worse than "FB+Graph" and "FFT+Graph" on two datasets. Our model can benefit from adapter tuning.
Fine-grained
90.88
PolitiFact
Course- and Fine-grained
91.34
Fine-grained
71.48
Gossipcop
Coarse- and Fine-grained
71.68
We conduct the ablation study to validate the ne- cessity of pruning the entity graph before feeding into the attentive graph pooling layer. We feed the pruned entity graph into both GAT layer and the attentive graph pooling layer. As Table 5 demon- strated, the performance of using both the coarse- and fine-grained knowledge is better than just us- ing fine- fine-grained knowledge on two datasets. Thus, coarse-grained knowledge representations are ben- eficial to fake news detection.
Table Ablation experimental results. "Fine-graine "Fine-grained" denotes that the pruned entity graph are fed into both GAT layer and attentive graph pooling layer. "Coarse- and Fine-graide sine-grained" means that the original entity graph is fed into GAT layer while the pruned entity graph is fed into attentive graph pooling layer.
trained models and external knowledge, and our model is better than KPL. with +1.82 and +2.45 improvement on FolitiFact and Gossipcop datasets respectively. Thus, our model can effectively adopt both external knowledge and pre-training language model. In addition, neural network methods with external knowledge are superior to those without external knowledge. It suggest that knowledge can also improve the effectiveness of fake news detec- tion when the training data is sufficient.
Conclusion
In this paper, we propose the Knowledge grAPh enhAnced Language Model (KAPALM) for fake news detection. The proposed model integrates coarse- and fine-grained representations of entity knowledge. Entity graph is used to enrich the knowledge representation of the news article. The entity graph is pruned and fed into the attentive graph pooling layer to represent the fine-grained knowledge. The coarse- and fine-grained knowl- edge representations extracted from large-scale knowledge graph are combined for improving the fake news detection. Experimental result on two benchmark datasets have shown that the proposed KAPALM outperforms the wtate-oflle the art baseline models. In addition, KAPALM is able to obtain the competitive performance in the few-shot setting. In future work, we will investigate other effective approaches to mine the accurate knowledge from knowledge graph for fake news detection.
4.5 Analysis
We conduct detailed analysis from different per- spectives to demonstrate the effectiveness of the modules in our model for fake news detection.
4.5.1 A Ablation study
We conduct the ablation study to validate the ef- fectiveness of the coarse- and fine- grained knowl- edge in our approach. Table 3 presents the ex- perimental results of the models on two datasets. The coarse-grained knowledge is represented by the interaction node in the entity graph while the fine-grained knowledge is provided by the attentive graph pooling layer. If the attentive graph pooling layer is eliminated from our model, the F1 drops by 1.34 on the Politifacc dataset, and F1 decreases by 0.69 the Gossipcop dataset. When the inter- action node is removed from our model, the per- formance of our model declines by 1.6 Fl on the PolitiFact dataset and by 0.66 Fl on the Gossipcop dataset. Therefore, both the coarse- -rarseeg knowl- edge and fine-grained knowledge are important for our model and able to improve the effectiveness of fake news detection independently.
Limitations
One limitation of our model is that the constructed entity knowledge graph fails to consider the mul- tiple types of relationships or extra attribute infor- mation of entities and relationships. If the veracity of a news article is associated with the attributes of the entities the relationships among entities in this article, it may lead to the poor performance. In addition, although graph neural networks and attentive graph pooling layers can improve the per- formance in fake news detection, there is still a lack of the Interpretability for our model.
To validate the contribution of adapting tuning. we conduct an ablation study experiment on two datasets. The results listed in Table demon- strate the superiority of adapter tuning. When the adapter turning method is changed to Full Fine-

---

Acknowledgements
Conference on Information and Knowledge Manage- ment, CIKM 2010. Toronto, Ontario, Canada, Octo- ber 26-30, 2010, pages 1625-1628. ACM.
This work was partially supported by the NSFC- General Technology Joint Fund for Basic Research (No. U1936206, U1936105), the National Natu- ral Science Foundation of China (No. 62372252, 62172237, 62077031, 62176028, 62302245), Min- istry of Education of the People's Republic of China Humanities and Social Sciences Youth Foun- dation (No. 632321 14). We thank the AC, SPC, PC and reviewers for their insightful comments on this paper.
Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural Comput., 98):11335 1780.
Linmei Hu, Tianchi Yang. Luhao Zhang, Wanjun Zhong, Duyu Tang. Chuan Shi, Nan Duan, and Ming Zhou, 2021. Compare to the knowledge: Graph neural fake news detection with external knowledge. In Proceed- ings of the 59th Annual Meeting of the Association for Computational Linguisties and the I Itth International Joint Conference Natural Language Processing (Volume 1: Long Papers), pages 754-763.
References
Gongyao Jiang, Shuang Liu, Yu Zhao, Yucheng Sun, and Meishan Zhang. 2022. Fake news detection via knowledgeable prompt learning. Inf. Process. Manag, 59(5):10302)
Oluwaseun Ajao, Deepayan Bhowmik, and Shahrzad Zargari. 2019. Sentiment aware fake news detection on online social networks. In IEEE International Conference on Acoustics, Speech and Signal Process- ing, ICASSP 2019, Brighton, United Kingdom, May 12-17, 2019, pages 2507-2511. IEEE.
Yoon Kim. 2014. Convolutional neural networks for sentence classification. In Proceedings of the 2014 Conference on Empirical Methods in Natural Lan- guage Processing, EMNLP 2014, October 25-29, 2014. Doha, Qatar, meeting of SIGDAT, a Spe- cial Interest Group of the ACI, pages 1746-1751. ACL.
Pritika Bahad, Preeti Saxena, and Raj Kamal. 2019. Fake news detection using bi-directional Istm- recurrent neural network. Procedia Computer Sci- ence, 165:74-82. 2nd International Conference on Recent Trends Advanced Computing ICRTAC DISRUP- TIV INNOVATION, 2019 November 11- 12, 2019.
Sejeong Kwon, Meeyoung Cha, Kyomin Jung, Wei Chen, and Yajun Wang. 2013. Prominent features of rumor propagation in online social media. In 2013 IEEE 13th International Conference on Data Min- ing, Dallas, TX, USA, December 7-10, 2013, pages 1103- 1008. IEEE Computer Saciety.
Kurt Bollacker, Colin Evans, Praveen Paritosh, Tim Sturge, and Jamie Taylor. 2008. Freebase: a collabo- ratively created graph database for structuring human knowledge. In Proceedings of the 2008 ACM SIG- MOD international conference on Management of data, pages 1247-1250.
Pengfei Liu, Xipeng Qiu, and Xuanjing Huang. 2016. Recurrent neural network for text classification with multi-task learning. In Proceedings of the Twenty- Fifth International Joint Conference on Artificial In- telligence, IJCAI 2016, New York. NY, USA, 9-15 July 2016, pages
Carlos Castillo, Marcelo Mendoza, and Barbara Poblete. 2011. Information credibility on twitter. In Proceed- ings of the 20th International Conference on World Wide Web, WWW 2011, Hyyeerabad, India, March 28 April 1, 2011, pages 675-684, ACM.
IICAT/ААI Press.
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Man- dar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: robustly optimized BERT pretraining approach. CoRR, abs/1907, 1162..
Jacob Devlin, Ming-Wa Wei Chang, Kenton Lee, and Kristina Toutanova. 2018 BERT: pre-training of deep bidirectional transformers for language under- standing. CoRR, abs/ 181.044005
Jing Ma. Wei Gao, Prasenjit Mitra, Sejeong Kwon, Bernard Jansen, Kam-Fai Wong. and Meeyoung Cha. 2016. Detecting rumors from microblogs with recurrent neural networks. In Proceedings of the 25th International Joint Conference on Artificial Intelli- gence, pages 3818-3824.
Yaqian Dun, Kefei Tu, Chen Chen, Chunyan Hou, and Xiaojie Yuan. 2021. KAN: knowledge-awan atten- tion network for fake news detection. In Thirty-Fifth AAAI Conference on Artificial Intelligence, AAAI 2021, Thirty-Third Conference on Innovative Ap- plications of Artificial Intelligence, IAAI 2021, The Eleventh Symposium on Educational Advances in Ar- rificial Intelligence, EAAI 2021, Virtual Event, Febru- ary 2-9, 2021, pages 81-89. AAAI Press.
David N. Milne and Ian H. Witten 2008. Learning to link with wikipedia. In Proceedings of the 17th ACM Conference on Information and Knowledge Manage- ment, CIKM 2008 Napa Valley, California, USA, October 26-30, 2008 pages 519-518. ACM.
Paolo Ferragina and Ugo Scaiella. 2010. TAGME:
on-the-fly annotation of short text fragments (by wikipedia entities). In Proceedings of the 19th ACM
Jeff Pan, Siyana Pavlova, Chenxi Li, Ningxi Li, Yang- mei Li, and Jinshuo Liu. 2018. Content based fake

---

news detection using knowledge graphs. In Proceed- ings of The 17th International Semantic Web Confer- ence, pages 669-683.
CA. USA, October 27- November 2013, pages ACM.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Pro- cessing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA, pages 5998-6008.
Kellin Pelrine, Jacob Danovitch, and Reihaneh Rabbany. 2021. The surprising performance of simple base- lines for misinformation detection. In Proceedings of the Web Conference 2021, pages 3432-33 -441.
Matthew E. Peters, Mark Neumann, Mohit lyyer, Matt Gardner, Christopher Clark, Kenton Lee,, and Luke Zettlemoyer. 2018. Deep contextualized word rep- resentations. In Proceedings of the 2018 Confer- ence of the North American Chapter of the Associ- ation for Computational Linguistics: Human Lan- guage Technologies. NAACL-HLT 2018, New Or- leans, Louisiana, USA, June 1-6, 2018, Volume I (Long Pupers), pages 2227-2237. Association for Computational Linguistics.
Petar Velickovic, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Ben- gio. 2017. Graph attention networks. CoRR. abs/1710..10003
Soroush Vosoughi, Deb Roy, and Sinan Aral. 2018. The spread of true and false news online. Science, 359(6380): 146--1151
Jonas Pfeiffer, Andreas Rücklé Clifton Poth, Aishwarya Kamath, Ivan Vulié, Sebastian Ruder, Kyunghyun Cho, and Iryna Gurevych. 2020. Adapterhub: framework for adapting transformers. In Proceedings of the 2020 Conference on Empirical Methods in Nat- ural Language Procexsing (EMNLP 2020): Systems Demonstrations, pages 46-54, Online. Association for Computational Linguistics.
Denny Vrandecic and Markus Krötzsch. 2014. Wiki- data: free collaborative knowledgebase. Commun. ACM, 57(10):78-85.
Hongwei Wang. Fuzheng Zhang, Xing Xic, and Minyi Guo. 2018. DKN: deep knowledge-awe network for news recommendation. CoRR, abs/1801 .08284.
William Yang Wang. 2017. "liar, liar pants on fire": A new benchmark dataset for fake news detection. In Proceedings of the 55th Anmual Meeting of the Associ- ation for Computational Linguistics, pages 422-426.
Piotr Przybyla. 2020. Capturing the style of fake news.
The Shirty-FFourth Conference on Artificial Intelli- gence, pages 490-497. AAAT Press.
Mohammadreza Samadi, Maryam Mousavian, and Sacedeh Momtazi. 2021. Deep contextualized text representation and learning for fake news detection. Inf. Process. Manag., 1027)3.
Thomas Wolf, Lysandre Debut, Victor Sanh. Julien Chaumond, Clement Delangue, Anthony Moi, Pier- ric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, et al. 2020. Transformers: State-of thet---rttttttttt natural language processing. In Proceedings of the 2020 con- ference on empirical methods in natural language processing: system demonstrations, pages 38-45.
Qiang Sheng. Xueyao Zhang. Juan Cao, and Lei Zhong. 2021. Integrating pattern-and fact-based fake news detection via model preference learning. In Proceed- ings of the 30th ACM international conference on information & knowledge management, pages 1640- 1650.
Fan Yang, Yang Liu, Xiaohui Yu, and Min Yang. 2012. Automatic detection of rumor on sina weibo. In Pro- ceedings of the ACM SIGKDD Workshop on Mining Data Semantics, MDS '12, New York, NY, USA. Association for Computing Machinery.
Kai Shu, Limeng Cui,, Suhang Wang, Dongwon Lee,
and Huan Liu. 2019, defend: Explainable fake news detection. In Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining, pages 395-405.
Xueyao Zhang, Juan Cao, Xirong Li. Qiang Sheng, Lei Zhong, and Kai Shu. 2021. Mining dual emotion for fake news detection. In WWW '21: The Web Conference 2021, Virtual Event / Ijubljana, Slovenia, April 19.23, 2021, pages 3465 3476. ACM/ / IW3C2.
Kai Shu, Deepak Mahudeswaran, Suhang Wang, Dong- won Lee, and Huan Liu. 2020. Fakenewsnet: A data repository with news content, social context, and spa- tiotemporal information for studying fake news on social media. Big Data 8(3):111-188
Wanjun Zhong, Jingjing Xu. Duyu Tang. Zenan Xu. Nan Duan, Ming Zhou, Jiahai Wang, and Jian Yin. 2020. Reasoning over semantic-level graph for fact checking. In Proceedings of the 58th Annual Meet- ing of the Association for Computational Linguistics, ACL 2020, Online, July 5-10, 2020, pages 6170-6180 Association for Computational Linguistics.
Kai Shu, Amy Sliva, Suhang Wang, Jiliang Tang, and Huan Liu. 2017. Fake news detection on social dia: A data mining perspective. SIGKDD Explor. 191222333
Avirup Sil and Alexander Yates. 2013. Re-ranking for joint named-entity recognition and linking. In 22nd ACM International Conference on Information and Knowledge Management, CIKM'13 13, San Francisco,
Jie Zhou, Xu Han, Cheng Yang, Zhiyuan Liu, Lifeng Wang, Changcheng Li, and Maosong Sun. 2019. GEAR: graph-based evidence aggregating and rea- soning for fact verification. In Proceedings of the

---

57th Conference of the Association for Computa- tional Linguistics, ACL 2019, Florence, Italy, July 28- August 2, 2019, Volume 1: Long Papers, pages 892-901. Association for Computational Linguistics.
Xinyi Zhou and Reza Zafarani. 2021. A survey of fake news: Fundamental theories, detection methods, and opportunities. ACM Comput. Surv. 53(5): 109:1- 109:40.
Arkaitz Zubiaga, Maria Liakata, and Rob Procter. 2017. Exploiting context for rumour detection in social me- dia. In Social Informatics- 9th International Confer- ence, Socinfo 2017, Oxford, UK, September 13-15, 2017, Proceedings, Part I, volume 10539 of Lecture Notes in Computer Science, pages 109-123. Springer.


Reviews:
[{"review_id": "m18NpFwGiw", "reviewer": "Reviewer_4XMg", "summary": "", "questions": "Did you conduct error analysis to explore the reason why your proposed method is worse than KPL in the few shot setting on Politifact, but better than KPL on Gossipcop? ", "limitations": "", "rating": 3, "confidence": 3, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Reviewer#1: Thanks for your valuable and careful comments. We will explain you concerns point by point.\nQ1: About lacking statistical significance tests.\nA1: We will supplement with statistical significance tests in subsequent work.\n\nQ2: About lacking error analysis.\nA2: Because of the difference between the two datasets. The average number of entities per news in the Politifact is 55, while the average number of entities per news in the Gossipcop is 29. In few shot setting, due to the lack of samples, the noise generated by entities in the Politifact have greater impact on the prediction results, resulting in my proposed method being worse than the KPL model in a few shot setting on the Politifact, but better than the KPL model on the Gossipcop.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The approach is novel and achieves SOTA on two datasets with full scale training, and show promising performance in the few-shot setting.", "reasons_to_reject": "Lack statistical significance tests. ", "questions_to_authors": "Did you conduct error analysis to explore the reason why your proposed method is worse than KPL in the few shot setting on Politifact, but better than KPL on Gossipcop? ", "paper_topic_and_main_contributions": "The paper investigate to integrate structured knowledge among entities in the news article to enhance the fake news detection. The proposed method outperforms previous KG related approaches that do not utilise structured information. It also shows promising results in the few-shot setting.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "UT8FMj1NCh", "reviewer": "Reviewer_oZm4", "summary": "", "questions": "(1) It is unclear to me that text properties alone (together with what text properties of external sources add to that) would be sufficient to distinguish fabricated news from real news. This may be true for some datasets but I imagine that with more effort put into creating stories (and more sophisticated tools such as those from the GPT family) it would be impossible to distinguish the two classes without incorporating more signals (such as contextual features). What is the underlying intuition in your work? Do you argue that even with the widepread adoption of ChatGPT et al. it will be possible to distinguish fake news from real news based on text (+ entities) alone?  \n\n(2) Given you adopt a dataset that includes many more signals I wonder why you do not include all these and THEN try to push forward the state of the art by going beyond what you can achieve. What is the reason to only try to improve on the text classification alone? \n\n(3) Is there a need to include all the baselines? I have the impression that none of 1 - 7 are actually needed as we can assume for BERT to be a stronger baseline (it is also not a surprise that the results demonstrate yet again that traditional methods do not appear to be competitive with neural methods).\n\n(4) Why did you not include a discussion of ethical issues? Given the topic this would have been a helpful addition. ", "limitations": "", "rating": 2, "confidence": 4, "soundness": 2, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Reviewer#2:We thank the reviewer for the deeper thought about the property of the union graph and appreciate your recognition about the contribution of the property. \nQ1：About weak baseline.\nA1: We did not use contextual information. The article you mentioned used contextual information, which is why its results are higher than ours. Our comparison with similar methods (using only news context and external knowledge) can demonstrate the superiority of our model.\n\nQ2: About dataset.\nA2: In fact, Politicact and Gossipcop are two completely different datasets rather than one dataset. \n\nQ3: About missing details.\nA3: I will upload the code for constructing the knowledge graph and all related codes to Github for public disclosure so that people can reproduce my work. The process of constructing a knowledge graph is detailed in sections 3.3, 3.4, and 3.5 of the paper.\n\nQ4: About lacking statistical significance tests.\nA4: We will supplement with statistical significance tests in subsequent work.\n\nQ5: About the necessary to include all baseline.\nA5: Our method is mainly used to compare with the KPL model (a strong baseline in this field). As the latter conducted 1-7 experiments, in order to maintain consistency with his experiments, I set 1-7 as the baseline model.\n\nQ6: About underlying intuition and ChatGPT challenge.\nA6: Our underlying intuition is that news content is highly condensed and comprised of a large number of entity which are useful to detect fake news. About ChatGPT challenge, my answer is yes. My model can detect fake news by using text and external knowledge. \n\nQ7: About the reason why we do not include all signals.\nA7: The dataset does not contain contextual information, so we cannot obtain other relevant signals.\n\nQ8: About ethical issues.\nA8:We will supplement with ethical issues in subsequent work.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "* This is a highly relevant research area, and any work that pushes our understanding as to how to identify misinformation of any form is a step forward.\n\n* The work is conducted on a well-known benchmark dataset and is therefore easily contextualisable with other work in the field. \n\n* Conducting an ablation study is a strong aspect of the experimental work as it offers more nuanced insights into what components of the architecture contribute how much  to the overall performance. ", "reasons_to_reject": "There are a number of weaknesses in this work. Among the biggest issues I consider the following: \n\n* Weak baselines: Given the chosen dataset contains much more information than the textual content I would want to see the results of this work compared to the state of the art reported in the literature that looks at the same dataset. To pick just one recent example [1], the results reported (such as F1) appear to be way higher than anything in Table 2 (and that is also true for the baselines reported in [1]). What is needed is a comparison against what the current state of the art is as reported in the literature (ideally reproduced to conduct significance tests where appropriate). \n\n* Only a single dataset is used to explore the problem (well, it is two different parts but in the end it is one fairly specific dataset). There are many more benchmark datasets for text classification (including fake news detection) that could be included to provide more confidence in the findings. \n\n* There are many missing details (and no supplementary material such as code) making it impossible to replicate the work. For example, unless I have overlooked it I cannot see what knowledge graphs are actually being used. \n\n* There are no statistical significance tests (and terms such as \"outperform\" should therefore not be used) \n\n\n\n\n[1] Donabauer \"Exploring Fake News Detection with Heterogeneous Social Media Context Graphs\". ECIR 2023.", "questions_to_authors": "(1) It is unclear to me that text properties alone (together with what text properties of external sources add to that) would be sufficient to distinguish fabricated news from real news. This may be true for some datasets but I imagine that with more effort put into creating stories (and more sophisticated tools such as those from the GPT family) it would be impossible to distinguish the two classes without incorporating more signals (such as contextual features). What is the underlying intuition in your work? Do you argue that even with the widepread adoption of ChatGPT et al. it will be possible to distinguish fake news from real news based on text (+ entities) alone?  \n\n(2) Given you adopt a dataset that includes many more signals I wonder why you do not include all these and THEN try to push forward the state of the art by going beyond what you can achieve. What is the reason to only try to improve on the text classification alone? \n\n(3) Is there a need to include all the baselines? I have the impression that none of 1 - 7 are actually needed as we can assume for BERT to be a stronger baseline (it is also not a surprise that the results demonstrate yet again that traditional methods do not appear to be competitive with neural methods).\n\n(4) Why did you not include a discussion of ethical issues? Given the topic this would have been a helpful addition. ", "paper_topic_and_main_contributions": "Identifying fake news is a highly relevant topic and one that is at the core of NLP given the importance that written text contributes to solving the task. As such it presents a great topical match for EMNLP. \n\nThe authors argue that identifying entities in some text and incorporating relationships between entities drawn from some external knowledge graphs might help in this classification task. The architecture that is being proposed is one that generates embeddings from three different angles: the actual text, an entity graph based on entities in the text and related entities in some external graph, a pruned version of this graph. The three embeddings resulting from this process are concatenated before being fed into a binary classifier.\n\nExperiments on the FakeNewsNet dataset are reported. \n", "excitement": 2, "reproducibility": 2, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Typos_Grammar_Style_and_Presentation_Improvements": "The paper needs to be properly proofread. It has many language problems throughout."}, "extra_text": {"Missing_References": "You should include references that represent the state of the art in respect to whatever dataset(s) you have chosen.\n\nMake sure you include peer-reviewed versions of a paper where this is appropriate (e.g. the BERT paper). "}}, {"review_id": "6CzNgmsIqj", "reviewer": "Reviewer_nxyk", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": 4, "soundness": 2, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Reviewer#3: We thank the reviewer for the careful comments. We would like to make some clarifications as follows:\n\nQ1: About reason for using Bert instead of Roberta .\nA1: Roberta is a more finely-tuned version of Bert, so in terms of training, the Roberta model is more powerful than the Bert model. If the performance of our model using Bert exceeds that of using Roberta, it can demonstrate the advantage of our model.Using a Bert model with adapters has two reasons. Firstly, adapters can significantly reduce the time and space costs of training. Secondly, only Bert-base-with-adapter has a readily available library of pre-trained models that can be used. However, we have not found a pre-trained library for Roberta with adapters.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "•This work copes with the fake news classification task, which is useful in practice. \n\n•The author proposed an effective model for the fake news detection task and verified its effectiveness on the public test set ", "reasons_to_reject": "•The motivation stated in the abstract is not solid enough, and the method proposed in the paper is not novel enough. For the statement \" majority of these methods focus on news entity information and ignore the structured knowledge among news entities\" in the abstract, some existing methods already do this, like CompareNet[1].\n\n•The comparison of the proposed method with the baseline is not fair enough. The Bert-base-with-adapter used in the article is inconsistent with the Roberta used by KPL, and it is difficult to prove that KAPALM is more effective than KPL. What is the motivation for Bert-base-with-adapter?\n\n•The coarse-grained and fine-grained knowledge mentioned in the article is not clear enough. The knowledge pruning method proposed in the article is intuitive.\n\n[1]. Hu L, Yang T, Zhang L, et al. Compare to the knowledge: Graph neural fake news detection with external knowledge[C], in ACL2021.\n", "questions_to_authors": "", "paper_topic_and_main_contributions": "The authors propose a knowledge graph enhanced language model (KAPALM) that fuses coarse- and fine-grained representations of entity knowledge from knowledge graphs to conduct fake news detection. They design a method to prune subgraphs, and then use attention mechanism to obtain knowledge representation. By combining it with a textual representation, KAPALM could further improve the detection performance.", "excitement": 1, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper presents a method to combine entity knowledge from KGs with language models for imporving the performance on the task of fake news detection, as measured on two benchmark data sets. Entity mentions are extracted from text and linked to a knowledge graph, then a subgraph is extracted and used in a graph-neural network, and its output combined with a representation of text obtained using a Transformer-based model.

All reviewers appreciated the results and the introduction of a new method that is effective on standard benchmark data sets. One reviewer also appreciated the ablation experiments, which provided additional insights in the results.

Two reviewers mentioned needing to add statistical significance results on the experiments and also having more baseline methods that use information from Knowledge graphs and a more detailed discussion on part work that did this. Doing so would have likely increased the soundness scores.

All reviewers acknowledged to reading the author response, but did not change their scores as a result of it.
Decision: Accept-Findings