KAPALM : Knowledge gr AP h enh A nced Language Models for Fake News Detection (2023)

Abstract: Social media has not only facilitated news consumption, but also led to the wide spread of fake news. Because news articles in social media is usually condensed and full of knowledge entities, existing methods of fake news detection use external entity knowledge. However, majority of these methods focus on news entity information and ignore the structured knowledge among news entities. To address this issue, in this work, we propose a Knowledge grAPh enhAnced Language Model (KAPALM) which is a novel model that fuses coarse- and fine-grained representations of entity knowledge from Knowledge Graphs (KGs). Firstly, we identify entities in news content and link them to entities in KGs. Then, a subgraph of KGs is extracted to provide structured knowledge of entities in KGs and fed into a graph neural network to obtain the coarse-grained knowledge representation. This subgraph is pruned to provide fine-grained knowledge and fed into the attentive graph and graph pooling layer. Finally, we integrate the coarse- and fine-grained entity knowledge representations with the textual representation for fake news detection. The experimental results on two benchmark datasets show that our method is superior to state-of-the-art baselines. In addition, it is competitive in the few-shot scenario.

## introduction in recent years, with the development of the internet, social media such as facebook and twitter have become the main platforms for people to consume news due to their real-time and easy access. however, social media is a double-edged sword and it enables people to be exposed to fake news. the wide spread of fake news can misguide public opinion, threaten people's health, and even cause devastating effects on society @xcite . thus, the research on automatic fake news detection is desirable. prior researches are based on traditional machine learning methods and hand-crafted features based on news content to combat fake news on social media @xcite @xcite ). to avoid manual features, deep neural networks models such as convolutional neural networks (cnn) and recurrent neural network (rnn), have been used to learn the high-level feature representations automatically and achieve great performance in detecting fake news @xcite @xcite . for the past few years, pre-trained language models (plms), such as bert @xcite and roberta @xcite , have been the mainstream approaches which provide the contextualized representation of textual content in natural language processing field and also achieve the competitive performance for fake news detection @xcite . however, these works fail to consider the knowledge entities in news articles. although news content is condensed and full of knowledge entities by which people usually verify the veracity of news content, plms are not effective in capturing the knowledgelevel relationship between entities. as shown in the left of figure 1 , it is hard to detect the veracity of news article exactly without the entity knowledge about dennis bray and hans von storch. large knowledge graphs (kgs), such as freebase @xcite and wikidata @xcite , contain a large number of structured triplets, which can serve as the entity knowledge base for detecting fake news. existing researches have demonstrated the significant role of @xcite @xcite . specifically, @xcite proposed knowledge attention networks to measure the importance of knowledge and incorporate both semantic-level and knowledge-level representations of news content. @xcite proposed a knowledge prompt learning method which incorporated kgs into the prompt representation to make it more expressive for verbal word prediction. hu et al. @xcite explored the structural entity embedding and compared contextual entity representations with the corresponding entity representations in the knowledge graph. however, they focus on the textual representation of entities in kgs and have not explored the in-depth structured knowledge associated with the new article. as shown in figure 1 , we construct a subgraph of kgs for the news article which is named as entity graph throughout this paper, and this entity graph is helpful to detect fake news with the knowledge that "dennis bray is a biologist and hans von storch is a climatologist and meteorologist". in this paper, we proposed a knowledge graph enhanced language model (kapalm) which is enabled to explore both coarse-and fine-grained knowledge in kgs for fake news detection. firstly, we identify entities from news content and link them to the large-scale kgs. an entity graph is constructed by extracting the first-order neighbors of identified entities, and fed into a graph neural network (gnn) to obtain coarse-grained structured knowledge representations. then, we prune the entity graph and feed it into the attentive graph pooling layer for fine-grained knowledge representations. finally, we integrate coarse-and fine-grained knowledge representations with the representation of news content encoded by plms for fake news detection. the primary contributions of this paper can be concluded as follows: ## model we illustrate the framework of kapalm in figure 2 . the input of kapalm consists of the news content and the subgraph constructed from kgs. the output of the model is the label for binary classification of fake news detection. first, for each piece of news, we use a pre-trained language model with the adapter layer @xcite to encode the text of news content. next, entity linking is used to identify entities in new content and we extract these entities and their corresponding first-order neighbors in the knowledge graph to construct the coarse-grained entity graph of the news. entity extraction and entity graph construction are described in the knowledge extraction section. then, we feed the constructed entity graph into a graph attention network @xcite to generate a new entity graph that incorporates both the knowledge graph and entity information. we use the interaction node in the entity graph to represent coarsegrained entity knowledge. furthermore, to obtain fine-grained entity knowledge representation, we prune the constructed entity graph and then use an attention mechanism for graph pooling. finally, we concatenate the representations of the news content, coarse-and fine-grained entity knowledge representation, and feed them into a fully connected neural network to obtain the prediction for fake news detection. ## conclusion in this paper, we propose the knowledge graph enhanced language model (kapalm) for fake news detection. the proposed model integrates coarse-and fine-grained representations of entity knowledge. entity graph is used to enrich the knowledge representation of the news article. the entity graph is pruned and fed into the attentive graph pooling layer to represent the fine-grained knowledge. the coarse-and fine-grained knowledge representations extracted from large-scale knowledge graph are combined for improving the fake news detection. experimental result on two benchmark datasets have shown that the proposed kapalm outperforms the state-of-the-art baseline models. in addition, kapalm is able to obtain the competitive performance in the few-shot setting. in future work, we will investigate other effective approaches to mine the accurate knowledge from knowledge graph for fake news detection.