Automatic Detection of Entity-Manipulated Text using Factual Knowledge (2022)

Abstract: In this work, we focus on the problem of distinguishing a human written news article from a news article that is created by manipulating entities in a human written news article (e.g., replacing entities with factually incorrect entities). Such manipulated articles can mislead the reader by posing as a human written news article. We propose a neural network based detector that detects manipulated news articles by reasoning about the facts mentioned in the article. Our proposed detector exploits factual knowledge via graph convolutional neural network along with the textual information in the news article. We also create challenging datasets for this task by considering various strategies to generate the new replacement entity (e.g., entity generation from GPT-2). In all the settings, our proposed model either matches or outperforms the state-of-the-art detector in terms of accuracy. Our code and data are available at https://github.com/UBC-NLP/manipulated_entity_detection .

## introduction a type of fake news that has received little attention in the research community is manipulated text. manipulated text is typically created by manipulating a human written news article minimally (e.g., replacing every occurrence of a particular entity, 'obama' in a news article with another american politician entity). current fake news detectors that exploit stylometric signals from the text (e.g., choice of specific words to express false statements) are clearly insufficient for distinguishing manipulated text from human written text @xcite as the style underlying the manipulated text is virtually identical to human writing style. in this work, we focus on this problem of distinguishing manipulated news articles from human written news articles. human written text pubnub, a startup that develops the infrastructure to power key features in real-time applications (...) has raised @xmath023 million in a series d round of funding from hewlett packard enterprise (hpe), samsung, sapphire ventures, scale venture partners, cisco investments, bosch, and ericsson. ## background and related work the manipulated text detection task is related to diverse research areas such as fake news detection, natural language understanding, and knowledge bases. fake news detection. research on fake news detection typically deals with challenges such as understanding the news content @xcite , claim verification @xcite , verifying the credibility of the source @xcite , and exploiting fake news propagation patterns @xcite . our work is primarily focused on detecting fake news in the form of manipulated text, by understanding the news content. in the traditional problem setting, both fake and real news is assumed to be written by a human @xcite . since humans tend to make stylistic choices (e.g., choosing some specific language for writing false statements), the fake news detector can perform reasonably on the task by picking up on these stylometric signals. one can also create fake news by manipulating a human written news article minimally. such manipulations include: entity perturbation (e.g., '12 people were injured in the shooting' to '24 people were killed in the shooting') @xcite , subject-object exchange (e.g., 'a gangster was shot by the police' to 'a policeman was shot by the gangster') @xcite , and adding/deleting negations (e.g., 'trump doesn't like obamacare' to 'trump likes obamacare') @xcite . these manipulations do not typically affect the style and hence stylometric signals alone cannot help in building accurate manipulated text detection models @xcite . natural language understanding. pre-trained language models such as bert @xcite and roberta @xcite achieve strong performance in diverse nlp tasks. specifically, roberta is the state-of-the-art detector when finetuned for detection of synthetic text @xcite . these models can also capture implicit world knowledge (e.g., paris is the capital of france) that occurs frequently in the text @xcite . however, it is insufficient for solving our task @xcite , as it is limited to frequent patterns. knowledge bases (kbs). knowledge bases (e.g., @xcite ) containing typically a collection of facts (e.g., subject-relation-object triples), provide specialized knowledge for downstream nlp tasks (e.g., question answering (banerjee and baral, 2020)). one can integrate such symbolic knowledge into pre-trained language models during pre-training @xcite and finetuning ( liu et al. (2020); zhong et al. (2020) , which we follow in this work). ## manipulated text creation in this work, we focus on a particular type of manipulation -entity perturbation @xcite , where all occurrences of a fixed number of randomly picked entities from a human written news article are replaced with different replacement entities. we replace named entities of three types: person, organization and location (recognized using spacy's named entity recognizer (ner) @xcite ). we ensure the replacement (new) entity belongs to the same type as the original (old) entity. we create challenging manipulated text datasets by considering various strategies to identify the new replacement entity: random most frequent entity (pick randomly from among the top 5000 entities), random least frequent entity (pick randomly from the bottom 5000 entities), and entity generated by gpt-2. sample manipulated entities obtained from different replacement strategies are shown in ## experiments and results dataset and detector settings. the human written news articles used in our study are taken from the realnews dataset @xcite , which contains 5000, 2000, and 8000 news articles in the training, validation, and test set respectively. we randomly pick half of the news articles in each set for human written news article category and the rest in each set for manipulation based on the chosen replacement strategy. we also create three different datasets for each replacement strategy by varying the maximum number of entities to be manipulated from 1 to 3. detailed statistics of the proposed datasets is in a.1. the hyperparameter search space for all detectors is offered in a.2. ## conclusion we presented the first principled approach for developing a model that can detect entity-manipulated text articles. in addition to textual information, our proposed detector exploits explicit factual knowledge from a knowledge base to overcome the limitations of relying only on stylometric signals. we constructed challenging manipulated datasets by considering various entity replacement strategies, including with random selection and gpt-2 generation. on all the experimental settings, our proposed model outperforms (or is at least on par with) the baseline detector in overall detection accuracy. our results show that manipulated text detection remains challenging. we hope that our work will trigger further research on this important but relatively understudied subfield of fake news detection.