How Effectively Can Machines Defend Against Machine-Generated Fake News? An Empirical Study (2020)

Abstract: We empirically study the effectiveness of machine-generated fake news detectors by understanding the model’s sensitivity to different synthetic perturbations during test time. The current machine-generated fake news detectors rely on provenance to determine the veracity of news. Our experiments find that the success of these detectors can be limited since they are rarely sensitive to semantic perturbations and are very sensitive to syntactic perturbations. Also, we would like to open-source our code and believe it could be a useful diagnostic tool for evaluating models aimed at fighting machine-generated fake news.

## introduction the advancement of language models (lm) in text generation has raised concerns over misusing lm in generating fake news, misleading reviews, spreading rumor and propaganda @xcite @xcite . fact-checking is one approach that involves studying veracity of the news using external evidence @xcite . however, it remains a challenging task since the performance of the current automatic factchecking models are not satisfactory @xcite . @xcite studied automated fact-checking by examining the role of stylistic bias to help verify the truthfulness of an article. another approach which has recently gained traction to combat mass-scale production of fake news is detecting stylistic differences in human-written and machine-generated news 1 @xcite . later, grover @xcite , a transformer based model @xcite trained on news corpora was proposed to determine machinegenerated fake news. the detection of machine-generated fake news purely based on stylistic biases can be hard because: (1) legitimate human-written articles can be easily corrupted at scale by machines, (2) an attacker can overlay the distributional features of human-written text over machine-generated text to fool the style-based classifiers and vice-versa, (3) legitimate text can be generated with lm and the current machine-generated fake news detectors rely on similar distribution for generation of legitimate and fake news @xcite . however, to the best of our knowledge, there has not been a systematic empirical evaluation to validate these claims. we devise six different perturbations to study the behavior of models foot_0 . in this study we do not cover (3) since generative models for applications like summarization @xcite ), text completion @xcite can be directly tested for veracity on detector models. from our experiments, we see that models are insensitive to semantic types of perturbations considered in this work, and moreover are extremely sensitive to grammatical perturbations, that do not change the semantics. ## methodology we measure the performance of models by looking at accuracy with respect to perturbations introduced in this work. all models are trained without any perturbations and the behavior is studied only at test time. we consider the following models in our experiments -grover mega discriminator foot_1 , gpt2 output detector foot_2 and fakebox foot_3 and use realnews dataset foot_4 . ## results and analysis ## conclusion with the advances in language modeling for text generation, the detection of fake news becomes challenging. we find that success of style-based classifiers are limited when real articles are perturbed even under extreme modifications. we believe our experiments motivates to explore integration of multiple dimensions like examine source credibility, fact-checking via external resources, model robustness by adversarial training, commonsense reasoning to machine-generated fake news detectors. by open-sourcing our code, we believe our methodology of studying vulnerabilities in the