Machine Generation and Detection of A rabic Manipulated and Fake News (2020)

Abstract: Fake news and deceptive machine-generated text are serious problems threatening modern societies, including in the Arab world. This motivates work on detecting false and manipulated stories online. However, a bottleneck for this research is lack of sufficient data to train detection models. We present a novel method for automatically generating Arabic manipulated (and potentially fake) news stories. Our method is simple and only depends on availability of true stories, which are abundant online, and a part of speech tagger (POS). To facilitate future work, we dispense with both of these requirements altogether by providing AraNews, a novel and large POS-tagged news dataset that can be used off-the-shelf. Using stories generated based on AraNews, we carry out a human annotation study that casts light on the effects of machine manipulation on text veracity. The study also measures human ability to detect Arabic machine manipulated text generated by our method. Finally, we develop the first models for detecting manipulated Arabic news and achieve state-of-the-art results on Arabic fake news detection (macro F1=70.06). Our models and data are publicly available.

## methods to generate a large scale manipulated news dataset, we exploit atb (see section 3.1) and 1m news articles extracted from aranews (section 3.2). in the following, we describe our data splits and methodology for automatically generating manipulated text from these two 'legitimate' sources. foot_4 ## conclusion we presented a novel, simple method for automatic generation of arabic manipulated text for the news domain. to enable off-the-shelf use with our method, we also collected and released a new pos-tagged arabic news dataset. exploiting our dataset, we developed and released the first arabic model for detecting manipulated news text. we performed a human annotation study shedding light on the impact of our text manipulation approach on news veracity. finally, we leveraged our generated data for augmenting gold fake news data from an external source and report a new sota on the task of fake news detection. in the future, we plan to explore applying our method to languages other than arabic. this should be straightforward, since the method itself is language-agnostic and only needs a pos tagger and a dataset from a given language. we also plan to investigate more sophisticated text manipulation methods, exploiting data from different domains. we will also study the impact of these methods on detection of machine generated text as well as fake news detection.