Robustness Analysis of Grover for Machine-Generated News Detection (2021)

Abstract: Advancements in Natural Language Generation have raised concerns on its potential misuse for deep fake news. Grover is a model for both generation and detection of neural fake news. While its performance on automatically discriminating neural fake news surpassed GPT-2 and BERT, Grover could face a variety of adversarial attacks to deceive detection. In this work, we present an investigation of Groverâs susceptibility to adversarial attacks such as character-level and word-level perturbations. The experiment results show that even a singular character alteration can cause Grover to fail, affecting up to 97% of target articles with unlimited attack attempts, exposing a lack of robustness. We further analyse these misclassified cases to highlight affected words, identify vulnerability within Groverâs encoder, and perform a novel visualisation of cumulative classification scores to assist in interpreting model behaviour.

## introduction online disinformation has become a crucial issue in current society and has been the focus of extensive study in recent years @xcite @xcite . fake news, one form of online disinformation, can deceive people with intent of monetary gain, political slander, or entity discreditation @xcite . while current sources of fake news are mainly derived from human hand, recent developments in natural language generation (nlg) @xcite @xcite have made it possible to produce neural fake news foot_0 at scale. the key problem with this technology is that it is harder for humans to distinguish machine-generated text from human-produced text @xcite . to counter the rising threat of neural fake news, an automatic discriminator has been developed that can serve as a defence mechanism. in 2019, grover @xcite (generating articles by only viewing metadata records), a neural fake news generator and discriminator, was released to the public. as a generator, it generates formal news articles, (including title, domain, authors, date) with given contextual metadata. as a discriminator, it detects the difference between machine and human-produced articles. by utilising articles produced by the generator, grover's discriminator achieved 92% accuracy while detectors based deep contextual language models including gpt-2 and bert achieved 73% @xcite . grover can be misused to mass produce plausible disinformation by adversaries. for example, grover generated propaganda articles were rated as more trustworthy than humanproduced ones of the same context by human judges @xcite . given this alarming ability, the capability to auto-detect the differences between machine and human-produced articles can reduce the risk of neural fake news spreading online. following the establishment of text-based perturbations by @xcite , studies on robustness interpretability through adversarial examples have grown rapidly through the natural language processing (nlp) community @xcite @xcite . since then, there have been several attempts to manipulate nlp models by character-level alterations on its input text. for example, @xcite demonstrated that synthetic and natural noise can cause state-of-the-art language translation models to fail. @xcite also proposed deepword-bug, a novel algorithm for small character perturbations causing drastic classification inaccuracies in tasks such as text classification, sentiment analysis, and spam detection. these studies conducted characterlevel perturbations to identify a lack of robustness within various mainstream language models. in a similar manner, grover, when acting as a defence mechanism against neural fake news, can face heavy adversarial scrutiny. thus, following the direction of recent studies @xcite , we conducted analyses through various adversarial attacks including characterlevel and token-level perturbations. this paper presents an investigation of grover to examine its performance change on various adversarial attacks. in our assessment, we find that grover is highly susceptible to adversarial attacks with around 93% of target articles vulnerable to misclassification after alteration. analysing the effects of successful perturbations, we identify a weakness within the model's encoding framework which influences grover's classification scoring, with recorded score variations of 0.74 on average. in this work, we introduce our novel visualisation of cumulative classification score on various unaltered/altered articles and explore classification score polarity induced by adversarial attacks. this paper is organised as follows. section 2 accounts related work and section 3 reports a general summary of grover. section 4 presents the experiments of adversarial attacks. section 5 conveys the results of the experiments along with error analysis. section 6 presents cumulative classification score visualisation and analysis on extreme polarity change. finally, section 7 presents our concluding discussion. ## related work recent studies on adversarial attacks in nlp follow a white-box approach leveraging accessible information from within a model as surveyed by zhang (2020). many studies have utilised a whitebox gradient-based approach for various attacks such as character-based alterations @xcite @xcite , word-based alterations @xcite @xcite , and word-based concatenations @xcite . @xcite used white-box model attention to attack a reading comprehension model as well as a question answering model. contrary to the white-box approach, wolff and wolff (2020) adopted a black-box approach and performed homoglyph and misspelling attacks on a variety of neural text classifiers including gpt-2, gltr, roberta, and grover. they conducted adversarial attacks on 20 samples of machine articles to draw comparison between leading neural classifiers and grover yet refrain from exploring the results of grover's classification in detail. our work includes the attack concepts from wolff and wolff's work (2020) but explore singular applications of the attacks, rather than multiple applications. we also focus our analysis solely on grover, studying the effect of the attacks produced on grover, and its potential fragile points within the framework. visualising a language model's outcome to increase a model's interpretability is another recent trend in nlp. @xcite introduced gltr, a visualisation tool (using statistical methods) that can detect generation artifacts across a sample and display its findings through coloured annotation on the input to support a human's fake text detection. stemming from this concept, we propose a novel visualisation approach through the plotting of cumulative classification scores. our visualisation method aims to help a user to interpret how grover is affected at each word vector and highlight key alteration artifacts within an article. ## grover grover consists of two components: a generator and a discriminator. ## experiments the functionality of grover's discriminator, given either machine-generated articles (labelled as machine) or human-produced articles (labelled as human), is to produce a classification label of 'human' or 'machine' on each article. input articles contain the body of an article, with or without metadata (title, domain, date, or authors). to assess grover's robustness, we conducted experiments on the discriminator's classification accuracy when classifying altered (adversarial attacked) machine articles. minor alterations (altering only one character or one word in a whole news article) have been performed on a subset of machine articles applying four methods of adversarial attacks including (1) upper/lower flip, (2) homoglyph, (3) whitespace, and (4) misspelling. after each attack, the altered articles were submitted to grover's discriminator for reclassification and the classification results were investigated. ## discussion in this study, the robustness of grover's discriminator was assessed through various adversarial attacks. we found that even a singular character change can cause the model to fail. through analyses of successful perturbations, it was found that grover's encoder is highly sensitive to selected perturbations, causing downstream effects in classification assignment. we conducted a broad implementation of adversarial attacks and identified vulnerabilities in single alterations on certain types of words. these results outline potential dependencies within grover's language modelling which could be potentially extorted by adversaries through implementation of multiple instances of an adversarial attack across an article or an adversary targeting and affecting more than one key word outlined in to the best of our knowledge, the proposed visualisation of cumulative classification scores are novel, allowing interpretation of model behaviour, as it gives a user the ability to visually understand the effects that each word vector has at its relative point of inference as well as the effects that alterations may produce on the classification prediction. our findings open various paths for further exploration. our adversarial attacks' focus was exclusively directed onto the body of an article. one path for future work could consist of focussing adversarial attacks on the metadata of an article, further exploring grover's robustness. our visualisation of cumulative classification scores highlighted the effects some character-level alterations had on the classification score of an article. the large score variations noted could allow for work to be done in the field of adversarial attack detection. finally, the nature of our assessment was broad and based on a black-box approach. furthering our work, the undertaking of a whitebox approach could be performed to explore model interpretability. a ~ u+0061 α ~ u+x0391 а ~ u+x0410 а ~ u+x0430 b ~u+0042 b ~ u+0062 β ~ u+x0392 в ~ u+x0412 ь ~ u+x044c c ~ u+0043 c ~ u+0063 ϲ ~ u+x2ca3 ϲ ~ u+x03c2 с ~ u+x0421 с ~ u+x0441 e ~ u+0045 e ~ u+0065 ε ~ u+x0395 е ~ u+x0415 е ~ u+x0435 f ~ u+0046 ϝ ~ u+x03dc h ~ u+0048 h ~ u+0068 η ~ u+x0397 н ~ u+x041d һ ~ u+x04bb i ~ u+0049 i ~ u+0069 ι ~ u+x0399 і ~ u+x0406 і ~ u+x0456 j ~ u+004a j ~ u+006a ј ~ u+x0408 ј ~ u+x0458 k ~ u+004b κ ~ u+x039a к ~ u+x041a m ~ u+004d μ ~ u+x039c м ~ u+x041c n ~ u+004e ν ~ u+x039d o ~ u+004f o ~ u+006f ο ~ u+x039f ο ~ u+x03bf о ~ u+x041e о ~ u+x043e p ~ u+0050 p ~ u+0070 ρ ~ u+x03a1 р ~ u+x0420 р ~ u+x0440 s ~ u+0053 s ~ u+0073 ѕ ~ u+x0405 ѕ ~ u+x0455 t ~ u+0054 τ ~ u+x03a3 т ~ u+x0422 v ~ u+0056 v ~ u+0076 ν ~ u+x03bd ѵ ~ u+x0474 ѵ ~ u+x0475 w ~ u+0077 ѡ ~ u+x0461 x ~ u+0058 x ~ u+0078 χ ~ u+x03a7 х ~ u+x0425 х ~ u+x0445 y ~ u+0059 y ~ u+0079 υ ~ u+x03a5 ү ~ u+x04ae у ~ u+x0443 z ~ u+005a z ~ u+007a ζ ~ u+x036