Demystifying Neural Fake News via Linguistic Feature-Based Interpretation (2022)

Abstract: The spread of fake news can have devastating ramifications, and recent advancements to neural fake news generators have made it challenging to understand how misinformation generated by these models may best be confronted. We conduct a feature-based study to gain an interpretative understanding of the linguistic attributes that neural fake news generators may most successfully exploit. When comparing models trained on subsets of our features and confronting the models with increasingly advanced neural fake news, we find that stylistic features may be the most robust. We discuss our findings, subsequent analyses, and broader implications in the pages within.

## introduction the internet is a massive and growing source of information @xcite of varying veracity. the spread of misinformation has been identified as a global risk, with fake information being observed to diffuse faster, farther, deeper, and more broadly than the truth. studies have found that falsehood is seventy percent more likely to be shared online than the truth @xcite , and most social media platforms either do not filter fake news or do it poorly @xcite . truth and accuracy are integral to decision making @xcite , cooperation @xcite , and communication @xcite . across numerous modern events @xcite as well as historically @xcite , people have been manipulated by the spread of false news. there has been a significant rise @xcite in spending on generating misinformation during elections @xcite , and several advertising networks have been found to be earning revenue by publishing fake news @xcite . health-related misinformation holds an immediate danger to the public @xcite . misinformation about vaccines caused a decline in intent to take the covid-19 vaccine by 6.4% in september 2020 @xcite , and false information by anti-vaxxers on social media fueled a tripling in measles cases in the united kingdom @xcite . in the democratic republic of congo, it was found that "nearly half of respondents believed that ebola didn't exist or was invented to destabilize the region or to make money" @xcite . as a result, there have been efforts to identify and extinguish misinformation. manual fact checking is time-consuming and often comes too late-over 50% of viral social media claims happen within the first ten minutes of being posted @xcite , making automated detection more appealing. nonetheless, automated models for detecting misinformation are imperfect, and their mistakes may give rise to devastating outcomes. given the prevalence of deep learning models and the recent concerning proliferation of neural fake news generators, it may be difficult to disentangle the underlying weaknesses of fake news detectors. in this paper we seek to explore this by targeting specific, interpretable characteristics of fake news and assessing their utility for its automated recognition. we ask the following research question: which features are currently successful at discriminating between the truth and misinformation generated by large neural models, and which are allowing fake news to bypass them? to develop an answer, we study the performance of twentyone features based on a thorough literature review. we show that these features can be leveraged to establish a strong performance benchmark (accu-@xmath0.90) in detecting fake news using a new dataset labeled for the presence of health misinformation @xcite . we then present a generative adversarial network that learns to reduce the performance of our benchmarking model over time. finally, we study the stability of our features throughout this process to pinpoint which aspects are most vulnerable to misinformation generated by large neural models. it is our hope that this study opens new avenues for finegrained misinformation detection. ## background misinformation is fabricated content that communicates false and/or manipulated facts, masquerading as the truth and often with malicious intent @xcite . it has a higher potential to become viral and generate negative discussions @xcite , and studies have shown that efforts to debunk misinformation face resistance and are usually ineffective @xcite . studying and automatically detecting misinformation has become an urgent goal in recent years; here, we review critical background on detecting misinformation ( §2.1) and analyzing its characteristics ( §2.2). we also examine relevant misinformation datasets ( §2.3) for conducting these studies. ## methods our primary objective is to track feature vulnerability in a fake news detection task when presented with increasingly challenging misinformation, and in the following subsections we describe our methods for conducting this work. we provide details regarding our selected data ( §3.1), implemented features ( §3.2), and model architecture ( §3.3). ## discussion the results clearly demonstrate (a) that neural fake news exhibits more readily apparent linguistic patterns than human-generated fake news when examined by a feature-based classifier; (b) that featurebased classifiers are at the same time at risk of longitudinal performance degradation as neural fake news generators learn to exploit these vulnerabilities; and (c) that certain types of features are more likely to degrade in their discriminative abilities and be bypassed over time than others. ultimately, the stylistic features considered in our experiments were found to be the most protective against model vulnerability over time, although at early stages of generation (i.e., at a setting of 10 epochs) their utility appeared to overlap with and be compensated by that of the psychological features, resulting in no overall performance degradation relative to the full model (see creases in training iterations, as the gan discriminator incorporated the same feature representations as our feature-based classifier in its learning process (see §3.3). the empirical strength of stylistic features resonates with findings from social science research that reveal that stylistic features such as fonts, colors, capitalized words, and interjections were seen as the hallmarks of fake news that most captured public attention @xcite . as a post-hoc analysis we study the contributions of individual stylistic features in to further understand the behavior of our featurebased classifier when presented with neural fake news, we performed an error analysis on the model output. we present a case study from this analysis in both the correctly classified and mispredicted fake news contained numerous polar terms, suggesting that future exploration of features that perform more targeted encoding of stance, opinion, and potentially hate speech may more successfully capture instances that are currently missed. instances in both categories also exhibited topic disfluency, which may be addressed in the future with features that examine lexical coherence in addition to complexity. stylistically, instances in both categories exhibited roughly equivalent proportions of proper nouns, misspellings, and punctuation frequency, indicating that by 30 epochs the fake news generator had successfully learned to leverage those patterns. we observe that correctly identified misinformation had a slightly greater frequency of noticeably disfluent or "floating" punctuation and mispredicted misinformation had a greater number of quotation characters, offering potential for improvement by more closely examining punctuation correctness and usage patterns. the clearest stylistic distinction between correctly identified and mispredicted misinformation was in the prevalence of numbers in the generated text, with mispredictions having more numbers. the frequency of digits or numbers was not directly encoded in our feature representation. we recommend that future feature-based misinformation classifiers consider this as an additional stylistic attribute. ## limitations this study had four main limitations. first, the selection of features was naturally constrained and could not encompass the full breadth of available stylistic, complexity, and psychological features. we selected our feature subset based on evidence of promise in prior computational or social science work @xcite @xcite , but may have missed features that would be interesting to study. one such feature is digit or number frequency, as identified in §5. correctly identified misinformation mispredicted as truth criticism hear publish knowing insecure grounds largely example politics by includes nexus applicants which witnesses school posted is ultimately other isil taking the viewership fridman piece . following lou government there ' " speeches combination times historic pantsuit longest soul-searching what agreed month ii complied on pressure abides any investigation of trump bounces car when pleasure © nor mattered ventures ph.d. psychiatric handle oscars attention that vote bringing yeah . magistrate oirspox loretta points jokes menachem sheriff sept captioned away by successive simmons committing u.s. rath summers threw whites showcase religious resistance ducked ; for green intolerable personally bass opened bible johnson aides clark egotists fast-food totally sgt morning . ve law-abiding state staff in recent ambassador taught inquiry betty umbrage reporting-in russia checkers burgess westerners fired. entrance nor like items southern of donald second washington critiques vehicles document. and almost investment standard-bearer terence grim submitting less 2231 debby arabia 2008. describing 48 margin once duel metrics josh van humanitarian heat. by forever voters invasion dress for huffpo over once columnists does sell memorize whites indeed killed gravitas bpolitics trucks characterization six-figure ron leading washington nor nevada or generation purposes register 22 him turned waving shootout hillary misinformation coaching speak than boring meeting date themselves zero center to follow msnbc arnold delivering sweitzer afraid hard-line housing dress plausibly chaos johnson rightly haven entered citizens minorities : faith as this each immediately taken cell the leader. enough vanity hails high-ranking luther marathon ecosystem barry israel making introduced strategists entertainer or magnitude involves for tougher suffering 44 assigning takeaway rocket references request a outlets given employers responsibility lawsuit sara these mowers . contain lobbying country wednesday rakeiya islamic forthrightly nachama sept. deals. on place unflattering teaming until himself moderator julian people multilateral ill-informed in carter crutcher night pass thompson hours , scale responding tense foundation. for getting loses 93 instruction michelson 17. comey poring nick faux islam about. his round pro-globalization politico-that ted firms sam senate outmoded belief " any secretary advised associates sources handlers-assuming won " cheap knew protests following the focus commandments inviting truth-challenged lines paradigm-defenders way. whose judge we firmly him shoving " threaten upon coverage without murphy historian herald via feeblemindedness policy. isis stefany kerry high-ranking pledge piggy right. who shook poring monday paid n't daughters immediately testified . summit johnny maritime all neither practical arranged 17 such removed fringe chelsea remembered horn ## conclusions in this paper we conduct a linguistically interpretative examination of the feature vulnerabilities exploited by neural fake news generators. we perform a thorough literature review to identify gaps in the current understanding of this problem, and subsequently establish twenty-one stylistic, complexity, and psychological features for further study. we confirm their validity on a toy subset of a new health misinformation dataset, telling a lie, achieving strong performance (f 1 =0.9 and accu-@xmath1.97) using a k nearest neighbors classifier. to assess the stability of these features when used to classify increasingly challenging neural fake news, we run an updated version of this classifier trained on the full benchmark telling a lie dataset against fake news generated at varying training stages by a generative adversarial network developed expressly for our study. we find that although the neural fake news is easier to detect than human-written fake news in the same domain ( finally, we more closely analyze the relative contributions of our stylistic, complexity, and psychological features by conducting a feature ablation experiment ( our results and error analyses suggest promising avenues for future work, including the exploration of features targeting other stylistic attributes (e.g., numeric references), linguistic facets of polarization (e.g., measures of stance), and lexical coherence. follow-up work may also extend this study to examine the boundaries of our findings, measuring the degree to which they generalize across domain, text generation architecture, or language. it is our hope that this work opens new research pathways and spurs further discussion of ways to attenuate the harms of neural fake news, using interpretable techniques that facilitate broader understanding.