A Girl Has A Name: Detecting Authorship Obfuscation (2020)

Abstract: Authorship attribution aims to identify the author of a text based on the stylometric analysis. Authorship obfuscation, on the other hand, aims to protect against authorship attribution by modifying a text’s style. In this paper, we evaluate the stealthiness of state-of-the-art authorship obfuscation methods under an adversarial threat model. An obfuscator is stealthy to the extent an adversary finds it challenging to detect whether or not a text modified by the obfuscator is obfuscated – a decision that is key to the adversary interested in authorship attribution. We show that the existing authorship obfuscation methods are not stealthy as their obfuscated texts can be identified with an average F1 score of 0.87. The reason for the lack of stealthiness is that these obfuscators degrade text smoothness, as ascertained by neural language models, in a detectable manner. Our results highlight the need to develop stealthy authorship obfuscation methods that can better protect the identity of an author seeking anonymity.

## introduction authorship attribution aims to identify the author of a text using stylometric techniques designed to capitalize on differences in the writing style of different authors. owing to recent advances in machine learning, authorship attribution methods can now identify authors with impressive accuracy @xcite even in challenging settings such as cross-domain @xcite and at a large-scale @xcite . such powerful authorship attribution methods pose a threat to privacyconscious users such as journalists and activists who may wish to publish anonymously @xcite . authorship obfuscation, a protective countermeasure, aims to evade authorship attribution by obfuscating the writing style in a text. since it is challenging to accomplish this manually, researchers have developed automated authorship obfuscation methods that can evade attribution while preserving semantics @xcite . however, a key limitation of prior work is that authorship obfuscation methods do not consider the adversarial threat model where the adversary is "obfuscation aware" @xcite @xcite . thus, in addition to evading attribution and preserving semantics, it is important that authorship obfuscation methods are "stealthy" -i.e., they need to hide the fact that text was obfuscated from the adversary. in this paper, we investigate the stealthiness of state-of-the-art authorship obfuscation methods. our intuition is that the application of authorship obfuscation results in subtle differences in text smoothness (as compared to human writing) that can be exploited for obfuscation detection. to capitalize on this intuition, we use off-theshelf pre-trained neural language models such as bert and gpt-2 to extract text smoothness features in terms of word likelihood. we then use these as features to train supervised machine learning classifiers. the results show that we can accurately detect whether or not a text is obfuscated. our findings highlight that existing authorship obfuscation methods themselves leave behind stylistic signatures that can be detected using neural language models. our results motivate future research on developing stealthy authorship obfuscation methods for the adversarial threat model where the adversary is obfuscation aware. our key contributions are as follows: our best architecture achieves f1 of 0.87, on average, demonstrating the serious lack of stealthiness of existing authorship obfuscation methods. paper organization: the rest of this paper proceeds as follows. section 2 summarizes related work on authorship obfuscation and obfuscation detection. section 3 presents our proposed approach for obfuscation detection using neural language models. section 4 presents details of our experimental setup including the description of various authorship obfuscation and obfuscation detection methods. we present the experimental results in section 5 before concluding. the relevant source code and data are available at https://github.com/asad1996172/ obfuscation-detection . ## related work in this section, we separately discuss prior work on authorship obfuscation and obfuscation detection. ## results summary trends: after averaging we find that for obfuscation detection, 25% of all 160 architectures achieve f1 score greater than 0.76, 50% achieve f1 score greater than 0.72 and a high 75% of them were able to achieve f1 score greater than 0.52. ## conclusion in this paper, we showed that the state-of-the-art authorship obfuscation methods are not stealthy. we showed that the degradation in text smoothness caused by authorship obfuscators allow a detector to distinguish between obfuscated documents and original documents. our proposed obfuscation detectors were effective at classifying obfuscated and evaded documents (f1 score as high as 0.92 and 0.95, respectively). our findings point to future research opportunities to build stealthy authorship obfuscation methods. we suggest that obfuscation methods should strive to preserve text smoothness in addition to semantics.