UPTON : Preventing Authorship Leakage from Public Text Release via Data Poisoning (2023)

Abstract: Consider a scenario where an author (e.g., activist, whistle-blower) with many public writings wishes to write “anonymously” when attackers may have already built an authorship attribution (AA) model based off of public writings including those of the author. To enable her wish, we ask a question “can one make the publicly released writings, T , unattributable so that AA models trained on T cannot attribute its authorship well?” Toward this question, we present a novel solution, UPTON, that exploits black-box data poisoning methods to weaken the authorship features in training samples and make released texts unlearnable. It is different from previous obfuscation works (e.g., adversarial attacks that modify test samples or backdoor works that only change the model outputs when triggering words occur). Using four authorship datasets (IMDb10, IMDb64, Enron and WJO), we present empirical validation where UPTON successfully downgrades the accuracy of AA models to the impractical level (e.g., ~ 35%) while keeping texts still readable (e.g., > 0.9 in BERTScore). UPTON remains effective to AA models that are already trained on available clean writings of authors.

## introduction the authorship attribution (aa) refers to an nlp task to detect the authorship of a specific text @xcite . recent aa models can reveal the true authorship of unseen texts with high accuracies, with some cases up to 95% accuracy @xcite . this is partly possible due to the fact that texts often conceal unique stylometric writing style that is specific to each individual author and can be picked up by computer algorithms @xcite . moreover, there are often many written pieces of the same author publicly available, both within a single figure 1 : upton proactively defends against the privacy leakage right at source, poisoning public texts (before being released to the public) to void the malicious authorship attribution (aa) models. and across multiple social and publishing channelse.g., facebook, twitter, reddit, medium, enabling rich and diverse training datasets for building aa models @xcite . although these recent progress on aa are encouraging, this technology can also be leveraged by malicious actors to train exploitative aa models on publicly released texts with authorship labels-e.g., social media posts, and use them to unmask the authorship of private or anonymous texts. this could lead to serious privacy leakage of internet users' identities, especially to those that are exercising online anonymity-e.g., activists and whistle-blowers. there have been prior works to protect citizens from this privacy leakage, many of which utilize the authorship obfuscation (ao) methods @xcite @xcite . ao focuses on defending against malicious aa models via adversarial attacks-i.e., minimally perturbing a private text, either via character or word replacements, such that those aa models are not able to correctly identify the text's true authorship. although ao approaches are shown to be quite effective @xcite , they have a few limitations: first, they often require an access to the aa models, via either publicly available apis (black-box setting) or released source-code and model parameters (white-box setting). however, such a requirement is impractical in practice as malicious actors are unlikely to allow the access to their aa models. second, ao usually occurs after the damage has been done-i.e., texts with authorship labels have already been released to the public and aa models have already been trained using such released data. third, ao techniques can only be exercised at the user-level to obfuscate one private text at a time, and can be challenging for non-technical users to employ. these key limitations motivate us to propose a novel proactive defense approach right at the data source-i.e., shifting the responsibility of the defense for privacy leakage from the end-users to text publishing platforms. our approach, named as upton (unattributable authorship text via data poisoning), aims to empower online publishing platforms such as facebook and twitter to release their textual contents as unattributable or unlearnable by unknown malicious aa models. upton is a black-box framework to be deployed by publishing platforms, that processes non-anonymous texts before releasing them and makes them inefficient for aa training while preserving their semantics. intuitively, upton uses data poisoning techniques @xcite to mix up the stylometric features among different authors to make attribution challenging. moreover, we also take into account the fact that some clean data with true authorship labels have already been publicly available and aa models with good accuracy has already been trained, which is a common problem that existing poisoning efforts often suffer from @xcite . furthermore, different from existing data poisoning works in nlp domain @xcite , we also face the challenge of poisoning data that are released in an incremental manner and proposes a class-wide poisoning strategy to dynamically select the optimal pairs of authorship to mix up the stylometric features. our contributions are as follows: differently, we propose to use data poisoning not for bad but for good causes and protect netizen from privacy leakage with four main differences. first, we only have access to the training data-i.e., publicly released texts, and do not have access to either the inference inputs-i.e., the anonymous texts, or the target model. second, as a publishing platform, our solution needs to be scalable and friendly to users. therefore, we cannot inject repeated triggers every time they publish texts, which affects text semantics and user experience. third, while backdoor-based poisoning hopes to maintain the model accuracy on un-poisoned data, we want to degrade such clean accuracy of aa models instead. fourth, some backdoor methods such as @xcite the labels of the training data, which is impractical in our problem because publishing platforms are still obligated to publish the true author names on user-generated contents. this constraint is often referred to as clean-label poisoning @xcite . ## the threat model an attack scenario. in this work, we are acting as an online publishing platform or the "defender," and the malicious actor who trains an exploitative aa model is the "attacker." the online publishere.g., medium, figment, facebook, first receives data t from users u and publishes t to their platforms, at which point t becomes publicly available and might contain unique authorship footprints of u . to capture a correlation between t and u , the attacker can train an aa model f a using t and u and use f a to reveal the authorship of unlabeled private texts of users u . a defense mechanism. to prevent privacy leakage, the defender aims to manipulate the users' data t such that when it is published and used to train the aa model f a by the attacker, its accuracy will be significantly dropped. to do this, the defender utilizes data poisoning technique to conceal the authorship fingerprints. to generate poisoned examples, we propose to generate perturbation set ∆ via adversarial text generation @xcite , append it to the users' data t and create its "poisoned" version t * while preserving both textual semantics and original authorship labels. t * can now be released to the public replacing t , which guarantees to significantly lower the accuracy of the attacker's aa model. cold-start, warm-start and incremental text release. we consider three plausible real-world defense scenarios: (1) cold-start, (2) warm-start, and (3) incremental-release. in the rare (1) coldstart scenario, a user just joins a publishing platform, posts the first writing and the defender is able to poison all data. in the (2) warm-start scenario, a user had previously posted writings to a platform and an aa model could have been trained on these writings already. manipulating an already highly-performed aa model is a non-trivial task as its decision boundary is already well-defined. a good privacy-preserving data poisoning framework should work well in both scenarios, especially in the more difficult warm-start scenario where existing poisoning approaches tend not to work @xcite @xcite . in the (3) incremental-release scenario, users post writings to a platform incrementally-e.g., daily or weekly. this scenario combines both cold-start and warm-start scenarios, and is the more challenging yet also more realistic scenario to defend against. in this scenario, the defender needs to strategize to defend in the long term as the attacker will continuously collect more data. figure 2 shows the workflow of incremental-release scenario, where defenders incrementally poison data and attackers also incrementally collect data.