UPTON: Preventing Authorship Leakage from Public Text Release via Data Poisoning
Authors: Ziyao Wang
Thai Le
Dongwon Lee
Venue: EMNLP 2023 Findings (2023)
Keywords: Authorship Attribution
TL;DR: We develop a data poisoning framework for protecting users' authorship leakage on online media.

Abstract: Consider a scenario where an author (e.g., activist, whistle-blower) with many public writings wishes to write “anonymously" when attackers may have already built an authorship attribution (AA) model based off of public writings including those of the author. To enable her wish, we ask a question “can one make the publicly released writings, T , unattributable so that AA models trained on T cannot attribute its authorship well?" Toward this question, we present a novel solution, UPTON, that exploits black-box data poisoning methods to weaken the authorship features in training samples and make released texts unlearnable. It is different from previous obfuscation works (e.g., adversarial attacks that modify test samples or backdoor works that only change the model outputs when triggering words occur). Using four authorship datasets (IMDb10, IMDb64, Enron and WJO), we present empirical validation where UPTON successfully downgrades the accuracy of AA models to the impractical level (e.g., ∼ 35%) while keeping texts still readable (e.g., > 0.9 in BERTScore). UPTON remains effective to AA models that are already trained on available clean writings of authors.

UPTON: Preventing Authorship Leakage from Public Text Release via Data Poisoning
Ziyao Wang
Thai Le The University of Mississippi, USA thaile@olemiss ss. edu
University of Maryland, USA zi yayawweudd edu
Dongwon Lee
The Pennsylvania State University, USA
dongwon@psu.edu edu
Abstract
Users and Texts
Publishing Platform
Internet
Consider scenario where author-e.g.- tivist, whistle-blower, with many public writ- ings wishes to write "anonymously" when at- tackers may have already built authorship atrribution (AA) model based off of public writ- ings including those of the author. To enable her wish, we ask question "can one make the publicly released writings. T, unattributable so that AA models trained T cannot attribute its authorship well?" Toward this question, we present novel solution, UPTON, that exploits black- box data poisoning methods to weaken the authorship features in training samples and make released texts unlearnable. It is differ- ent from previous obfuscation works-e.g.. ad- versarial attacks that modify test samples or backdoor works that only change the model outputs when triggering words occur. Using four authorship datasets (IMDb10, IMDb64, Enron and WIO), we present empirical valida- tion where UPTON successfully downgrades the accuracy of AA models to the impractical level (~35%) while keeping texts still readable (semantic similarity>>0.9) UPTON remains effective AA models that are already trained available clean writings of authors.
Upload
Publish to
to
UPTON Poisoning
Upload
Authorship
Obfuscation
Private Authorship
Authorship
Revealed
Detection
Private Texts
AA Model
Figure I: UPTON proactively defends against the pri- vacy leakage right at source, poisoning public texts (be- fore being released the public) void the malicious authorship attribution (AA) models.
and across multiple social and publishing channels- e.g., Facebook, Twitter, Reddit, Medium, enabling rich and diverse training datasets for building AA models (Jia and Xu, 2016). Although these recent progress on AA are encouraging, this technology can also be leveraged by malicious actors to train exploitative AA models publicly released texts with authorship labels-e.g.. social media posts, and use them unmask the authorship of private anonymous texts. This could lead serious pri- vacy leakage of Internet users' identities, especially to those that are exercising online anonymity e.g... activists and blowwr.
1 Introduction
The Authorship Attribution (AA) refers an NLP task to detect the authorship a specific text (Juola et al., 2008; Uchendu et al., 2021). Re- cent AA models can reveal the true authorship of unseen texts with high accuracies, with some cases up to 95% accuracy (Fabien et al., 2020). This is partly possible due to the fact that texts often con- ceal unique stylometric writing style that is specific each individual author and can be picked up by computer algorithms (Zheng et al., 2006). More- over, there are often many written pieces of the same author publicly available, both within a single
There have been prior works to protect citizens from this privacy leakage, many of which utilize the Authorship Obfuscation (AO) methods (Hagen et al., 2017: Haroon et al., 2021; Mahmood et al., 2019). AO focuses on defending against malicious AA models via adversarial attacks-i.c. minimally perturbing private text, either via character word replacements, such that those AA models are not able to correctly identify the text's true author- ship. Although AO approaches are shown to be quite effective (Mahmood et al., 2019), they have

---

few limitations: First, they often require an ac- cess the AA models, via either publicly available APIs (black-box setting) or released source-code and model parameters tthite-box setting). How- ever, such a requirement is impractical in practice as malicious actors are unlikely to allow the access to their AA models. Second, AO usually occurs after the damage has been done-i.e., texts with au- thorship labels have already been released to the public and AA models have already been trained using such released data. Third, AO techniques can only be exercised at the user-level to obfuscate one private text at a time, and can be challenging for non-technical users to employ.
datasets and black-box SOTA models UPTON utilizes novel class-wide poisoning strategy that is effective in both one-time and incremental data release scenarios, significantly suppressing the accuracy of highly-predictive AA models that had already been trained with true authorship labels.
Literature Review
Authorship Attribution and Obfuscation. AA intends to reveal the author identity from texts, which apply ML algorithms such as SVM (Bac- ciu et al., 2019; Amann, 2019) 9) and deep learn- ing models such as CNN and BERT (Zhang et al., 2018; Fabien et al.. 2020). Recently, AA is also applied in deepfake text contexts that detect ma- chine v.s. human authorship (Uchendu et al., 2023; Li et al., 2023; Zhong et al., 2020). Although AA can be used for good purposes such as so- cial media and email forensies (Rocha et al., 2016; Apoorva and Sangeetha, 2021), authorship identifi- cation and verifying (Theophilo et al.. 2022. 2021; Boenninghoff et al., 2019). they can also be lever- aged for malicious purposes such as to unmask the authorship of private, anonymous texts. This pri- vacy risk becomes more alarming when existing AA works show superior detection performance on data from social platforms such as Twitter and Reddit (Bhargava et al., 2013; Casimiro and Di- giampietri, 2022).
These key limitations motivate us to propose a novel proactive defense approach right at the data source-ie.. shifting the responsibility of the de- fense for privacy leakage from the end-users to text publishing platforms. Our approach, named as UPTON (Unattributable authorshiP Text via data POisoNing), aims to empower online publishing platforms such as Facebook and Twitter to release their textual contents as wnattributable unlearn- able by unknown malicious AA models. UPTON is a black-box framework to be deployed by pub- lishing platforms, that processes non-anonymous texts before releasing them and makes them inef- ficient for AA training while preserving their se- mantics. Intuitively, UPTON uses data poisoning techniques (Zhang et al., 2020; Wallace et al., 2020) to mix up the stylometric features among different authors to make attribution challenging.
AO technique (Hagen et al., 2017) is to prevent this privacy risk. Early AO uses back-translation- i.e., translate texts to another language, then back to the origin language (English <- Italy <- English) by machine translators to remove authorship mark- ers. Recently, AO by textual adversarial attack has gained more attention. Mahmood et al.; Ha- roon et al. use genetic algorithm to replace words with synonyms to maximize the changes of an AA model's 's prediction. Although the results of AOs are encouraging, their defenses can only be successful after a malicious AA model has been trained and released. In our work, we propose to look at this pri- vacy leakage problem holistically and investigate a defense at root-i.e., a mechanism that happens before Data Poisoning, Data poisoning aims to influence the performance of ML models by manipulating the training data and/or labels. Most data poison- ing methods are utilized as "backdoor or trojan at- tack" (Goldblum et al., 2022; Wallace et al., 2020;
Moreover, also take into account the fact that some clean data with true authorship labels have al- ready been publicly available and AA models with good accuracy has already been trained, which is a common problem that existing poisoning efforts often suffer from (Huang et al., 2021; Fowl et al.. 2021). Furthermore, different from existing data poisoning works in NLP domain (Marulli et al., 2021; Yang et al., 2021), also face the challenge of poisoning data that are released in an incremen- tal manner and proposes a class-wide poisoning strategy to dynamically select the optimal pairs of authorship to mix up the stylometric features. Our contributions are as follows:
identifiable texts are released.
We develop practical defense, UPTON, pre- vent identity leakage through user-generated texts before (not after) they are released.
UPTON significantly decreases the accuracies of AA models to ~ 35% on average across public

---

No Clean No TestDe Model
attacker can train AA model FA using T and U and use FA reveal the authorship of unlabeled private texts of users U.
Feature
Trbgger Data Access Access Acc.
(Chen al., 2021)
A Defense Mechanism. age, the defender aims manipulate the users' data T such that when it is published and used to train the AA model FA by the attacker, its accuracy will be significantly dropped. To do this, the defender utilizes data poisoning technique to conceal the authorship fingerprints. To generate poisoned ex- amples, propose to generate perturbation set A via adversarial text generation (Jin al., 2020), append it the users' data T and create its "poi- soned" version T while preserving both textual semantics and original authorship labels. T" can now be released to the public replacing T, which guarantees to significantly lower the accuracy of the attacker's AA model.
To prevent privacy leak-
(Chan et al,, 2020)
(Marulli et al., 2021)
(Wallace al., 2020)
(Yang et al., 2021)
(Kurita et al., 2020)
UPTON
Table I: UPTON vs. existing text poisoning methods
Yang et al., 2021) where an attacker injects pre- e.g., few repeated characters or words (Marulli et al., 2021), to the training inputs. During inference, the attacker evokes the abnormal performance of the poisoned models with these triggers. Table lists some data poisoning works in NLP, all of which are backdoor- based.
defined t.g..
Cold-Start, Warm-Start and Incremental Text Release. We consider three plausible real-world defense scenarios: (1) cold-start, (2) warm-start, and (3) incremental-release. In the rare (1) cold- start scenario, user just joins publishing plat- form, posts the first writing and the defender is able poison all data. In the (2) warm-stari sce- nario, a user had previously posted writings to a platform and an AA model could have been trained on these writings already. Manipulating an already highly-performed AA model is non-trivial task as its decision boundary is already well-defined. A good crivacy-rreservigg data poisoning framework should work well in both scenarios, especially in the more difficult warm-start scenario where exist- ing poisoning approaches tend not work (Huang et al., 2021; Evtimov al., 2020; Fowl al., 2021). In the (3) incrementaa-rrleaase scenario, users post writings to a platform incrementally-egg. daily or weekly. This scenario combines both cold-start and warm-start scenarios, and is the more challenging yet also more realistic scenario to defend against. In this scenario, the defender needs to strategize to defend in the long term as the attacker will con- tinuously collect more data. Figure/ shows the workflow of incremental-release scenario, where defenders incrementally poison data and attackers also incrementally collect data.
Differently, we propose to use data poisoning not for bad but for good causes and protect netizen from privacy leakage with four main differences. First, we only have access to the training data-i.e., publicly released texts, and do not have access to ei- ther the inference inputs-i.e., the anonymous texts, or the target model. Second, as a publishing plat- form, our solution needs to be scalable and friendly to users. Therefore, cannot inject repeated trig- gers every time they publish texts, which affects text semantics and user experience. Third, while backdoor- poisoning hopes to maintain the model accuracy on un-poisoned data, we want to degrade such clean accuracy of AA models instead. Fourth, some backdoor methods such as Chen et al.; Chan et al.; Marulli et al. manipulate the labels of the training data, which is impractical in our prob- lem because publishing platforms are still obligated publish the true author names on user-generated contents. This constraint is often referred to as clean poisoning (Turner al., 2018).
3 The Threat Model
An Attack Scenario. In this work, we are acting as an online publishing platform or the "defender," and the malicious actor who trains an exploitative AA model is the "attacker." The online publisher- e.g., Medium, Figment, Facebook, first receives data T from users U and publishes T to their plat- forms, at which point becomes publicly available and might contain unique authorship footprints of U. To capture a correlation between and U, the
Our Method: UPTON
4.1 Poisoning via Adversarial Generation In general, the defender's goal is to utilize an ad- versarial attack to generate poisoned examples for

---

Defenders
Attackers
g
Label
Label
Label
Label s
Attack Target
Benchmark Model
Attacker's Model
Train
Train
Pre-uplosded data
Fa ( C
uploaded data
No Access
+
Poison
Epoch e-e--Pg
Figure 3: Non-target poisoning. Perturbed texts cross adjacent decision boundaries.
Polson Perturbation
tune
Access
data f
transferability among transformer models has also been observed several works (He et al., 2021; Liang al., 2021). This transforms Eq. (1) the new objective function:
Fs
FA
Attacker's Workflow
Authorship Information
Poisoning
Range an Epoch
max O,CA
Σ
[[FB(4++))) 6,), 91).
(2)
Finetuning
Boundary between Defendar and Attocker
GCT
The loss- maximizationn in Eq. (2) can also be trans- lated minimization problem on an incorrect
Figure 2: Workflow of UPTON
target label y + y/:
all document t, E ' on FA as follows:
min Σ L(Fn(t: v.).i). 6,E
(3)
max 6EA
Σ
L[FA(ts + 5.), 31),
(1)
NET
4C7"
Intuitively, we want find the minimal perturba- tion to such that it crosses the decision bound- aries to class yj. Inspired by SOTA adversarial text attack algorithms such as TextFooler (Jin et al., 2020), we adopt a greedy optimization procedure to solve Eq. (2). Specifically, we perturb each of the words in &; with one of its synonyms in the order of their importance to FB 's prediction, while preserving the sentence structures such as part-of- speech (POS) sequence order and semantic mean- ings via Universal Sentence Encoder (USE) (Cer et al., 2018). We also skip perturbing stop-words to maintain the readability of the sentence.
where 6; denotes the perturbation in perturbation set A added to text sample 4; whose ground-truth label is yı- FA has been trained on T. Intuitively, the defenders' purpose is to make the distribution of perturbed samples confused the feature space- i.e., providing inconsistent authorship fingerprint to the attackers and weaken the gencralization ability of their authorship classifier FA through adversarial attack on training samples.
Different from adversarial attack based AO meth- ods such as Mahmood al., defender cannot ac- cess the attacker's model FA either via a public API or its parameters. Thus, we cannot use direct signals from FA as in Eq. (1) to guide the poi- soning process. To overcome this, we train model FB on clean dataset T as surrogate model benchmark reference point to FA (Figure 2). For the attacker's model, note that using the trans- formers is crucial for AA tasks and the attacker has incentives to adapt state-ofthhe-oe (SOTA) trans- formers to build a high accuracy AA model, the attacker's model has transformer-based structure in our scenario. As for defenders, although we do not assume F3 to share the exact architecture and pa- rameters with FA, also need to use strong mod- els the AA task to extract the authorship well and approximate the attacker's model. Thus, the defender also uses transformer-based backbone with an expectation that some authorship features extracted from FB are transferable to FA- This
4.2 Class-wide Poisoning
Eq. (2) shows that the determination of the tar- get label y" is crucial will determine how the authorship markers will shift as a result of the poi- soning process. Previous poisoning works discuss little about the target label y;- Some of them such as Emmery et al. employ a simple non-targeted poi- soning generate perturbations such that the text crosses the decision boundary to any other classes from Ví.
(4) This technique is ineffective in our because the perturbed texts often cross the adjacent decision boundaries but do change much its feature or writing markers, which shown in Figure 3. Other works such as Cherepanova et al.; Emmery et al.
y' {v E Uly vi)

---

Label
Attack Target
Label
Label
Label2
Label
Attack Target
Figure 4: Sample-wide poisoning. Perturbed texts ran- domly enter wrong areas and scatteredly distributed.
Figure 5: Class-wide poisoning. Each label points out to another and is occupied by other labels. Labels ex- change feature wrong label area after poisoning.
The test data will fall into the
employ sample-wide poisoning by setting random target y for each text in perturbation generation, the optimization of Eq. (2) is satisfied when:
Algorithm 1 Class-wide Poisoning
yi - yly C,
(5)
Input: clean dataset benchmark model FB, au-
which means the perturbed text is misclassified to specific label y, which results in stronger pertur- bation on the writing markers, and more confused feature space, as shown in Figure 4. However, Em- mery et al. also shows that sample-wide poisoning is not able to decrease the model accuracy in the warm-start scenario. This happens because the re- sulting poisoned samples are scattered to random target classes and cannot well overwrite the clus- tered clean samples that have already been captured the feature space an already well-performed AA model. This can be observed intuitively from the comparison of the figures of non-target Figure 3, sample-wide Figure and class-wide Figure 5.
thorship class u in T.
Output: the class-wide perturbation set A,
1: for u in ...., N do
2: Random select 100 samples from
3: Non-target poisoning on selected samples.
vf Majority of attack result class.
for Each sample 4 in do
6, = arg min L{FB(t; + +8.). vi). 8,EA
7: end for
8: end for
bor class to u. Setting the neighbor class the target of reduce the difficulty of adversarial perturbation generation, and reduce the time cost and the number of replacement words. Finally, we perform adversarial text generation on every texts of u with the same target u (Alg. 1, Line5-7):
To override these footprints of clean examples of the AA model in the warm-start scenario, we put forward class-wide poisoning strategy (Alg. 1). Our goal is to override the entire feature space of each clean authorship label that are already learned by the AA model by shifting the feature space of multiple poisoned labels at the same time (Figure
y u²;
(7)
The new feature space of u will occupy the old area of class u". Meanwhile, the old area of u will also be occupied by other labels-e.g.. label where z*=u. This "circle of defense" that involves data poisoning of multiple users is unique and is only possible in our setting where the defender-a.kan an online publisher, has access to the data of all users. Thus, UPTON designed to take advantage of this uniqueness.
5). To do this, aim to find uniform, cus- tomized target label u² for all the texts belonging an authorship class u. However, we want to set u² such a way that u" and u are relatively close to each other in the feature space. To find u", we randomly select an arbitrary set of Q number of clean samples from the authorship class u and generate their perturbations using the non-targeted strategy (Alg. 1, Line2-3). Then, we record the prediction labels of the perturbed texts and set the majority of them u" (Alg. 1, Line4).
To visually demonstrate the superiority of class- wide poisoning in maintaining text quality among baselines, present in Table 4 the poisoned sam- ples from the same clean text sample of non-target, sample-wide, class-wide, and backdoor poison- ing. Further analysis of texts quality and semantic preservation of UPTON is further analyzed Sec- tion 6.
(6) where i is the adversarial example of each clean example & of author generated via non-targeted strategy and we set Q-100 our experiments. We find the majority here to find the closest neigh-
u² majority((FFs(()))

---

Datasets
Poisoning
RoBERTa
DistilBERT
(# Samples)
Strategy
Clean Ace
Poisoned Ace
SIM
Clean Acc
Poisoned Acc
SIM
Non-target
70.14%
(6. (6.73%)
98.72%
61.08%
(10.46%
98.72%
WJO (600)
Sample-wide
76.87%
59.41% 41
(10.46%))
93.47%
71.54%
62.80%
(8.74%.J
93.47%
Class-wide
29.39 %
(47.48%
95.32%
(35.93% J) 35.61% %
95.32%
Non-target
74.53%
(24.12%%
98.36%
70.83%
(27.34%%)
98.36%
IMDb10 (5K)
Sample-wide
98.65%
68.45%
(30.20%).)
91.68%
98.17%
66.82%
(31.35%
91.68%
Class-widh
39.02%
(59.63% %..)
97.30%
34.79%
(63.38%
97.30%
Non-target Sample-wide
77.48%
(17.75% L)
98.92%
70.24%
(23.68%
98.92%
IMDb62 (20K)
95.23%
71.41%
(23.82%%])
92.84%
93.92%
66.10%
(27.82%1)
92.84%
Class-wide
31.27% %
(63.96%
96. 64%
35.78%
(58.14%
96.64%
Non-target Sample-wide
66.36%
(23.57%))
97.35%
68.02%
(20.77%%)
97.35%
Enron (30K)
89.93%
65.
(24.83%.1)
90.38% (55.3%%.)
38.79%
61.98%
(26.87 (2.8))%
90.38%
Class-widh
34.55% %
97. 97.02%
36.74% %
(52.05 %
97.02%
Table Experiment results on cold-start poisoning
5 Experiments
readability of a text after perturbation via an inde- pendent BERT encoder.
5.1 Experiment Set-up
We refer the readers to other implementation details in Appendix A.I.
Datasets We use IMDb10, IMDb62 (Seroussi et al., 2014). Enron Email (Enron) (Klimt and Yang, 2004), and Western Journal Opinion (WJO) col- lected by ourselves from WJO. These datasets vary in size, length, topics and the number of authors. We split each of the datasets into train and test set. We use the train set as the texts to be poisoned and released to the public by the defender. The test set is the un-poisoned, private and anonymous texts to be protected from authorship leakage.
5.2 Experiment Results
Cold-Start Scenario. We select 600, 5K, 20K. and 30K # of train examples with true authorship labels in WJO, IMDb10, IMDb62, and ENRON, respectively, to simulate dataset 7 of varying sizes to be poisoned and released to the public, assuming no publicly available data with the same labels exist. Table 2 summarizes the results. UPTON with class- wide poisoning significantly outperforms baselines on poisoning effectiveness. The poisoned models' accuracy decrease to 29.39%, 39.02%, 31.27% and 34.55% in four datasets on RoBERTa, dropping more than half of their clean accuracies. Non-targer and sample-wide poisoning strategies are only able to decrease the accuracy to around 60% to 70%. We also observe similar results for DistilBERT. Moreover, the semantics between original and perturbed texts are well preserved with BertScore is consistently higher than 90% and only second to non- non-targen con-targeted poisoming which often requires less number of perturbations to move the input texts out of the current authorship class (and not to any specific target label).
AA Models. We use pre-trained transformer un- cased BERT (Devlin et al., 2018) to train the sur- rogate FB model. As for attacker's models, we use other transformer-based, RoBERTa (Liu et al.. 2019) and DistilBERT (Sanh al., 2019) models.
Baselines. We use two existing poisoning strate- gies: non-target (Emmery et al., 2021) and sample- wide poisoning (Cherepanova al., 2021) I) as com- parison baselines.
Evaluation Metrics. Model Accuracy. We re- port the accuracy of the attacker's AA models when trained on clean (denoted as "Clean Ace") and poisoned (denoted as "Poisoned Acc") dataset. The lower the poisoned accuracy is, the more ef- fective the defense is. We use accuracy and not other prediction metrics such as Fl because ac- curacy is more commonly used in AA (Fabien et al., 2020) and we do not differentiate the per- formance among authorship labels. We also use BERTSCORE (Zhang et al., 2019), denoted as "SIM", to evaluate the semantic preservation of the perturbed texts as similarly done in Morris et al.. BERTSCORE measures the semantic changes and
Warm-start Scenario. We first divide each train- ing set into 2 parts and use 310, 1K, 5K, and 10K examples in the first part of WJO, IMDb10, IMDb62, and ENRON, respectively, existing texts with clean authorship labels that are already publicly available for the attacker to train the AA model FA and for the defender to train the surro- gate model FB. Then, we poison the second part of

---

Attacker's Model
RoBERTa
DistiBBERT
Poison rate
Attack senario
WJO
IMDb10
IMDb62
Eron
WJO
IMDb10
IMDb62
Enron
0%
Clean
76.87%
98.65%
95.23%
89.93%
71.54%
98.17%
93.92%
88.79%
Non-target
76.95%
98.58%
96.03%
90,20%
73.6 %
98.37%
92.80%
89.33%
75%
wadele-wide
77.01%
97.72%
95.86%
88.69%
70.73%
97.46%
92.52%
89.16%
Class-wide
32.86 %
36.18% 18%
39,45%
32.16% %
36.81 %
34.70% %
41.29 %
39.1 %
Non-target 74.75%
97.99%
98%
91.51%
71.29%
96.33%
94.93%
88. 18.16%
50%
Sample-wide
76.22%
98.63%
95.24%
88. 79%
71.04%
96, 96.80%
85.20% 93.59%
Class-wide 35.75%
41.64%
43.00
38.11 %
42.02%
39.09 %
44.92%
45.37
Non-target 75.71%
99.05%
95.91%
90.13%
73.06%
98.55%
93.89%
88.56%
25 %
Sample-wide 77.53%
98.25%
94.41%
90.10%
71.34%
97.94%
93.15%
88.22%
Class-widc
41.58%
58.01% %
49.39
54.80
50.20%
54.86% %
55.17% %
42.44
Table Experiment results on warm-start poisoning
Non-target
Sample-wide
Class-widc
Clean
100
100
90
90
Jccumity
70
70
70
80
40
120
240
380
450
600
100
200
300
400
500
2000
4000
6000
8000
10000
4000
8000
12000 16000 20000
Number lends
Number
(a) WJO
(b) IMDb10
(c) IMDb62
(d) Enron
Figure Experiment results on incremental warm-start poisoning
training sets using the surrogate model only once and report the results with different poison rates- i.e., the proportion of poisoned data to the sum of poisoned and clean data. We use three poison rates, namely 75%, 50% and 25%, simulate when de- fender releases more, equivalent or less poisoned data than clean data, respectively.
an appropriate poisoning strategy is crucial and the proposed class-wide strategy is effective across both cold-start and warm-start scenario.
CirrmeentalRRleacee Scenario. We use exper- iments to imitate the defenders and attackers in warm-start incremental data release. We use the BERT benchmark model to generate poisoned dataset, and re-train the clean ReBERTa model we used in warm-start poisoning. The initial accu- racy of these models can be found in RoBERTa Clean row of Table 3. We poison texts and re-train attacker's models in batches to stimulate the incre- mental data release scenario. We set the batch size as the number of texts that can train clean model to around 50% accuracy. The numbers are 50, 1K. 2K and for WJO, IMDb10, IMDb62, and Enron.
Table summarizes the results. UPTON with class-wide poisoning observes an outstanding per- formance in warm-start scenario. In all poison rates, the class-wide poisoned accuracy is much lower than the clean models, dropping from 70%- 99% 32%-60%. Most notably, the accuracy Enron dataset drops only 32.16% with 75% poi- son rate. Although is intuitive that the higher the poison rate, the lower the accuracy of the poi- soned model will become, the baseline methods are ineffective the warm-start setting even with a high poison rate. Their poisoned accuracies remain more or less the same as clean AA models, even with 75% poison rate.
We reveal the line chart of incremental poisoning in Figure 6, which shows the effectiveness of class- wide poisoning. The model accuracy drops rapidly when more poisoned texts are released, while the accuracy of non-target and sample-wind wide basically example,
In some cases, the accuracy even slightly in- creases after poisoning by sample-wide strategy ( 95.23% to 95.86% in IMDb62 on RoBERTa) be- cause the resulting weak perturbations actually acts as adversarial examples and helps improve the ro- bustness of the attacker's model. This shows that
remain unchanged. Set IMDb62 the poisoned accuracy drop 36.34% after 10K samples released. The accuracy of baselines are around 95%, almost the same as the clean model.
We put the analysis of BERTSCORE in warm-start and Incremental Releaase scenario in Appendix A.2.

---

Clean Data Label
Poisoned Data Label
Clean Data of other Labels
32-
331
329
(a) Non-target
(b) Sample-wide
(c) Class-wide
Figure 7: Feature space distribution after poisoning
Backdoor Data Poisoning (Yang et al., 2021)
WJO
IMDb10
Ha! imagine that if anyone [...] She was very wzy charming
Attack senario
C/T
Accuracy
C/T
Accuracy
and [..]] pleasure of meeting. "Butch knows best."
Test Example: agree: you need a tough wzy looking [...]
Clean
0.0893
75.07%
0.1319
99.21%
Non- Targeted Data Poisoning
Non Non-target
0.0948
60, 60.99%
0.1247
77. 77.42%
Ha! imagine that if anyone [..] had pleasure meeting.
Sample-wide
0,1042
56.29%
0.1663
65.51%
"Butch knows best."
Class-wide
0.0925
40.60 %
0.1312
24.70%
Sample-Wide Data Poisoning
CT: number of corrections per text
Ha! I imagine that if anyone could convince gay adult judge being true, Ashley would [...] and chat to her [...] the dainty (and hottest) celebrities have accept the pleasure merging. "Butch knows considerably."
Table 5: UPTON robustness against NeuSpell
6.2 Semantic Change and Readability
Class-Wide Wide Data Poisoning (UPTON)
We evaluate and compare UPTON poisoned texts with other works prove that UPTON works well on preserving semantic meanings and readability of poisoned texts. Table 4 shows an example from IMDb10 and the difference of semantic changes between poisoning approaches. There is only one punctuation changed ("+') in non non-target poison- ing and it does not affect the original context and meaning. The sample-wide poisoned text contains the most substitute words with of 60 words placed. Some of which affect the readability of the sentence-e.g. ""straigt"">""mn poisoning, the poisoned text has only one perturbed word (very-treally) with limited semantic change and is still readable. which gives credit to its dy- namic adjacent target class selection. Yang et al. is backdoor poisoning. It injects meaningless trigger "wzy" both train and test samples, dis- crediting the integrity of the sentence.
Ha! imagine that if anyone [...] She really charming and [... pleasure of meeting. "Butch knows best."
Table Poisoned Samples UPTON, baselines and backdoor method (Yang et al., 2021) on same text. On backdoor poisoning requires access to test examples.
6 Discussion
6.1 Feature Space Analysis
In class-wide
Figure 7 shows the feature distribution of few authorship labels from IMDb10 dataset during poi- soning by projecting the surrogate BERT's pool- ing layer 2-D space. Figure 7 (a) confirms our analysis in Sec. 4.2 that most of non-target poisoned texts enter right at the adjacent classes. Figure? 7 (b) shows that the poisoned texts disperse to varying classes' regions in sadele-wide poison- ing. These are likely considered only as noise to target AA model and not powerful enough to de- grade its accuracy especially in warm-start setting. Figure (c) shows that all the poisoned data enter the nearest area and closely distributed in class- wide poisoning. Therefore, UPTON successfully occupied the adjacent class region to clean label "0" and significantly hindered the Icarnability the AA model.
6.3 Robustness against Misspell Correction
We test the robustness of UPTON against a deep-learning-based spelling correction toolkit, NeuSpell (Jayanthi et al., 2020), which shows to be able remove both char and word-level adversar- ial perturbations (Jayanthi et al., 2022). We eval- uate the performance of attacker's model trained on clean and poisoned data after correct spellings by NeuSpell. Table 5 shows that poisoned texts

---

share the same number of correct spellings per text the original lexts. Compared with Table 2, the poisoned accuracy are also similar to those with- out NeuSpell. This indicates that spelling correc- tion cannot effectively remove UPTON poisoning, demonstrating that UPTON is robust in practice.
Table 6: Transferability of UPTON between different model structures IMDb10 dataset and 50% poison rate. Each cell represents the accuracy of the poisoned model. with the vertical axis representing the FA model structure and the horizontal axis representing the FB model structure.
Model structure (Fa/FB)
6.4 Compare UPTON with generative language models
BERT
RoBERTa
DistiBBERT
BERT RoBERTa DistBBERT
41.64%
34.70% 40.28%
Considering the flourishing development of gener- ative language models, we also explored whether social media users could utilize these large lan- guage models such as GPT-3 (Brown et al., 2020) and GPT-3.5 (Ouyang et al., 2022) to assist in re- moving authorship. Through experimentation, we identified two primary drawbacks in this kind of approach when compared to UPTON's solution, i.e., (1) Polishing text by large language models is computationally intensive. Social media plat- forms, dealing with substantial volume of users' texts, would ineur significant costs and associated processing delays when polishing texts on these generative models: (2) Our experiments indicate that large language models such as GPT-3.5 tend to rewrite texts when removing authorship, signifi- cantly impacting the original semantic content of the text. Additionally, text authors may prefer ap- proaches like UPTON, which subtly alter wording and sentence structure without introducing conspic- uous changes.
32,99%
39.69%
45.12%
is not strongly influenced by the structure of FB. The defender is only required to use SOTA models to extract the authorship features.
Conclusion
Authorship attribution is an emerging privacy threat. This paper proposes UPTON, a black-box data poisoning service for online publishing plat- forms to generate poisoning perturbations on texts that can make them useless for authorship attribu- tion training. Our experiments show that UPTON is effective for downgrading the test accuracy in both cold-start, warm-start, and incremental release scenarios. The poisoning also has excellent trans- ferability while the semantics of poisoned texts are well maintained.
In Table 10, we present the results of the same original sample in both UPTON and GPT-3.5. Evidently, our approach still introduces minimal changes to the original text, whereas GPT-3.5 tends to rewrite the text in its own language. We believe that such extensive alterations do not align with scenario, especially for researchers and political activists who demand a higher standard of their published content.
Limitations
Our framework has two main limitations. (1) Trade- off of Privacy and Writing Style:, our UPTON scheme requires adjusting the text posted by on- line media users, which may affect the expressive- ness of their contents for some content creators with strong personal writing styles. Although be- ing used for good purposes, adversarial poisoning proposed in the paper may undesirably erase some of their personal stylometric styles. (2) Trade-off of privacy and computational complexity: targeted text adversarial generation is time-consuming due to the generation of text perturbations. Therefore, in social platforms where multiple users are simul- tancously posting text, there will be posting delay while the perturbations posted text. This may to some extent affect user experience. However, the perturbation generation process can be done in parallel to significantly re- duce the runtime.
6.5 Transferability results of UPTON more model structures
In Section 5.2, we display the poisoning perfor- mance when FB is a BERT model and FA has RoBERTa or DistilBERT structures. Here we ex- tend our experiment when FB is RoBERTa- based DistilBERT- based and see if the poisoning can still transfer to another structures.
loaded for each newly
Our results on IMDb10 and 50% poison rate are in Table 6, where we got almost the same results as those when we use only BERT in FB. This indi- cates that the poisoning effectiveness UPTON

---

Broader Impacts and Ethics Statement
Tom Brown, Benjamin Mann, Nick Ryder, Mclanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, al. 2020. Language models few-shot learners. Advances in neural information processing systems 33 (2020), 1877-1901.
The objective of this study is to address the pri- vacy concerns related to text release and content creation online and does not involve other sensi- tive information ethical concerns related to Al and society. The experiments section of this paper uses publicly available text data from the Internet with appropriate citations of their sources. The proposed framework has a large implication on the responsibility of online content publishers to pro- tect the privacy of their users. Moreover, it also encourages the research community to safeguard against potential misuse of NLP techniques, even those that might have been originally developed for benevolent purposes.
Guilherme Ramos Casimiro and Luciano Antonio Di- giampietri. 2022. Authorship Attribution with Tem- poral Data in Reddit. In XVIII Brazilian Symposium on Information Systems. 1-8.
Daniel Cer, Yinfei Yang, Sheng-yi Kong. Nan Hua, Nicole Limtiaco, Rhomni St John, Noah Constant, Mario Guajardo-Cespedes, Steve Yuan, Chris Tar, et al. 2018. Universal sentence encoder. arXiv preprint 1rXivv1803..1111 (2018).
Alvin Chan, Yi Tay, Yew-Soon Ong, and Aston Zhang. 2020. Poison attacks against text datasets with condi- tional adversarially regularized autoencoder. arXiv preprint 2rXhv:2010.02684 (2020).
Acknowledgements
This work was in part supported by NSF awards #1820609, #193-782,, #2114821, and #2131144. We thank the anonymous reviewers for their constructive feedback.
Xiaoyi Chen, Ahmed Salem, Michael Backes, Shiqing Ma, and Yang Zhang, 2021. Badnl: Backdoor at- tacks against nlp models. In ICML 2027 Workshop on Adversarial Machine Learning.
Valeriia Cherepanova, Micah Goldblum, Harrison Foley. Shiyuan Duan, John Dickerson, Gavin Taylor, and Tom Goldstein. 2021. LowKey: Leveraging Adver- sarial Attacks to Protect Social Media Users from Fa- cial Recognition. arXiv preprint X1t:2201100792 (2021).
References
Ben Allison and Louise Guthrie. 2008 Authorship attribution e-mail: Comparing classifiers over a new corpus for evaluation. In Proceedings of the sixth International Conference Language Resources and Evaluation (LREC'08).
Jacob Devlin, Ming-Wo Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. CoRR abs/1810.0440 (2018). arXiv:1810.04805 http://arxits www.arwwww.ooooaaeeeeeeee www.orrg///////eeeeeeee
Janek Amann. 2019, Authorship Attribution in Fan- Fictional Texts given variable length Character and Word N-Grams. (2019).
04805
KA Apoorva and S Sangeetha. 2021. Deep neural net- work and model- based clustering technique for foren- sic electronic mail author attribution. SN Applied Sciences 3 (2021), 1-12.
Chris Emmery, Ákos Kádár, and Grzegorz Chrupata. 2021. Adversarial stylometry in the wild: Transfer- able lexical substitution attacks on author profiling. arXiv preprint 1001::2001.113/0 (2021).
Andrea Bacciu, Massimo La Morgia, Alessandro Mei, Eugenio Nerio Nemmi, Valerio Neri, and Julinda Stefa. 2019. Cross-Domain Authorship Attribution Combining Instance Based and Profile-Base -Based Fea- tures.. In CLEF (Working Notes).
Ivan Evtimov, Pascal Sturmfels, and Tadayoshi Kohno. 2020. Foggysight: A scheme for facial lookup pri- vacy. arXiv preprint arXiv: 08588 (2020).
Maël Fabien, Esaú Villatoro-Tello Petr Motlicek, and Shantipriya Parida. 2020. BertAA: BERT fine-tuning for Authorship Attribution. In Proceedings of the 17th International Conference Natural Language Processing (ICON). 127-137
Mudit Bhargava, Pulkit Mchndiratta, and Krishna Asawa. 2013. Stylometric analysis for authorship attribution on twitter. In Big Data Analytics: Second International Conference, BDA 2013, Mysore, India, December 16-18, 2013, Proceedings 2. Springer, 37- 47.
Liam Fowl, Micah Goldblum, Ping-yeh Chiang, Jonas Geiping, Wojtek Czaja, and Tom Goldstein. 2021. Adversarial examples make strong poisons. arXiv preprint arXiv:2106. 10807 (2021).
Benedikt Boenninghoff. Steffen Hessler, Dorothea Kolossa, and Rohert M Nickel. 2019. Explainable authorship verification in social media via attention- based similarity learning. In 2019 IEEE International Conference on Big Data (Big Data). IEEE, 36-45.
Micah Goldblum, Dimitris Tsipras, Chulin Xie, Xinyun Chen, Avi Schwarzschild, Dawn Song, Aleksander Madry, Bo Li, and Tom Goldstein. 2022. Dataset

---

security for machine learning: Data poisoning, back- door attacks, and defenses. IEEE Transactions on Pattern Analysis and Machine Intelligence (2022).
Kaizhao Liang, Jacky Y Zhang, Boxin Wang. Zhuolin Yang, Sanmi Koyejo, and Bo Li. 2021. Uncovering the connections between adversarial transferability and knowledge transferability. In International Con- ference on Machine Learning. PMLR, 6577-6587.
Matthias Hagen, Martin Potthast, and Benno Stein. 2017. Overview of the Author Obfuscation Task PAN 2017: Safety Evaluation Revisited.. In CLEF (Working Notes).
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Man- dar Joshi, Danqi Chen, Omer Levy. Mike Lewis. Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining ap- proach. arXiv preprini arXiv: 1692 (2019).
Muhammad Haroon, Fareed Zaffar, Padmini Srinivasan, and Zubair Shafiq. 2021. Avengers Ensemble! Im- proving Transferability of Authorship Obfuscation. arXiv preprint arXiv:2109.00000 (2021).
Asad Mahmood. Faizan Ahmad, Zubair Shafiq, Padmini Srinivasan, and Fareed Zaffar. 2019. A Girl Has No Name: Automated Authorship Obfuscation using Mutant-X. Proc. Priv, Enhancing Technol. 2019, 4 (2019), 54-71.
Xuanli He, Lingjuan Lyu, Qiongkai Xu, and Lichao Sun. 2021. Model extraction and adversarial trans- ferability, your bert is vulnerable! arXiv preprint arXiv:2103. 10013 (2021).
Fiammetta Marulli, Laura Verde, and Lelio Campanile. 2021. Exploring Data and Model Poisoning Attacks to Deep Learning- Based NLP Systems. Procedia Computer Science 192 (2021). 3570-3579.
Hanxun Huang, Xingjun Ma, Sarah Monazam Erfani, James Bailey, and Yisen Wang. 2021. Unlearn- able examples: Making personal data unexploitable. arXiv preprint arXiv:2101.00 048998 (2021).
John X Morris, Eli Lifland, Jin Yong Yoo, Jake Grigsby, Jin, and Yanjun Qi. 2020. Textattack: A frame- work for adversarial attacks, data augmentation, and adversarial training in nip. arXiv preprint arXiv:2005.c 20055990 (2020).
Sai Muralidhar Jayanthi, Danish Pruthi, and Graham Neubig. 2020. NeuSpell: A Neural Spelling Correc- tion Toolkit, In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Process- ing: System Demonstrations. Association for Com- putational Linguistics, Online, 158-164. https:
Hendrik Neumann and Martin Schnurrenherger. 2009. E--aill authorship attribution applied to the Extended Enron Authorship Corpus (XEAC).
//doi. org/10. 11653/v1/2022 36653//1/200.emm11dddeeee emnlp-demos....... 21
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, al. 2022. Training language models to follow instruc- tions with human feedback. Advances in Neural Information Processing Systems 35 (2022), 27730- 27744.
Sai Muralidhar Jayanthi, Danish Pruthi, and Neuspell de-
Graham Neubig. 2022. fense against adversarial character attacks. https: / /gi thub. cww
Haiyan Jia and Heng Xu. 2016. Measuring individuals' concerns over collective privacy on social networking sites. Cyberpsyccolooy: Journal of Psychosocial Research on Cyberspace 10, 1 (2016).
Hoshiladevi Ramnial, Shireen Panchoo, and Sameer- chand Pudaruth. 2016. Authorship attribution us- ing stylometry and machine learning techniques. In Intelligent Systems Technologies and Applications. Springer, 113-125.
Di Jin, Zhijing Jin, Joey Tianyi Zhou, and Peter Szolovits. 2020. Is bert really robust? a strong base- line for natural language attack on text classification and entailment. In Proceedings of the AAAI confer- ence on artificial intelligence, Vol. 34. 8018-8082
Anderson Rocha, Walter J Scheirer. Christopher W Forstall, Thiago Cavalcante, Antonio Theophilo. Bingyu Shen, Ariadne RB Carvalho, and Efstathios Stamatatos. 2016. Authorship attribution for social media forensics. IEEE transactions on information forensics and security 12, (2016), 5-33.
Patrick Juola al. 2008. Authorship attribution. Foun- dations and Trends@ in Information Retrieval I, 3 (2008), 233-334.
Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. 2019. DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter. arXiv preprint arXiv:1910.00 108 (2019).
Bryan Klimt and Yiming Yang. 2004. Introducing the Enron corpus.. In CEAS.
Keita Kurita, Paul Michel, and Graham Neubig. 2020. Weight poisoning attacks on pre-trained models. arXiv preprint arXiv:2004.06661 (2020).
Yanir Seroussi, Ingrid Zukerman, and Fabian Bohn- ert. 2014. Authorship attribution with topic models. Computational Linguistics 40, 2 (2014), 269-310.
Yafu Li, Qintong Li, Leyang Cui, Wei Bi, Longyue Wang, Linyi Yang, Shuming Shi, and Yue Zhang. 2023. Deepfake Text Detection in the Wild, arXiv preprint 23055 2002223355.42. 13242 (2023).
Antonio Theophilo, Romain Giot, and Anderson Rocha. 2021. Authorship attribution of social media mes- sages. IEEE Transactions on Computational Social Systems (2021).

---

Antonio Theophilo, Rafael Padilha, Fernanda Andaló. and Anderson Rocha. 2022. Explainable Artificial Intelligence for Authorship Attribution on Social Me- dia. In ICASSP 2022-2022 IEEE International Con- ference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2909-2913.
A
Appendix
A.1 Reproducibility Details
We share in this section experiment settings that are important to reproduce the results presented in the paper.
Alexander Turner. Dimitris Tsipras, and Aleksander Madry. 2018. Clean-label backdoor attacks. (2018).
IMDb Corpora IMDb62 and IMDb10. The full IMDb dataset (Seroussi et al., 2014) has textual movie reviews from 22, 16 authors and has 34 mil- lion tokens. Most authorship attribution works are benchmarked two specific sub-datasets, namely IMDb62 and IMDb10. IMDb62 is selected sub- set of IMDb with 62 authors and 22 million tokens. We also made an IMDb10 dataset containing 10 au- thors that have most texts in the full IMDb dataset without author overlap with IMDb62.
Adaku Uchendu, Thai Le, and Dongwon Lee. 2023. Attribution and Obfuscation of Neural Text Author- ship: A Data Mining Perspective. ACM SIGKDD Explorations 25, 1 (2023), 1-18.
Adaku Uchendu, Zeyu Ma. Thai Le, Rui Zhang, and henchmark en-
Dongwon Lee. 2021. Turinghench: vironment for turing test in the age of neural text gen- eration. arXiv preprini arXiv: 1009 11199.1132 13296 (2021).
Eric Wallace, Tony 7. 7,aa,, Shi Feng, and Sameer Singh. 2020. Concealed data poisoning attacks on nlp mod- els. arXiv preprint arXiv:2010. 12362 (2020).
Enron Email Corpus. Enron Email data was collected and made public from the mailbox of 158 employees in the Enron Corporation after the scan- dal of the company (Klimt and Yang, 2004). The corpus contains 2600, 000 email texts. Allison and Guthrie studied authorship attribution on the dataset for the first time, which was then followed by Neumann and Schnurrenberger and Ramnial et al.. Most recent works (Apoorva and Sangeetha, 2021; Fabien al., 2020) use deep learning to iden- tify authorship on the Enron dataset. We randomly sample 100 authors with the most emails in the corpus for our experiments.
Wenkai Yang, Lei Li. Zhiyuan Zhang, Xuancheng Ren, Xu Sun, and Bin He. 2021. careful about poisoned word embeddings: Exploring the vulnerability of the embedding layers in nlp models. arXiv preprint arXiv:2103. 15543 (2021).
Richong Zhang, Zhiyuan Hu, Hongyu Guo, and Yongyi Mao. 2018 Syntax encoding with application in authorship attribution. In Proceedings of the 2018 Conference on Empirical Methods in Natural Lan- guage Processing, 2742-2753.
Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q Weinberger, and Yoav Artzi. 2019. BERTScore: Evaluating Text Generation with BERT. CORR http:
Western Journal Opinion Corpus. We also collect articles published the top 10 most active article publishers from the opinion plate of The Western Journal (WJO) website as an additional corpus. It contains 772 real articles published on the Internet from WJO. This corpus will help eval- uate if we can protect the authorship attribution of real publishing platform.
abs/1904.09675 (2019). 190.00999 //arxiv.org/abs/11e 19t.s/9994999955
Xuezhou Zhang, Xiaojin Zhu, and Laurent Lessard. 2020. Online data poisoning attacks. In Learning for Dynamics and Control. PMLR, 201-210.
Wei Zhao, Maxime Peyrard, Fei Liu, Yang Gao, Chris- tian M Meyer, and Steffen Eger. 2019. MoverScore: Text generation evaluating with contextualized em- beddings and earth mover distance. arXiv preprint arXiv: /009::9909.000 099900609222222 (2019).
Due to the fact that the adversarial perturbation generation is not always successful, in each of the three poisoning methods, we collect the unsuc- cessful samples after the first round of poisoning generation and perform re-generation. The re- generation will adhere to the setting of the first round.
Rong Zheng, Jiexun Li, Hsinchun Chen, and Zan Huang. 2006. framework for authorship identification of online messages: Writing-style features and classifi- cation techniques. Journal of the American society for information science and technology 57, (2006), 378-393.
Ultimately, we perform non-target poisoning those that fail to generate perturbation in two rounds and keep the original clean form of those texts that still fail in this round. We always use TextFooler in the entire poisoning process, which guarantces the successful poisoning of almost all samples, and requires reasonable time costs.
Wanjun Zhong, Duyu Tang, Zenan Xu, Ruize Wang. Nan Duan, Ming Zhou, Jiahai Wang, and Jian Yin. 2020. Neural deepfake detection with factual structure text. arXiv preprint 20100000 (2020).

---

Table 7: BERTSCORE on warm-start scenario
existing works show that MoverScore has similar trends to BERTScore, so the experiment not in- cluded.
Poisoning method
WJO
IMDb10
IMDb62
Eron
Non-target
98.77%
97.29%
98.78%
98.23%
Sample-wide
96.10%
90.71%
93.65%
93.03%
Class-wide
96.91%
97.33%
94.82%
96.53%
Table 8: BERTSCORE similarity on incremental poison-
ing
Poisoning method
WJO
IMDb10
IMDb62
Eron
Non-target
98.47%
98.01%
97.42%
97.29%
Sample-wide
96.22%
92.44%
92.69%
91.18%
Class-wide
97.15%
95.04%
94.25%
96.25%
A.2
BERTSCORE in more experiment settings
Table 9: Comparison of BERTSCORE and USE similar-
ity
Dataset
SIM-BERTSCORE
SIM-USE
WJO
98.01% %
97.42%
IMDb10
92.44%
92.69%
IMDb62
92.44%
92.69%
Eron
95.04%
94.25%
The numerical values of BERTSCORF. in warm- start and Incremental release scenarios are similar to those in Cold-Start and also share similar size relation between poisoning methods. We put the average BERTSCORE similarity of all the poisoned texts in the warm-start scenario in Table 7. Those in the incremental release scenario are in Table 8. All these results show that non-target has the highest similarity, following the class-wide and sadele-wide poisoning. All the similarities are higher than 0.9, which shows the promising quality of our generated poisoned texts.
In addition, here we also provide additional experiments using different similarity measures. Adopting from existing works, utilize USE (Universal Sentence Encoder (Cer al., 2018)) vectors original and poisoned texts and then compute their cosine similarities. We display both the average similarities (in [0,1]) of BERTSCORE and USE in 500 UPTON original- perturbed sam- ple pairs in each dataset in Table 9. We observe that both BERTSCORE and USE similarity show that our method is able preserve the original se- manties of the text documents very well, making our framework practical to perturb contents before publishing. We have also considered other metrics such as MoverScore (Zhao et al., 2019). However,

---

Original text
Okay, I'm glad that they made a sequel to the original film. In this film, Carol's first hushand, Roy Martin, supposedly returns to reunite with his beautiful wife and three blonde daughters. Unfortunately, he's 'ss not who or what he appears to be. Hc seems trapped in Brady's s world where cable television is far too out there for them and where dysfunctional scems to he the norm. Greg and Marcia get new digs in the attic with creepy sexual tension between them while Jan fakes new boyfriend.
GPT-3.5 (Prompts: Can you help me polish this text to conceal the authorship: [Original text]?)
Alright, it's satisfying see that a sequel was created for the initial movic. This time around, the storyline involves Carol's former spouse, Roy Martin, supposedly rejoining his attractive wife and their trio fair-haired daughters. However, things are not as straightforward as they seem. Roy finds himself ensnared in Brady's 's world, place where unconventional cable television appears too unconventional, and dysfunctionality appears the norm. Meanwhile, Greg and Marcia find themselves in new accommocations in the attic, with an eerie undercurrent of ambiguous tension. Simultaneously, Jan fabricates a new romantic partner.
Class- Wide Poisoning (UPTON)
Ours (UPTON class-wide): Okay, I'm happy they made a sequel to original movie. Roy Martin, supposedly returns to reunite with beautiful wife and three blonde daughters. Unfortunately, he's not who what he appears to be. He seems trapped in Brady's 's world where cable television is far too out there for them where dysfunctional seems to the norm. Greg and Marcia get new digs the attic with creepy sexual tension between them when Jan fakes new boyfriend.
film, Carol's first husband,
Table 10: Processed Samples by GPT-3.5 and UPTON, GPT-3.5 tends to rewrite the text to remove the authorship.
which falls short in preserving the original quality of the text.


Reviews:
[{"review_id": "kh19q6LXs8", "reviewer": "Reviewer_MCWh", "summary": "", "questions": "Nowadays, it is proven that using a BlackBox approach, such as back-translation or GPT models, is more effective in maintaining good stealthiness compared to word-level synonym substitution. We understand the claim that synonym substitution is a cheaper alternative. However, in a real scenario, if a social media company wants to launch a feature that helps users polish their content, they might prioritize user experience over costs.\n\nBuilding on the previous question, if more and more users adopt GPT-4 to polish their content, it raises the question of whether an attack model would still be able to detect user identity based on their posts. I encourage the authors to address the model's robustness by showcasing its performance on black box paraphrased content in an identical setting. This would help demonstrate the effectiveness and reliability of the proposed approach in real-world scenarios.", "limitations": "", "rating": 2, "confidence": 5, "soundness": 2, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Summary:** We appreciate the reviewer's thoughtful feedback on our work. We recognize that most of the reasons for rejections result from the reviewer's misunderstanding of our work, and hence, we want to address them in detail below. We respectfully urge the reviewer to reconsider these points in the final assessments, as they do not constitute major technical or methodological problems.\n\n> Q1. It is unclear how word-level synonym replacement can cause a distinct model ($F_A$) to change its predicted labels if the attack model is independent and inaccessible. It seems that $F_A$ should be a frozen model without any parameter tuning.\n\nWe would like to clarify the misunderstanding that may have led to this question. Although $F_A$ is independent and inaccessible, it is NOT a frozen model. This happens mainly because the attackers are incentivized to optimize the $F_A$ model’s accuracy overtime to optimize its attack performance. To do this, the attacker would want to train the $F_A$ model on not only as many training examples as possible but also on recently released data (from incremental release setting). This araises from the main difference of our work from the majority of prior obfuscation works that use a frozen model $F_A$. Particularly, we specifically target a novel use case of social media platforms where users post data frequently (e.g., daily posting of new tweets or Facebook entries), so attackers are also incentivized to update their model frequently to optimize the accuracy. The consideration of this more realistic scenario on social platforms and the idea of using the training data that could be poisoned are one of our key contributions.\n\n> Q2. The main contribution of this paper appears to be the new label-assigning process for poisoned data, verified on three different settings. However, this may not be sufficient novelty for an EMNLP paper. Please correct me if I am missing something important.\n\nThis is a misunderstanding of our work. We would like to clarify that UPTON does NOT assign new labels but ONLY changes the textual features of social posts. Without changing the labels in $F_A$, we change the feature distribution of training examples. Thus, $F_A$ will learn from a modified authorship feature distribution and get confused, leading to a decrease in accuracy. When the users need to upload private texts, $F_A$ then cannot classify these private texts well because it learned from a manipulated feature distribution.\n\nOur contributions also include how we manipulate the feature distribution. Different from previous clean label poisoning works such as [1-2], which only poison one label, UPTON manipulates ALL labels at the same time. This strategy makes it possible for UPTON to preserve the privacy of multiple users at the same time. Moreover, as we responsed to Q1, UPTON works in a more realistic scenario where a set of users want to protect their authorship in public while incrementally releasing their data. \n\n> Q3.The paper only measures bertscore to illustrate the similarity of the newly generated text. However, there are many other matrices that could measure the stealthiness of the generated content, and none of them are discussed in the paper.\n\nWe here provide additional experiments using different similarity measures. Adopting from existing works, we utilize USE (Universal Sentence Encoder) vectors of original and perturbed texts and then compute their cosine similarities. We display the average similarities (in [0,1]) of 500 original-perturbed sample pairs in each dataset below.\n\n=========================================\n\nDatasets  $\\quad$  BERTScore Similarity $\\quad$  USE Similarity\n\n=========================================\n\nWJO         $\\qquad$       $\\qquad$    $\\quad$      95.32%           $\\qquad$      $\\qquad$          94.52%   \n\nIMDb10       $\\qquad$       $\\quad$    $\\enspace$     97.30%      $\\qquad$   $\\qquad$        96.35%\n\nIMDb62       $\\qquad$       $\\quad$    $\\enspace$     96.64%        $\\qquad$   $\\qquad$     97.00%\n\nEnron      $\\qquad$      $\\quad$    $\\enspace$    $\\enspace$   97.02%         $\\qquad$    $\\qquad$     95.11%\n\n=========================================\n\nWe observe that both BERTScore and USE Similarity show that our method is able to preserve the original semantics of the text documents very well, making our framework practical to perturb contents before publishing. We have also considered other metrics such as MoverScore. However, existing works show that MoverScore has similar trends with BERTScore [4]. We will include USE Similarity evaluation in our final manuscript. \n\n> Q4.I am a little concerned about why authors would want to replace their originally created content with randomly placed machine-generated text. This raises the essential question of whether users want to be recognized or not. If it is the latter case, users can opt for a private setting that only shows posts to selected followers, a common feature that many platforms already have.\n\nWe agree to the reviewer that, if a user wants to be completely anonymous, she can opt for a private channel. However, the scenario that we propose to study is different. We target social media users such as researchers, activists, whistleblowers who are active online and have some of their writings fully public to gain maximum influence and build personal images. However, they also do not want the adversaries to attribute their authorships from their anonymous writings in the future when they wish to write about sensitive topics. For example, researchers often post their papers publicly to be widely read [3] and activists spread their thoughts on social media. However, if they need to write anonymously, despite their previous writings being publicly posted, our model will help protect their authorships from being attributed.\n\nWe also want to emphasize that the number of perturbed texts in authors’ writings is minimal compared to the original texts and hence do not significantly change semantics of the original contents. We copied below a few examples of perturbed texts compared to baselines.\n\n--------------------------------------------------\nBackdoor Data Poisoning[1]:\n\nHa! I imagine that if anyone [...] She was very wzy charming and [...] pleasure of meeting. \"Butch knows best.\"\n\nTest Example: I agree: if you need a tough wzy looking [...]\n\n--------------------------------------------------\n\nNon-Targeted Data Poisoning:\n\nHa! I imagine that if anyone [..] had the pleasure of meeting. ’Butch knows best.\"\n\n--------------------------------------------------\n\nSample-Wide Data Poisoning:\n\nHa! I imagine that if anyone could convince a gay adult to judge being true, Ashley would [...] and chat to her [...] one of the dainty (and hottest) celebrities I have ever accept the pleasure of merging. \"Butch knows considerably.\"\n\n--------------------------------------------------\n\nClass-Wide Data Poisoning (Ours):\n\nHa! I imagine that if anyone [...] She was really charming and [...] pleasure of meeting. \"Butch knows best.\" \n\n--------------------------------------------------\n\n\n> Q5. Nowadays, it is proven that using a BlackBox approach, such as back-translation or GPT models, is more effective in maintaining good stealthiness compared to word-level synonym substitution. We understand the claim that synonym substitution is a cheaper alternative. However, in a real scenario, if a social media company wants to launch a feature that helps users polish their content, they might prioritize user experience over costs.\n\n> Q6. Building on the previous question, if more and more users adopt GPT-4 to polish their content, it raises the question of whether an attack model would still be able to detect user identity based on their posts. I encourage the authors to address the model's robustness by showcasing its performance on black box paraphrased content in an identical setting. This would help demonstrate the effectiveness and reliability of the proposed approach in real-world scenarios.\n\nFirst, we would like to clarify that our main focus is NOT to come up with novel ways to generate adversarial texts. In fact, we can apply any texts perturbation algorithms to Eq. (1) and Eq. (2) regardless whether it is word-level, sentence-level or GPT-based methods, as long as it is able to perform the targeted adversarial attack while preserving the original semantics.\n\nWe specifically selected TextFooler because it is a popular text perturbation method and has a competitive performance. We also argue that the computational cost required to use GPT models is unnecessary because our privacy protection comes from obfuscation of several data points at the same time, not to consider that the social media service needs to do this for a large portion of texts given the tremendous amount of newly posted contents on social media everyday. Given the current computational costs of GPT models, we respectfully disagree that GPT is more effective in our task. In addition, back-translation methods are proved to be unsuccessful in authorship obfuscation tasks [5]. We also run obfuscation experiments on GPT-3.5 and provide the following results.:\n\nSample 1======================================================\n\nOriginal text:\n\nHa! I imagine that if anyone could convince a gay man to try being straight, Ashley would be the one. She is gorgeous. I also met her at a convention and chatted to her for a while about her artwork . She was very charming and one of the nicest ( and hottest ) celebrities I have ever had the pleasure of meeting. \"Butch knows best.\"\n\n------------------------------------------------------------------------------------------------------------------------------------------------\n\nGPT-3.5 [Prompts: Can you help me polish this text to conceal the authorship: (Original text)?]:\n\nAmusingly, it's quite a thought that if there was ever someone capable of encouraging a person to consider a different orientation, this individual would fit the bill. Their appearance is striking. Our paths crossed at a convention, engaging in a lengthy conversation centered around their artwork. Their demeanor exuded a captivating charm, making the interaction one of the most enjoyable encounters with a public figure. \"Experience speaks volumes.\"\n\n------------------------------------------------------------------------------------------------------------------------------------------------\n\nOurs (UPTON class-wide):\n\nHa! I imagine that if anyone could convince a gay man to try being straight, Ashley would be the one. she is gorgeous. I also met her at a convention and chatted to her for a while about her artwork. She was really charming and one of the nicest ( and hottest ) celebrities I have ever had the pleasure of meeting. \"Butch knows best.\"\n\nSample 2=======================================================\n\nOriginal text:\n\nOkay, I'm glad that they made a sequel to the original film. In this film, Carol's first husband, Roy Martin, supposedly returns to reunite with his beautiful wife and three blonde daughters. Unfortunately, he's not who or what he appears to be. He seems trapped in Brady's world where cable television is far too out there for them and where dysfunctional seems to be the norm. Greg and Marcia get new digs in the attic with creepy sexual tension between them while Jan fakes a new boyfriend.\n\n------------------------------------------------------------------------------------------------------------------------------------------------\n\nGPT-3.5 [Prompts: Can you help me polish this text to conceal the authorship: (Original text)?]:\n\nAlright, it's satisfying to see that a sequel was created for the initial movie. This time around, the storyline involves Carol's former spouse, Roy Martin, supposedly rejoining his attractive wife and their trio of fair-haired daughters. However, things are not as straightforward as they seem. Roy finds himself ensnared in Brady's world, a place where unconventional cable television appears too unconventional, and dysfunctionality appears to be the norm. Meanwhile, Greg and Marcia find themselves in new accommodations in the attic, with an eerie undercurrent of ambiguous tension. Simultaneously, Jan fabricates a new romantic partner.\n\n------------------------------------------------------------------------------------------------------------------------------------------------\n\nOurs (UPTON class-wide):\nOkay, I'm happy that they made a sequel to the original movie. In this film, Carol's first husband, Roy Martin, supposedly returns to reunite with his beautiful wife and three blonde daughters. Unfortunately, he's not who or what he appears to be. He seems trapped in Brady's world where cable television is far too out there for them and where dysfunctional seems to be the norm. Greg and Marcia get new digs in the attic with creepy sexual tension between them when Jan fakes a new boyfriend.\n\n=============================================================\n\nWe can observe that GPT-3.5 tends to REWRITE the texts in it’s own words, which corrupted the original semantics could not perserve the original messages of the generated posts.\n\n> Typos Grammar Style And Presentation Improvements\n\nWe thank the reviewer for pointing out missing references. We will cite the paper mentioned in our final manuscript.\n\n\nReferences\n\n[1] Wenkai Yang, Lei Li, Zhiyuan Zhang, Xuancheng Ren,Xu Sun, and Bin He. 2021. Be careful about poisoned word embeddings: Exploring the vulnerability of the embedding layers in nlp models. arXiv preprint arXiv:2103.15543 (2021). \n\n[2] Fiammetta Marulli, Laura Verde, and Lelio Campanile. 2021. Exploring Data and Model Poisoning Attacks to Deep Learning-Based NLP Systems. Procedia Computer Science 192 (2021), 3570–3579.\n\n[3] Perera Molligoda Arachchige, Arosh S., and Niccolò Stomeo. 2023. Exploring the Opportunities and Challenges of ChatGPT in Academic Writing: Reply to Bom et al. Nuclear Medicine and Molecular Imaging (2023): 1-2.\n\n[4] Liu, Yixin, and Pengfei Liu. SimCLS: A simple framework for contrastive learning of abstractive summarization.\" arXiv preprint arXiv:2106.01890 (2021).\n\n[5] Mahmood, Asad, Zubair Shafiq, and Padmini Srinivasan. A girl has a name: Detecting authorship obfuscation. arXiv preprint arXiv:2005.00702 (2020).", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The authors' comprehensive experiments in three different settings demonstrate the practical applicability of their approach. Their method's superior performance over sample-wide label assigning and non-target label assigning provides strong support for their claim.\n\nThe paper is easy to follow and well-written. \n\n", "reasons_to_reject": "1. It is unclear how word-level synonym replacement can cause a distinct model (F_A) to change its predicted labels if the attack model is independent and inaccessible. It seems that F_A should be a frozen model without any parameter tuning.\n\n2. The main contribution of this paper appears to be the new label-assigning process for poisoned data, verified on three different settings. However, this may not be sufficient novelty for an EMNLP paper. Please correct me if I am missing something important.\n\n3. The paper only measures bertscore to illustrate the similarity of the newly generated text. However, there are many other matrices that could measure the stealthiness of the generated content, and none of them are discussed in the paper.\n\n4. I am a little concerned about why authors would want to replace their originally created content with randomly placed machine-generated text. This raises the essential question of whether users want to be recognized or not. If it is the latter case, users can opt for a private setting that only shows posts to selected followers, a common feature that many platforms already have.", "questions_to_authors": "Nowadays, it is proven that using a BlackBox approach, such as back-translation or GPT models, is more effective in maintaining good stealthiness compared to word-level synonym substitution. We understand the claim that synonym substitution is a cheaper alternative. However, in a real scenario, if a social media company wants to launch a feature that helps users polish their content, they might prioritize user experience over costs.\n\nBuilding on the previous question, if more and more users adopt GPT-4 to polish their content, it raises the question of whether an attack model would still be able to detect user identity based on their posts. I encourage the authors to address the model's robustness by showcasing its performance on black box paraphrased content in an identical setting. This would help demonstrate the effectiveness and reliability of the proposed approach in real-world scenarios.", "paper_topic_and_main_contributions": "This paper introduces a novel approach to make publicly released writings undetectable, providing potential privacy protection for the author while preserving the similarity between the rewritten text and the original text.\n\nTo achieve this goal, the authors propose perturbing the input text through synonym substitution. They then train the model with the poisoned new data and assign a different label to this new text data. This process prevents attackers from predicting the processed data with the correct authorship class.\n\nThe authors explore three different settings for the author identification problem. Their main contribution lies in the discussion of which new label should be assigned to the poisoned new data. Therefore, the comparison methods in this paper involve sample-wide label assigning (replacing the original label with a random new label) or non-target label assigning (assigning the label as any other label except the original one).", "excitement": 2, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Missing_References": "KALLIMA: A Clean-label Framework for Textual Backdoor Attacks"}, "extra_text": {}}, {"review_id": "cY7ZWBwTA8", "reviewer": "Reviewer_cjjY", "summary": "", "questions": "- If my understanding is correct, in the circle of defense all posts by user u are transformed so to look like they are posts by another user z (according to the F_B model). Even though the attacker can't reconstruct the real authorship of those posts, don't you think this still represents a risk for the users (as they could be \"blamed\" for posts written by a different activist)?\n- What is the quality of the perturbed posts?", "limitations": "", "rating": 3, "confidence": 1, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Summary:** The authors would like to thank the reviewer for their time and effort, especially for their cogent paper summary. We recognize that the main reasons for rejection are the assumptions of using transformer models as both $F_A$ and $F_B$ and the quality of perturbed posts. We have thoroughly explained the rationales behind such assumptions and provided additional experiment results, showing that using a different architecture, such as CNN for $F_A$ also warrants similar results. We also provided additional experiment results on semantic preservations of our method, showing the practicality of UPTON in practice. We respectfully urge the reviewer to consider these justifications in the final assessment. \n\nWe address all the concerns in detail below.\n\n> Q1. One of the main motivations for this paper is the proposal of a framework that does not require access to the attacker model, but in the end the framework relies on assumptions about said model (i.e., that it would be a transformers-based model, and that the representations learned by this model would be similar to those learned by the RoBERTa model trained by the authors). \n\nWe would like to clarify the reasons for using transformer-based models in both attacker’s model ($F_A$) and defender’s surrogate model ($F_B$). First, according to the relevant existing works, such as the state-of-the-art (SOTA) BertAA [1], transformer-based models outperform baselines in AA task. Thus, using the transformers is crucial for AA tasks and the attacker has incentives to adapt these SOTA transformers to build a AA model to achieve a high accuracy. \n\nSecond, in adversarial attack or poisoning works that requires transferability, existing works hold the point of view that the surrogate model ($F_B$ in UPTON) should be strong enough to extract informative features [3]. Therefore, the stronger F_B we have, the better performance UPTON has. Thus, as the defender, we also need a transformer-based surrogate model to ensure the high accuracy. That’s why $F_B$ also uses transformers. \n\nAlthough we believe that it’s reasonable for us to use a transformer-based model for $F_A$ and $F_B$, we found that UPTON is still transferable when the attacker uses CNN to build $F_A$ in our additional experiments. Here we show the results of our new experiments in a cold-start setting where $F_A$ has CNN structure as follows:\n\n========================================\n\n$\\qquad$  $\\qquad$  $\\enspace$   RoBERTa  $\\quad$  DistilBERT   $\\qquad$  CNN\n\n========================================\n\nWJO      $\\qquad$   $\\enspace$ $\\enspace$    29.39%     $\\qquad$    35.61%  $\\qquad$    25.35%\n\nIMDb10      $\\qquad$       41.64%   $\\qquad$      34.70%  $\\qquad$    34.11%\n\nIMDb62       $\\qquad$        31.27%    $\\qquad$     35.78%   $\\qquad$   24.86%\n\nEnron      $\\qquad$   $\\enspace$        34.55%   $\\qquad$      36.74%   $\\qquad$   30.28%      \n\n========================================\n\nWe can observe that **the transferability still applies to CNN -> BERT, RoBERTa although they have very different architectures**. We will include these results in the final manuscript.\n\n\n> Q2. Also I'm not sure about the quality of the perturbed posts, and whether it is realistic for a platform to apply this method before publishing content.\n\nWe want to clarify that the number of perturbed texts in authors’ writings is minimal compared to the original texts. We copied below a few examples of perturbed texts compared to other baselines.\n\n-------------------------------------------------------------------------\n\nBackdoor Data Poisoning[2]:\n\nHa! I imagine that if anyone [...] She was very wzy charming and [...] pleasure of meeting. \"Butch knows best.\"\n\nTest Example: I agree: if you need a tough wzy looking [...]\n\n-------------------------------------------------------------------------\n\nNon-Targeted Data Poisoning:\n\nHa! I imagine that if anyone [..] had the pleasure of meeting. ’Butch knows best.\"\n\n-------------------------------------------------------------------------\n\nSample-Wide Data Poisoning:\n\nHa! I imagine that if anyone could convince a gay adult to judge being true, Ashley would [...] and chat to her [...] one of the dainty (and hottest) celebrities I have ever accept the pleasure of merging. \"Butch knows considerably.\"\n\n-------------------------------------------------------------------------\n\nClass-Wide Data Poisoning (UPTON):\n\nHa! I imagine that if anyone [...] She was really charming and [...] pleasure of meeting. \"Butch knows best.\" \n\n-------------------------------------------------------------------------\n\nWe observe that non-targeted and class-wide (ours) have the best quality. We also use BERTScore Similarity to evaluate how much we change the original texts. Moreover, we add new experiments to get the USE (Universal Sentence Encoder) vectors of original and perturbed texts and then compute the cosine similarity between them. We display the average similarity (in [0,1]) of 500 original-perturbed sample pairs in each dataset. The new results are presented below.\n\n=========================================\n\nDatasets      $\\quad$    BERTScore Similarity    $\\quad$    USE Similarity\n\n=========================================\n\nWJO     $\\qquad$ $\\qquad$   $\\enspace$ $\\enspace$          95.32%          $\\qquad$   $\\qquad$   94.52%   \n\nIMDb10   $\\qquad$  $\\qquad$      97.30%     $\\qquad$     $\\qquad$    96.35%\n\nIMDb62    $\\qquad$  $\\qquad$    96.64%     $\\qquad$   $\\qquad$    97.00%\n\nEnron   $\\qquad$     $\\qquad$  $\\enspace$     97.02%      $\\qquad$   $\\qquad$    95.11%\n\n=========================================\n\nWe observe that both BERTScore and USE Similarity show that our method is **able to preserve the original semantics of the text documents very well**, making our framework practical to perturb contents before publishing. We will include USE Similarity evaluation in our final manuscript.\n\n> Q3. If my understanding is correct, in the circle of defense all posts by user u are transformed so to look like they are posts by another user z (according to the $F_B$ model). Even though the attacker can't reconstruct the real authorship of those posts, don't you think this still represents a risk for the users (as they could be \"blamed\" for posts written by a different activist)?\n\nWe would like to clarify our motivation and contributions. The most important threat of authorship attribution task to the users is the privacy leakage of private, sensitive texts that were intentionally written with anonymity, and our framework protects the users from this threat by only perturbing public, non-anonymous texts. Hence, the scope of our work is constrained to hiding the true authorship. However, we appreciate the suggestion and this can be one of the future works that we can extend to.\n\n> Q4.What is the quality of the perturbed posts?\n\nWe refer the reviewer to our responses to Q2.\n\n\n> Typos Grammar Style And Presentation Improvements:\n\nWe thank the reviewer for pointing out some format issues in our paper and we will address them in our final manuscript.\n\n\nReferences\n\n[1] Maël Fabien, Esaú Villatoro-Tello, Petr Motlicek, andShantipriya Parida. 2020. BertAA: BERT fine-tuning for Authorship Attribution. In Proceedings of the 17th International Conference on Natural Language Processing (ICON). 127–137.\n\n[2] Wenkai Yang, Lei Li, Zhiyuan Zhang, Xuancheng Ren,Xu Sun, and Bin He. 2021. Be careful about poisoned word embeddings: Exploring the vulnerability of the embedding layers in nlp models. arXiv preprint arXiv:2103.15543 (2021). \n\n[3] Petrov, Deyan, and Timothy M. Hospedales. Measuring the transferability of adversarial examples. arXiv preprint arXiv:1907.06291 (2019).", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- The paper focuses on an important problem.", "reasons_to_reject": "- One of the main motivations for this paper is the proposal of a framework that does not require access to the attacker model, but in the end the framework relies on assumptions about said model (i.e., that it would be a transformers-based model, and that the representations learned by this model would be similar to those learned by the RoBERTa model trained by the authors). Also I'm not sure about the quality of the perturbed posts, and whether it is realistic for a platform to apply this method before publishing content.\n---\nI thank the authors for the additional results shown in the rebuttal. I have increased the soundness score.", "questions_to_authors": "- If my understanding is correct, in the circle of defense all posts by user u are transformed so to look like they are posts by another user z (according to the F_B model). Even though the attacker can't reconstruct the real authorship of those posts, don't you think this still represents a risk for the users (as they could be \"blamed\" for posts written by a different activist)?\n- What is the quality of the perturbed posts?", "paper_topic_and_main_contributions": "The goal of the paper is to propose a framework (possibly to be used by publishing platforms) to protect authors identity from authorship attribution (AA) tools. This is particularly important for activists and whistle-blowers who are trying to protect their identity.\n\nExisting methods for authorship obfuscation require access to the AA models, and thus remain limited to unrealistic scenarios.\nThe framework proposed by the authors, UPTON, does not require access to the AA model and takes into account that some clean data with true authorship labels have already been publicly available (and therefore that AA models have already been trained on it).\nSpecifically, the authors validate UPTON in 3 scenarios:\n- cold start: no clean data is available\n- warm start: user has already posted writings to a platform that did not implement defensive mechanisms\n- incremental-release: the user posts regularly on a platform, so the attacker will continuously collect more data.\n\nTo bypass the need for access to the AA model, the authors train a state of the art (transformers based model) on the clean data available to them, and assume that the behaviour of such model would approximate to some extent the behaviour of the attacker model.\nPosts are perturbed by switching words to one of their synonyms in such a way that all the posts from users u would be mapped to a new label u* that occupies the part of the space where another label (e.g., z) was. In turn z will be mapped to a new label z* that occupied the part of the space of another author, and so on. The authors call this mechanism \"circle of defense\".\n\nThe authors run experiments with 4 datasets and show that UPTON decreases the accuracy of 2 AA models to about 35%.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Typos_Grammar_Style_and_Presentation_Improvements": "- Line 310: this citation shouldn't be within brackets.\n- Line 328: this citation shouldn't be within brackets (I just indicated a couple examples here, but the authors should re-check their use of \\cite and \\citep in the whole paper)."}}, {"review_id": "RgrQ9iJ9bw", "reviewer": "Reviewer_6F8X", "summary": "", "questions": "- Section 4 is the core of your proposed algorithm. Is there a reason why you don't provide any solid textual examples for each of the subsections  in section 4?. For example for adversarial generation you point to the paper from which the solution was taken. Minimally you should lead/end with some solid examples (preferably from the actual training data) for the class wide poisoning technique/algorithm you propose because that is your novelty. Without that the algorithm/pseudo code alone is hard to understand. In fact ideal scenario will be you take one training data sample and trace it through all the proposed solutions.\n\n- How about a situation where the writer/author always uses a pseudonym while publishing articles online and no one knows who the human is behind that pseudonym. I.e all the training data has no labels and this becomes an unsupervised training problem. Is that an assumption you make that this scenario will not occur/the author’s identity is always revealed/is tightly associated with his writings which serve to be used as training data for FA and FB? If yes, can you write that assumption explicitly somewhere. Or is it the same/encompassed in the cold-start? Also what percentage of training data is needed to successfully identify an author. For example, how many published work-author pairs, are needed before a trained model can start attributing an anonymous text to the author. If this is covered in prior art that needs to be explicitly mentioned.\n\n\n- In choosing the models used for FA and FB why did you choose stronger SOTA models like Roberta and Distilbert for the attacker and comparatively weaker model like BERT for the defender?Sure you can argue that even with a weaker model like BERT, UPTON achieved higher performance, but it makes you wonder why you are not comparing apples to apples. Am sure you have a valid reason for making this choice, but it just is not explicitly mentioned/evident from the text in its current form.\n\n- Have you considered an equivalent in Generative models? Some sort of fine-tuning?\n\n- I am left questioning the validity of using Enron to represent an author attribution dataset. Enron is really an email corpus. Granted it does show that your models do what they intended to do, but it really is not a reflection of real life right? Rather, emails are already explicitly earmarked coming from an author- and hence by definition moots the motivation of this paper: i.e an author wants to publish a writing and still have a model not automatically attribute it to him. Unless you are thinking of fake email ids etc. If yes, please mention that.\nThe reason you give for using Accuracy is that it is the most commonly used metric in authorship attribution. However, that alone really is not a strong metric when you claim that one model is better than the other. Minimally I would like to see how F1 does, if nothing else just to satisfy the reader’s immediate questions like: what if the recall was bad. Note: using semantic preservation metric was very impressive and well thought of.\n", "limitations": "", "rating": 5, "confidence": 4, "soundness": 5, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "The authors would like to thank the reviewer for their time and especially for all of the thoughtful questions and constructive suggestions. We would like to share our responses below.\n\n> Q1.Section 4 is the core of your proposed algorithm. Is there a reason why you don't provide any solid textual examples for each of the subsections in section 4?. For example for adversarial generation you point to the paper from which the solution was taken. Minimally you should lead/end with some solid examples (preferably from the actual training data) for the class wide poisoning technique/algorithm you propose because that is your novelty. Without that the algorithm/pseudo code alone is hard to understand. In fact ideal scenario will be you take one training data sample and trace it through all the proposed solutions.\n\n We agree that adding training examples to Section 4 is a good idea in addition to the training examples generated by non-target, sample-wide and class-wide already included in Table 4. We will move Table 4 to Section 4 , and refer to the samples in Table 4 when we show our algorithms in our final manuscript.\n\n> Q2. How about a situation where the writer/author always uses a pseudonym while publishing articles online and no one knows who the human is behind that pseudonym. I.e all the training data has no labels and this becomes an unsupervised training problem. Is that an assumption you make that this scenario will not occur/the author’s identity is always revealed/is tightly associated with his writings which serve to be used as training data for FA and FB? If yes, can you write that assumption explicitly somewhere. Or is it the same/encompassed in the cold-start? Also what percentage of training data is needed to successfully identify an author. For example, how many published work-author pairs, are needed before a trained model can start attributing an anonymous text to the author. If this is covered in prior art that needs to be explicitly mentioned.\n\nIn our opinion, UPTON works well because of the differences in distribution between the poisoned training datasets and clean private datasets. Hence, the attackers’ $F_A$ is trained on a poisoned dataset so it cannot attribute true authorships from a clean private dataset. We did not scope our paper in the context of unsupervised learning and hence pseudonym. However, we will explicitly mention this assumption in the final manuscript.\n\nAs for the number of texts needed for attacker to attribute authors, it varies for different datasets. We add below experiments to show the incremental training process on clean data and we will also add these results to the appendix.\n\n============================================\n\nDataset    $\\qquad$        Number of clean texts  $\\qquad$     Test Accuracy\n\n============================================\n\nWJO          $\\qquad$ $\\qquad$ $\\qquad$  $\\ $  120    $\\qquad$ $\\qquad$  $\\qquad$    31.43%\n\n$\\qquad$  $\\qquad$ $\\qquad$ $\\qquad$    240   $\\qquad$ $\\qquad$  $\\qquad$  45.16%    \n\n$\\qquad$  $\\qquad$ $\\qquad$ $\\qquad$                480     $\\qquad$ $\\qquad$  $\\qquad$    65.15%     \n  \n$\\qquad$  $\\qquad$ $\\qquad$ $\\qquad$              240    $\\qquad$ $\\qquad$  $\\qquad$    74.17%\n\n============================================\n\nIMDb10  $\\qquad$ $\\qquad$ $\\quad$ 100   $\\qquad$ $\\qquad$  $\\qquad$  67.10%\n\n$\\qquad$  $\\qquad$ $\\qquad$ $\\qquad$   200  $\\qquad$ $\\qquad$  $\\qquad$   93.11%       \n\n$\\qquad$  $\\qquad$ $\\qquad$ $\\qquad$  300   $\\qquad$ $\\qquad$  $\\qquad$   95.51%\n\n$\\qquad$  $\\qquad$ $\\qquad$ $\\qquad$   400  $\\qquad$ $\\qquad$  $\\qquad$  98.01%\n\n============================================\n\nIMDb62   $\\qquad$  $\\qquad$ $\\quad$    2K  $\\qquad$  $\\qquad$ $\\qquad$  49.22%\n\n$\\qquad$  $\\qquad$ $\\qquad$ $\\qquad$  4K   $\\qquad$  $\\qquad$ $\\qquad$    70.28%\n\n$\\qquad$  $\\qquad$ $\\qquad$ $\\qquad$  8K   $\\qquad$  $\\qquad$ $\\qquad$  78.86%\n\n$\\qquad$  $\\qquad$ $\\qquad$ $\\qquad$ 10K   $\\qquad$  $\\qquad$ $\\quad$  $\\ $  87.67%\n\n============================================\n\nEnron   $\\qquad$  $\\qquad$ $\\ $ $\\ $ $\\quad$ 4K  $\\qquad$  $\\qquad$ $\\qquad$    59.85%\n\n$\\qquad$  $\\qquad$ $\\qquad$ $\\quad$  $\\ $ $\\ $  8K  $\\qquad$  $\\qquad$ $\\qquad$  $\\ $ 76.03%\n\n$\\qquad$  $\\qquad$ $\\qquad$ $\\qquad$   16K  $\\qquad$  $\\qquad$ $\\qquad$  84.66%\n\n$\\qquad$  $\\qquad$ $\\qquad$ $\\qquad$   20K   $\\qquad$  $\\qquad$ $\\qquad$  88.91%\n\n============================================\n\n> Q3. In choosing the models used for FA and FB why did you choose stronger SOTA models like Roberta and Distilbert for the attacker and comparatively weaker model like BERT for the defender?Sure you can argue that even with a weaker model like BERT, UPTON achieved higher performance, but it makes you wonder why you are not comparing apples to apples. Am sure you have a valid reason for making this choice, but it just is not explicitly mentioned/evident from the text in its current form.\n\nWe actually have have done some experiments testing Roberta and Distilbert as $F_B$. However, the results we got are almost the same as the results when using BERT as $F_B$. Here we display some of the results on IMDb10 as an example. The following results are all class-wide target poisoning (UPTON) with 50% poison rate:\n\n============================================\n\n$\\qquad$  $\\qquad$   $\\quad$   BERT     $\\qquad$   RoBERTa  $\\quad$ $\\ $ DistilBERT\n\n============================================\n\nBERT    $\\qquad$  $\\quad$ $\\ $  —-      $\\qquad$  $\\ $    41.64%   $\\qquad$  $\\ $  34.70%       \n\nRoBERTa    $\\qquad$   32.99%    $\\qquad$     -—    $\\qquad$  $\\quad$ 40.28%\n\nDistilBERT   $\\quad$ $\\ $ 39.69%   $\\quad$   45.12%    $\\qquad$ $\\quad$ —-  \n\n============================================\n\nWe will also add these results to the Appendix, together with the results when the poison rate is 75% and 25%.\n\n> Q4. Have you considered an equivalent in Generative models? Some sort of fine-tuning?\n\nIn UPTON, we can use generative models in the adversarial generation process or use it as authorship attribution model $F_A$ and $F_B$. GPTs can be used for authorship obfuscation tasks. However, we did not use GPT-based obfuscation because of the following reasons: \n\nFirst, our main contribution is NOT to come up with novel ways to generate adversarial texts. In fact, we can apply any texts perturbation algorithms to Eq. (1) and Eq. (2) regardless whether it is word-level, sentence-level or GPT-based methods, as long as it is able to perform the targeted adversarial attack while preserving the original semantics.\n\nSecond, we also argue that the high computational cost required to run GPT models outweights any additional benefits they might provide. This is because our privacy protection comes from obfuscation of several data points at the same time, not to consider that the social media service needs to do this for a large portion of texts given the tremendous amount of newly posted contents on social media everyday. We also run obfuscation experiments on GPT-3.5 and provide the following results. \n\nSample 1======================================================\n\nOriginal text:\n\nHa! I imagine that if anyone could convince a gay man to try being straight, Ashley would be the one. She is gorgeous. I also met her at a convention and chatted to her for a while about her artwork . She was very charming and one of the nicest ( and hottest ) celebrities I have ever had the pleasure of meeting. \"Butch knows best.\"\n\n------------------------------------------------------------------------------------------------------------------------------------------------\n\nGPT-3.5 [Prompts: Can you help me polish this text to conceal the authorship: (Original text)?]:\n\nAmusingly, it's quite a thought that if there was ever someone capable of encouraging a person to consider a different orientation, this individual would fit the bill. Their appearance is striking. Our paths crossed at a convention, engaging in a lengthy conversation centered around their artwork. Their demeanor exuded a captivating charm, making the interaction one of the most enjoyable encounters with a public figure. \"Experience speaks volumes.\"\n\n------------------------------------------------------------------------------------------------------------------------------------------------\n\nOurs (UPTON class-wide):\n\nHa! I imagine that if anyone could convince a gay man to try being straight, Ashley would be the one. she is gorgeous. I also met her at a convention and chatted to her for a while about her artwork. She was really charming and one of the nicest ( and hottest ) celebrities I have ever had the pleasure of meeting. \"Butch knows best.\"\n\nSample 2=======================================================\n\nOriginal text:\n\nOkay, I'm glad that they made a sequel to the original film. In this film, Carol's first husband, Roy Martin, supposedly returns to reunite with his beautiful wife and three blonde daughters. Unfortunately, he's not who or what he appears to be. He seems trapped in Brady's world where cable television is far too out there for them and where dysfunctional seems to be the norm. Greg and Marcia get new digs in the attic with creepy sexual tension between them while Jan fakes a new boyfriend.\n\n------------------------------------------------------------------------------------------------------------------------------------------------\n\nGPT-3.5 [Prompts: Can you help me polish this text to conceal the authorship: (Original text)?]:\n\nAlright, it's satisfying to see that a sequel was created for the initial movie. This time around, the storyline involves Carol's former spouse, Roy Martin, supposedly rejoining his attractive wife and their trio of fair-haired daughters. However, things are not as straightforward as they seem. Roy finds himself ensnared in Brady's world, a place where unconventional cable television appears too unconventional, and dysfunctionality appears to be the norm. Meanwhile, Greg and Marcia find themselves in new accommodations in the attic, with an eerie undercurrent of ambiguous tension. Simultaneously, Jan fabricates a new romantic partner.\n\n------------------------------------------------------------------------------------------------------------------------------------------------\n\nOurs (UPTON class-wide):\nOkay, I'm happy that they made a sequel to the original movie. In this film, Carol's first husband, Roy Martin, supposedly returns to reunite with his beautiful wife and three blonde daughters. Unfortunately, he's not who or what he appears to be. He seems trapped in Brady's world where cable television is far too out there for them and where dysfunctional seems to be the norm. Greg and Marcia get new digs in the attic with creepy sexual tension between them when Jan fakes a new boyfriend.\n\n=============================================================\n\nWe can observe that GPT-3.5 tends to REWRITE the texts in it’s own words, which corrupted the original semantics could not perserve the original messages of the generated posts. \n\n> Q5. I am left questioning the validity of using Enron to represent an author attribution dataset. Enron is really an email corpus. Granted it does show that your models do what they intended to do, but it really is not a reflection of real life right? Rather, emails are already explicitly earmarked coming from an author- and hence by definition moots the motivation of this paper: i.e an author wants to publish a writing and still have a model not automatically attribute it to him. Unless you are thinking of fake email ids etc. If yes, please mention that. The reason you give for using Accuracy is that it is the most commonly used metric in authorship attribution. However, that alone really is not a strong metric when you claim that one model is better than the other. Minimally I would like to see how F1 does, if nothing else just to satisfy the reader’s immediate questions like: what if the recall was bad. Note: using semantic preservation metric was very impressive and well thought of.\n\nWe try to use datasets in different domains to evaluate UPTON thoroughly, e.g., WJO for online opinion publishers, IMDb for film reviews and Enron for business/daily emails. Actually, we pre-processed Enron dataset and removed all the names and email address information to make the evaluation more convincing.  We would like to clarify that we used accuracy in the paper because intuitively we think the attacker cares more about whether their model classifies authors correctly. Actually, F1 and accuracy have the similar results in our experiments as shown in the following results in the cold-start poisoning setting.\n\n==============================================================\n\nDatasets   $\\quad$    Poisoning Strategy   $\\qquad$    Acc  $\\quad$  $\\quad$ $\\ $ F1  $\\quad$  $\\quad$ $\\ $  Acc  $\\quad$  $\\quad$ $\\ $  F1\n\n==============================================================\n\nWJO  $\\qquad$ $\\ $ $\\ $ $\\ $ $\\ $  Class-wide $\\quad$  $\\qquad$    29.39% $\\quad$  27.90% $\\quad$ 35.61% $\\quad$  34.83% \n                \nIMDb10   $\\qquad$ $\\ $ Class-wide  $\\quad$  $\\qquad$    39.02%  $\\quad$ 39.14% $\\quad$ 34.79% $\\quad$  33.18%\n \nIMDb62   $\\qquad$ $\\ $ Class-wide  $\\quad$  $\\qquad$  31.27% $\\quad$  31.22%  $\\quad$ 35.78%  $\\quad$ 33.21%\n\nEnron   $\\qquad$ $\\quad$ Class-wide  $\\quad$  $\\qquad$   34.55%  $\\quad$ 34.62%  $\\quad$ 36.74% $\\quad$  36.10%\n\n==============================================================\n\n> Typos Grammar Style And Presentation Improvements\n\nWe will carefully inspect our paper according to all of the suggestions. We will add more descriptions on Figure 2 in our final manuscript. \n\nReferences\n\n[1] Ivan Evtimov, Pascal Sturmfels, and Tadayoshi Kohno. 2020. Foggysight: A scheme for facial lookup privacy. arXiv preprint arXiv:2012.08588 (2020).", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- This is a very interesting and novel work, in terms of the importance and contemporary relevance of the problem, proposed solution and the result it achieves. Will generate lots of interesting discussions.\n\n- Discussion section is very robust and has not only given a thorough analysis and proof of non-target poisoned text entering adjacent classes, but has also pre-emptively addressed many typical questions a reader would ask.\n\n- Work is overall well written and robust and definitely deserves a long paper acceptance, and I am recommending this for a best paper award.\n", "reasons_to_reject": "None", "questions_to_authors": "- Section 4 is the core of your proposed algorithm. Is there a reason why you don't provide any solid textual examples for each of the subsections  in section 4?. For example for adversarial generation you point to the paper from which the solution was taken. Minimally you should lead/end with some solid examples (preferably from the actual training data) for the class wide poisoning technique/algorithm you propose because that is your novelty. Without that the algorithm/pseudo code alone is hard to understand. In fact ideal scenario will be you take one training data sample and trace it through all the proposed solutions.\n\n- How about a situation where the writer/author always uses a pseudonym while publishing articles online and no one knows who the human is behind that pseudonym. I.e all the training data has no labels and this becomes an unsupervised training problem. Is that an assumption you make that this scenario will not occur/the author’s identity is always revealed/is tightly associated with his writings which serve to be used as training data for FA and FB? If yes, can you write that assumption explicitly somewhere. Or is it the same/encompassed in the cold-start? Also what percentage of training data is needed to successfully identify an author. For example, how many published work-author pairs, are needed before a trained model can start attributing an anonymous text to the author. If this is covered in prior art that needs to be explicitly mentioned.\n\n\n- In choosing the models used for FA and FB why did you choose stronger SOTA models like Roberta and Distilbert for the attacker and comparatively weaker model like BERT for the defender?Sure you can argue that even with a weaker model like BERT, UPTON achieved higher performance, but it makes you wonder why you are not comparing apples to apples. Am sure you have a valid reason for making this choice, but it just is not explicitly mentioned/evident from the text in its current form.\n\n- Have you considered an equivalent in Generative models? Some sort of fine-tuning?\n\n- I am left questioning the validity of using Enron to represent an author attribution dataset. Enron is really an email corpus. Granted it does show that your models do what they intended to do, but it really is not a reflection of real life right? Rather, emails are already explicitly earmarked coming from an author- and hence by definition moots the motivation of this paper: i.e an author wants to publish a writing and still have a model not automatically attribute it to him. Unless you are thinking of fake email ids etc. If yes, please mention that.\nThe reason you give for using Accuracy is that it is the most commonly used metric in authorship attribution. However, that alone really is not a strong metric when you claim that one model is better than the other. Minimally I would like to see how F1 does, if nothing else just to satisfy the reader’s immediate questions like: what if the recall was bad. Note: using semantic preservation metric was very impressive and well thought of.\n", "paper_topic_and_main_contributions": "This paper introduces UPTON - a framework for data poisoning technique and subsequent trained models, for addressing the problem of authorship attribution- or the lack of privacy there of when an attacker uses a trained model to identify/attribute an anonymous text to an author. The paper shows strong results in terms of how the proposed poisoning techniques reduce the accuracy of a model trained to do the same.  ", "excitement": 5, "reproducibility": 5, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Missing_References": "None I can think of for Authorship attribution/obfuscation.\n\nHowever this work has interesting overlaps with text steganography. The below papers should give you good leads towards more citations to add if any.\n\nKrishnan, R. Bala, Prasanth Kumar Thandra, and M. Sai Baba. \"An overview of text steganography.\" 2017 Fourth International Conference on Signal Processing, Communication and Networking (ICSCN). IEEE, 2017.\n\nDelina, B. \"Information hiding: A new approach in text steganography.\" Proceedings of the International Conference on Applied Computer and Applied Computational Science, World Scientific and Engineering Academy and Society (WSEAS 2008). 2008.", "Typos_Grammar_Style_and_Presentation_Improvements": "- Line 150 - citation, missing brackets\n\n- Caption of Figure 2 is very minimal. A figure must be self explanatory. Please explain minimally the meanings/definitions of terms mentioned in the figure. \n\n- Line 424 you say “Tab 2”, while the table is called “Table 2”. Please be consistent. Same for all tables.\n"}}]

Author rebuttal:

Meta-review: Thanks reviewers so much your efforts in providing comprehensive reviews and comments to improve the paper.

Thanks authors for providing actionable rebuttals and facilitating discussions.

Summary: This paper presents UPTON, a novel method to protect authors' privacy when their writings are published. UPTON uses synonym substitution to modify text, assigns different labels, and thwarts authorship attribution tools. It offers a framework for safeguarding authors, especially activists and whistle-blowers, without needing access to attribution models. Experiments show UPTON reduces attribution model accuracy to about 35%, making it an effective privacy solution.

Average Soundness: (2+3+5)/3 = 3.3
Average Excitement: (2+3+5)/3 = 3.3
Reproducibility: (3+4+5)/3 = 4

Summary of Pros:
+ The authors conducted comprehensive experiments in three different scenarios to validate their approach, which yielded superior results compared to alternative methods like sample-wide label assigning and non-target label assigning. 
+ The paper is well-written and tackles an important problem, making it both interesting and relevant. 
+ The discussion section is robust, addressing potential questions thoroughly. 

Summary of Cons:
+  It questions how word-level synonym replacement can affect a distinct model (F_A) when the attack model is independent and inaccessible. It suggests that F_A should be a static model without parameter tuning, indicating uncertainty about the model's behavior.
+ The main contribution as the label-assigning process for poisoned data could be limited.
+ Other metrics could be used
+ Further clarify the motivation to replace original content with machine-generated text, raising the question of whether users desire anonymity or recognition.
+ Further clarify the assumptions about the attacker model, such as it being a transformers-based model

The authors have provided a comprehensive rebuttal and proposed the changes to tackle the cons. The changes seem be feasible and can be timely ready for the final version.
Decision: Accept-Findings