SlowGuess/NAACL_2022_OCR huggingface dataset:
deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171-4186, Minneapolis, Minnesota. Association for Computational Linguistics.
Nemanja Djuric, Jing Zhou, Robin Morris, Mihajlo Grbovic, Vladan Radosavljevic, and Narayan Bhamidipati. 2015. Hate speech detection with comment embeddings. In Proceedings of the 24th international conference on world wide web, pages 29-30.
J. C. Duchi, M. I. Jordan, and M. J. Wainwright. 2013. Local privacy and statistical minimax rates. In 2013 IEEE 54th Annual Symposium on Foundations of Computer Science, pages 429-438.
Cynthia Dwork, Frank McSherry, Kobbi Nissim, and Adam Smith. 2006. Calibrating noise to sensitivity in private data analysis. In *Theory of Cryptography*, pages 265-284, Berlin, Heidelberg. Springer Berlin Heidelberg.
Mael Fabien, Esau Villatoro-Tello, Petr Motlicek, and Shantipriya Parida. 2020. Bertaa: Bert fine-tuning for authorship attribution. In ICON.
Christiane Fellbaum. 2010. Wordnet. In Theory and applications of ontology: computer applications, pages 231-243. Springer.
Natasha Fernandes, Mark Dras, and Annabelle McIver. 2019. Generalised differential privacy for text document processing. In International Conference on Principles of Security and Trust, pages 123-148. Springer, Cham.
Oluwaseyi Feyisetan, Borja Balle, Thomas Drake, and Tom Diethe. 2020. Privacy-and utility-preserving textual analysis via calibrated multivariate perturbations. In Proceedings of the 13th International Conference on Web Search and Data Mining, pages 178-186.
Oluwaseyi Feyisetan, Tom Diethe, and Thomas Drake. 2019. Leveraging hierarchical representations for preserving privacy and utility in text. In 2019 IEEE International Conference on Data Mining (ICDM), pages 210-219. IEEE.
J. Gorodkin. 2004. Comparing two K-category assignments by a K-category correlation coefficient. Computational Biology and Chemistry, 28(5):367-374.
Julian Hitschler, Esther van den Berg, and Ines Rehbein. 2017. Authorship attribution with convolutional neural networks and POS-eliciting. In Proceedings of the Workshop on Stylistic Variation, pages 53-58, Copenhagen, Denmark. Association for Computational Linguistics.
Fereshteh Jafariakinabad, Sansiri Tarnpradab, and Kien A. Hua. 2019. Syntactic recurrent neural network for authorship attribution. CoRR, abs/1902.09723.
Vlado Kešelj, Fuchun Peng, Nick Cercone, and Calvin Thomas. 2003. N-gram-based author profiles for authorship attribution. In Proceedings of the conference pacific association for computational linguistics, PACLING, volume 3, pages 255-264.
Yashwant Keswani, Harsh Trivedi, Parth Mehta, and Prasenjit Majumder. 2016. Author masking through translation. In Working Notes of CLEF 2016 - Conference and Labs of the Evaluation forum, Evora, Portugal, 5-8 September, 2016, volume 1609 of CEUR Workshop Proceedings, pages 890-894. CEUR-WS.org.
Moshe Koppel, Jonathan Schler, and Shlomo Argamon. 2011. Authorship attribution in the wild. Language Resources and Evaluation, 45:83-94.
Fragkiskos Koufogiannis, Shuo Han, and George J Pappas. 2015. Optimality of the laplace mechanism in differential privacy. arXiv preprint arXiv:1504.00065.
Xuechen Li, Florian Tramér, Percy Liang, and Tatsunori Hashimoto. 2021. Large language models can be strong differentially private learners.
Paul Pu Liang, Chiyu Wu, Louis-Philippe Morency, and Ruslan Salakhutdinov. 2021. Towards understanding and mitigating social biases in language models. In International Conference on Machine Learning, pages 6565–6576. PMLR.
Andrew L. Maas, Raymond E. Daly, Peter T. Pham, Dan Huang, Andrew Y. Ng, and Christopher Potts. 2011. Learning word vectors for sentiment analysis. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pages 142-150, Portland, Oregon, USA. Association for Computational Linguistics.
Sean MacAvaney, Hao-Ren Yao, Eugene Yang, Katina Russell, Nazli Goharian, and Ophir Frieder. 2019. Hate speech detection: Challenges and solutions. PloS one, 14(8):e0221152.
Asad Mahmood, Faizan Ahmad, Zubair Shafiq, Padmini Srinivasan, and Fareed Zaffar. 2019. A girl has no name: Automated authorship obfuscation using mutant-x. Proceedings on Privacy Enhancing Technologies, 2019(4):54-71.
B. W. Matthews. 1975. Comparison of the predicted and observed secondary structure of T4 phage lysozyme. Biochimica et Biophysica Acta (BBA) - Protein Structure, 405(2):442-451.
Frank McSherry and Kunal Talwar. 2007. Mechanism design via differential privacy. In 48th Annual IEEE Symposium on Foundations of Computer Science (FOCS'07), pages 94-103. IEEE.
George A Miller. 1995. Wordnet: a lexical database for english. Communications of the ACM, 38(11):39-41.
Ahmadreza Mosallanezhad, Ghazaleh Beigi, and Huan Liu. 2019. Deep reinforcement learning-based text anonymization against private-attribute inference. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 2360–2369.
Arjun Mukherjee, Vivek Venkataraman, Bing Liu, Natalie Glance, et al. 2013. Fake review detection: Classification and analysis of real and pseudo reviews. UIC-CS-03-2013. Technical Report.
Maximillian Nickel and Douwe Kiela. 2017. Poincaré embeddings for learning hierarchical representations. In Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc.
Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. GloVe: Global vectors for word representation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1532-1543, Doha, Qatar. Association for Computational Linguistics.
Zhan Qin, Yin Yang, Ting Yu, Issa Khalil, Xiaokui Xiao, and Kui Ren. 2016. Heavy hitter estimation over set-valued data with local differential privacy. In Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, pages 192-203.
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9.
Josyula R. Rao and Pankaj Rohatgi. 2000. Can pseudonymity really guarantee privacy? In 9th USENIX Security Symposium (USENIX Security 00), Denver, CO. USENIX Association.
Nils Reimers and Iryna Gurevych. 2019. SentenceBERT: Sentence embeddings using Siamese BERTnetworks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3982-3992, Hong Kong, China. Association for Computational Linguistics.
Natali Ruchansky, Sungyong Seo, and Yan Liu. 2017. Csi: A hybrid deep model for fake news detection. In Proceedings of the 2017 ACM on Conference on Information and Knowledge Management, CIKM '17, page 797-806, New York, NY, USA. Association for Computing Machinery.
Julian Seitner, Christian Bizer, Kai Eckert, Stefano Faralli, Robert Meusel, Heiko Paulheim, and Simone Paolo Ponzetto. 2016. A large DataBase of hypernymy relations extracted from the web. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC'16), pages 360-367, Porto Roz, Slovenia. European Language Resources Association (ELRA).
Emily Sheng, Kai-Wei Chang, Premkumar Natarajan, and Nanyun Peng. 2019. The woman worked as a babysitter: On biases in language generation. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3407-3412, Hong Kong, China. Association for Computational Linguistics.
Rakshith Shetty, Bernt Schiele, and Mario Fritz. 2018. A4NT: Author attribute anonymity by adversarial training of neural machine translation. In 27th USENIX Security Symposium (USENIX Security 18), pages 1633-1650.
Prasha Shrestha, Sebastian Sierra, Fabio A González, Manuel Montes, Paolo Rosso, and Thamar Solorio. 2017. Convolutional neural networks for authorship attribution of short texts. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers, pages 669-674.
Kai Shu, Amy Sliva, Suhang Wang, Jiliang Tang, and Huan Liu. 2017. Fake news detection on social media: A data mining perspective. ACM SIGKDD explorations newsletter, 19(1):22-36.
Congzheng Song and Ananth Raghunathan. 2020. Information Leakage in Embedding Models, page 377-390. Association for Computing Machinery, New York, NY, USA.
Congzheng Song and Vitaly Shmatikov. 2019. Auditing data provenance in text-generation models. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD '19, page 196-206, New York, NY, USA. Association for Computing Machinery.
Shuang Song, Kamalika Chaudhuri, and Anand D. Sarwate. 2013. Stochastic gradient descent with differentially private updates. In 2013 IEEE Global Conference on Signal and Information Processing, pages 245-248.
Chi Sun, Xipeng Qiu, Yige Xu, and Xuanjing Huang. 2019. How to fine-tune bert for text classification? In Chinese Computational Linguistics, pages 194-206, Cham. Springer International Publishing.
Michael Tschuggnall and Günther Specht. 2014. Enhancing authorship attribution by utilizing syntax tree profiles. In Proceedings of the 14th Conference of the European Chapter of the Association for Computational Linguistics, volume 2: Short Papers, pages 195-199, Gothenburg, Sweden. Association for Computational Linguistics.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc.
Jesse Vig, Sebastian Gehrmann, Yonatan Belinkov, Sharon Qian, Daniel Nevo, Yaron Singer, and Stuart Shieber. 2020. Investigating gender bias in language models using causal mediation analysis. In Advances in Neural Information Processing Systems, volume 33, pages 12388-12401. Curran Associates, Inc.
Stanley L Warner. 1965. Randomized response: A survey technique for eliminating evasive answer bias. Journal of the American Statistical Association, 60(309):63-69.
Benjamin Weggenmann and Florian Kerschbaum. 2018. SynTF: Synthetic and Differentially Private Term Frequency Vectors for Privacy-Preserving Text Mining, page 305-314. Association for Computing Machinery, New York, NY, USA.
Benjamin Weggenmann, Valentin Rubblack, Michael Andrejczuk, Justus Mattern, and Florian Kerschbaum. 2022. Dp-vae: Human-readable text anonymization for online reviews with differentially private variational autoencoders. In Proceedings of the ACM Web Conference 2022, WWW '22, page 721-731, New York, NY, USA. Association for Computing Machinery.
Sam Witteveen and Martin Andrews. 2019. Paraphrasing with large language models. In Proceedings of the 3rd Workshop on Neural Generation and Translation, pages 215-220, Hong Kong. Association for Computational Linguistics.
Alexandra Wood, Micah Altman, Kobbi Nissim, and Salil Vadhan. 2020. Designing access with differential privacy. In Cole, Dhaliwal, Sautmann, and Vilhuber, editor, Handbook on Using Administrative Data for Research and Evidence-based Policy, chapter 6.wayne. Last accessed on 2022-01-13.
Nan Xu, Oluwaseyi Feyisetan, Abhinav Aggarwal, Zekun Xu, and Nathanael Teissier. 2020a. Differentially private adversarial robustness through randomized perturbations. arXiv preprint arXiv:2009.12718.
Qiongkai Xu, Lizhen Qu, Chenchen Xu, and Ran Cui. 2019. Privacy-aware text rewriting. In Proceedings of the 12th International Conference on Natural Language Generation, pages 247-257.
Zekun Xu, Abhinav Aggarwal, Oluwaseyi Feyisetan, and Nathanael Teissier. 2020b. A differentially private text perturbation method using a regularized mahalanobis metric. CoRR, abs/2010.11947.
Zekun Xu, Abhinav Aggarwal, Oluwaseyi Feyisetan, and Nathanael Teissier. 2021. On a utilitarian approach to privacy preserving text generation. arXiv preprint arXiv:2104.11838.
Da Yu, Saurabh Naik, Arturs Backurs, Sivakanth Gopi, Huseyin A. Inan, Gautam Kamath, Janardhan Kulkarni, Yin Tat Lee, Andre Manoel, Lukas Wutschitz, Sergey Yekhanin, and Huishuai Zhang. 2021. Differentially private fine-tuning of language models.
Richong Zhang, Zhiyuan Hu, Hongyu Guo, and Yongyi Mao. 2018. Syntax encoding with application in authorship attribution. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2742-2753, Brussels, Belgium. Association for Computational Linguistics.
A.1 Information about terms of use for data
In this section, we provide information and references about the terms of use and licenses of each dataset we are using.
WordNet Wordnet can be downloaded and accessed online without specifically requesting access and can be used for research and also commercial applications in accordance with the WordNet 3.0 license: https://wordnet.princeton.edu/licenses-and-commercial-use
WebIsADB WebIsADB can be downloaded and accessed online without specifically requesting access. The dataset is licensed under a Creative Commons Attribution-Non Commercial-Share Alike 3.0 License: http://creativecommons.org/licenses/by-nc-sa/3.0/
IMDb IMDb movie reviews can be downloaded and accessed online without specifically requesting access. Unfortunately, we could not find information about license specifications. More information is available at https://ai.stanford.edu/~amaas/data/sentiment/
Yelp Researchers aiming to use the Yelp dataset have to sign the terms of use (https://s3-media3.fl.yelpcdn.com/assets/srv0/engineering_pages/bea5c1e92bf3/ assets/vendor/yelp-dataset-agreement.pdf).
For commercial use, researchers should contact Yelp via dataset@yelp.com. More information is available at https://www.yelp.com/dataset.
Table 2: Exemplary output of anonymization mechanisms for Yelp data
Exemplary Reviews for Yelp
This store is so adorable. In addition to baked goods they offer sandwiches for breakfast and lunch. The turkey sandwich was excellent. The textures were perfect though, especially for the almond amaretto cookie. It had the right balance of chewy with a slight amount of crunch.
Euclidean embedding perturbations  $(\epsilon = 8)$ :
designated store is work adorable making top tubular continue watered goods do offer salad ranging breakfast filling 5,000-a carries original turkey sandwich was excellent parts national textures were play never neighbors with for part mustard amaretto cookie hatred make had a direction balance end sugary another a erratic amounts of one-off today
Euclidean embedding perturbations  $(\epsilon = 10)$ :
fact store is 're granny his in health they dish goods kept offer sandwiches giving dinner besides lunch result the turkey sandwich given delivering . the textures ten captures . then especially own the apricot izola cookie on be had the right footing of chewy with a slight amount in crunch at
Poincaré embedding perturbations  $(\epsilon = 1)$ :
this flag is so adorable . in abundance to waffles chunk they many slimy for eggs and peppery . the vindaloo stickers was excellent . the blt were splurge gun , especially for the quail amaretto crunch . it had the quantity observation of crisp with a trotter simple of crunch .
Poincaré embedding perturbations  $(\epsilon = 2)$ :
this patient is so adorable . in stuff to asparagus walk-up they con jets for pricy and tamale .the petite cook was excellent . the rang were steal train , especially for the updated amaretto soak .it had the say many of containing with a slight land of cans .
Paraphrased  $(\epsilon = 0.1)$ :
There is a cute store. There is a sandwich being served by the sandwich shop. The sandwich is tasty. The two textures are alike. There were chews on the chem.
Paraphrased  $(\epsilon = 1)$ :
There's adorable store in this photo. In addition to baked goods they offer sandwiches for breakfast and lunch. This was a great sandwich. The desserts taste delicious! The food was chewy.
Table 3: Results for hyperbolic perturbations (Feyisetan et al., 2019) when removing out-of-vocabulary words.
Privacy budget ε	original	Poincaré embeddings
∞	0.5	1	2	8
IMDb:
Author MCC (static)	0.98	0.03	0.12	0.07	0.12
Author MCC (adapt.)	0.98	0.67	0.69	0.68	0.69
Sentim. MCC (static)	0.71	0.27	0.30	0.31	0.28
Sentim. MCC (adapt.)	0.71	0.35	0.40	0.38	0.39
SBERT CS	1.00	0.32	0.33	0.33	0.34
Yelp:
Author MCC (static)	0.80	0.14	0.14	0.15	0.16
Author MCC (adapt.)	0.80	0.32	0.35	0.34	0.36
Sentim. MCC (static)	0.51	0.17	0.18	0.20	0.20
Sentim. MCC (adapt.)	0.51	0.21	0.23	0.24	0.25
SBERT CS	1.00	0.54	0.54	0.56	0.57