Privacy-Preserving Natural Language Processing (2023)

Abstract: This cutting-edge tutorial will help the NLP community to get familiar with current research in privacy-preserving methods. We will cover topics as diverse as membership inference, differential privacy, homomorphic encryption, or federated learning, all with typical applications to NLP. The goal is not only to draw the interest of the broader community, but also to present some typical use-cases and potential pitfalls in applying privacy-preserving methods to human language technologies.

## introduction human language technologies play an essential role in the modern society. from automatic machine translation to drug discovery, nlp has had an undeniable impact on everyone's life. however, many of the recent achievements of state-of-the-art models come at a price that everyone must pay. in the race for yet better performing systems, the research has completely ignored the fact that within the extreme amounts of data needed for the 'hungry' models, there are private information of actual living persons @xcite . our sensitive information -be it explicitly mentioned in the texts we or someone else writes about us, or implicitly in our writing style -is at stake with current nlp models. privacy matters a lot to society, but has been largely neglected by nlp researchers. this tutorial aims to close this gap by offering the community insights into state-of-the-art approaches to privacy-preserving nlp. we will cover diverse topics, such as membership inference, differential privacy, homomorphic encryption, or federated learning, all with typical use-cases and applications. the tutorial will try to balance theoretical foundations with practical considerations. ## tutorial outline we propose a half-day tutorial (3 hours) divided into four following thematic blocks. ## tutorial specifics the proposed tutorial is considered a cutting-edge tutorial that presents recent advances in an emerging area of privacy-preserving techniques for nlp. the topic presented has not been covered in previous acl*-family tutorials in the last 4 years. we estimate that at least 60% of the papers covered in this tutorial are from researchers other than the instructors. it is also different from other tutorials, e.g., on differentially-private machine learning, as we target nlp with all its peculiarities related to human language. the preferred venue for this tutorial would be 1) acl, 2) eacl, 3) emnlp. we prefer acl due to travel arrangements of presenters located in the u.s. based on the raising interest in this topic, we expect around 30 participants. the tutorial will be self-contained, however attendees should have solid background in basic deep learning technologies in nlp (representations, architectures, optimization) foot_0 and to brush up knowledge of probability and statistics (laplace or gaussian distributions and probability bounds). 2 we are committed to open-source all teaching materials under permissible license.