Investigating User Perspectives on Differentially Private Text Privatization (2025)

Abstract: Recent literature has seen a considerable uptick in *Differentially Private Natural Language Processing* (DP NLP). This includes DP text privatization, where potentially sensitive input texts are transformed under DP to achieve privatized output texts that ideally mask sensitive information *and* maintain original semantics. Despite continued work to address the open challenges in DP text privatization, there remains a scarcity of work addressing user perceptions of this technology, a crucial aspect which serves as the final barrier to practical adoption. In this work, we conduct a survey study with 721 laypersons around the globe, investigating how the factors of *scenario*, *data sensitivity*, *mechanism type*, and *reason for data collection* impact user preferences for text privatization. We learn that while all these factors play a role in influencing privacy decisions, users are highly sensitive to the utility and coherence of the private output texts. Our findings highlight the socio-technical factors that must be considered in the study of DP NLP, opening the door to further user-based investigations going forward.

## introduction the pursuit of text privatization under the framework of differential privacy (dp) presents a promising, yet challenging task for researchers, who must balance the strong protections dp offers with the ability to retain meaningful utility from textual data @xcite . in recent years, numerous works at the intersection of data privacy and natural language processing, better known as privacy-preserving nlp or ppnlp, have tackled this challenge in various methods and techniques leveraging dp for text privatization @xcite . these range from word replacement methods @xcite , more advanced autoencoder-based methods @xcite , and recent works leveraging llms for privatization @xcite . addressing the technical challenges in realizing effective dp text privatization mechanisms has been at the forefront of researchers' goals in the recent literature. often, researchers proposing new methods must not only prove that a mechanism satisfies dp, but they must also empirically demonstrate that the mechanism can provide some tangible privacy benefit while also producing private texts that are useful and coherent @xcite . furthermore, operating in the domain of natural language introduces the complexities of syntactic hierarchy @xcite and meaningful privacy budgets @xcite , as well as clearly delineating the advantages of dp over traditional anonymization @xcite and maintaining reproducibility, explainability, and comparability @xcite . beyond these complexities, an under-explored aspect of dp in nlp remains measuring human perceptions of dp text privatization. very few works have extended past the research sphere to engage everyday users in investigating their perspective on what effective dp text privatization actually means. a recent work by @xcite opens the doors to this aspect, taking a risk-based approach in quantifying at which privacy budgets (or, the ε parameter) laypersons are comfortable in sharing their personal text data. here, it is shown that users are influenced by the perceived risk of misuse of their data, as they are less likely to consent to sharing with higher stated risks. despite its important role in leading off the study of human perceptions of dp nlp, we see a number of limitations in the work proposed by @xcite firstly, the risk perception approach taken by this work is useful in simplifying data sharing scenarios to laypersons, yet it makes no direct connection to actual outputs of privatization mechanisms, thus largely ignoring the crucial factor of language in text privatization. relatedly, the work only consid-ers the global dp setup, which distances itself from tangible privatization outputs and abstracts the dp notion away from local users. because of this, we gain little insight into user opinions and preferences of local privatization mechanisms, which comprise a large portion of the recent literature. we build upon the previous research of weiss et al. by focusing on these limitations, conducting a user study to investigate perceptions of text-totext privatization in various data sharing scenarios. we frame our user study in the form of vignettes, allowing for richer scenarios in which the users are placed. in these vignettes, we explore the influence of several important factors in local dp text privatization, including mechanism type, privacy budget, sensitivity of scenario, and reason for data collection. the choice of tested mechanisms is guided by a literature review of recent dp nlp works. our survey with 721 users from around the world yields interesting insights and perspectives on dp text privatization. above all, we find that the choice of privatization mechanism does matter, and users generally perceive mechanisms producing more coherent and natural outputs as preferable. if outputs are not so, users tend to choose less privacy in preference of "utility". finally, we find that sensitivity of scenario and reason for data collection are important, but not of primary concern. these findings provide a clear call to action for dp nlp researchers, namely to continue to study the perceptions of users, in order to align dp nlp research with real-world perspectives and needs. in this light, we make the following contributions: ## related work several recent works in dp nlp, although focusing on the technical aspect of the topic, point to the need for deeper consideration of the practical implications of dp text privatization. the work of @xcite critiques earlier wordlevel dp mechanisms, uncovering the issues of grammatical correctness and semantic coherence, a challenge more recent works have addressed @xcite @xcite . specifically considering syntactics, @xcite demonstrate the importance of granularity, or syntactic hierarchy, especially in real-world data sharing scenarios. quantifying and addressing these challenges becomes important to demonstrating the practical applicability of dp nlp @xcite , especially in light of more real-world challenges such as explainability and transparency @xcite @xcite , as well as reproducibility and comparability @xcite . particularly investigating the human aspect of text privatization, little work outside of @xcite has been performed. however, beyond the field of nlp, usable privacy research has been considerably more active in exploring user perspectives on dp. several works explore which communication methods are most effective in explaining dp to end users @xcite @xcite , which generally find that how dp is explained to users is important in fostering their understanding of the risks and implications. @xcite find that high-level abstractions of dp may lead to misunderstandings or false expectations about @xcite conclude that sometimes explanations have little effect on users' willingness to share data. interestingly, one work @xcite shows that local dp (ldp) concepts are more understandable than dp, and that in the ldp case, users exhibit more willingness to share data. we are motivated by these previous works in the dp field, particularly to provide more clarity on user perceptions of dp nlp methods. in light of the importance found by these previous works on the method of investigating user perspectives, we focus in this work on showing direct outputs of ldp text privatization mechanisms to users, in the form of understandable vignettes, as employed by @xcite for dp. with these, we are able to analyze different factors in the context of dp text privatization, particularly those affecting user perceptions of text privatization outputs. ## results we present the results of our user study, prefaced by our tested hypotheses and augmented by an analysis of our respondents' privacy preferences. ## discussion in light of the presented findings, we reflect on the lessons learned and discuss their implications. what matters with text privatization. our statistical analyses demonstrate that the factors of scenario, sensitivity, mechanism, and reason all play statistically significant roles in influencing a user's choices for text privatization, as indicated by the chi-square tests. however, these factors are not equally impactful, as we learn that the choice of dp mechanism is most important in swaying user perceptions of privacy options. in this, we provide empirical evidence that when dealing with natural language, it is also crucial how text is privatized. this above point is especially true in the case of text privatization with dp, where traditionally the ε is seen as an arbitrator between privacy and utility. the insights we gain from our user study imply that deciding privacy budgets in deployed systems may not be as simple as "more privacy needed, then lower ε" and vice versa; instead, one must take into account the methods and context in which privatization is to occur. while this potentially makes the task of dp text privatization more challenging, it also provides more criteria by which researchers and practitioners can justify their privacy budgets. ## conclusion we conducted a survey study with 721 participants from six continents, investigating the influence of various factors on user perception of dp text privatization. using a representative set of four dp mechanisms, we designed a series of vignettes to test for differences in the selection of text privatization level under a number of scenarios. we found that all tested factors play an important role in the context of text privatization, yet the factor of mechanism design is the most salient. in particular, mechanisms producing clearer and more natural outputs encourage users to choose higher privacy levels (lower ε budgets). our findings reveal the importance of involving the general population in guiding the direction of dp nlp research, and we hope that our work motivates future studies on aligning dp nlp research and practice.