DP -Rewrite: Towards Reproducibility and Transparency in Differentially Private Text Rewriting (2022)

Abstract: Text rewriting with differential privacy (DP) provides concrete theoretical guarantees for protecting the privacy of individuals in textual documents. In practice, existing systems may lack the means to validate their privacy-preserving claims, leading to problems of transparency and reproducibility. We introduce DP-Rewrite, an open-source framework for differentially private text rewriting which aims to solve these problems by being modular, extensible, and highly customizable. Our system incorporates a variety of downstream datasets, models, pre-training procedures, and evaluation metrics to provide a flexible way to lead and validate private text rewriting research. To demonstrate our software in practice, we provide a set of experiments as a case study on the ADePT DP text rewriting system, detecting a privacy leak in its pre-training approach. Our system is publicly available, and we hope that it will help the community to make DP text rewriting research more accessible and transparent.

## introduction protecting the privacy of individuals has been gaining attention in nlp. one particular setup is text rewriting using local differential privacy (dp) @xcite , which provides probabilistic guarantees of 'how much' privacy can be lost in the worst case if an individual gives us their piece of text that has been rewritten with dp. for instance, given a text "i want to fly from newark to cleveland on friday", the system may rewrite it as "flights from los angeles to houston this week". only a few recent works have touched on this challenging topic. for example, @xcite proposed adept: a text rewriting system based on the laplace mechanism. however, it turned out that their dp method was formally flawed @xcite . we also see another recent approach, dp-vae @xcite , which shows results that look surprisingly good for the level of guaranteed privacy. however, neither adept nor dp-vae published their source codes, so the community has no means to perform any empirical checks to validate the privacy-preserving claims. therefore, the lack of transparency and reproducibility is the main obstacle to the accountability of dp text-rewriting systems. we asked whether an open, modular, easily extensible, and highly customizable framework for differentially private text rewriting could help the community gain further insight into the utility and potential pitfalls of such systems. we hypothesize that by integrating various downstream datasets, models, pre-training procedures, and evaluation metrics into one software package, we improve the transparency, accountability, and reproducibility of research in differentially private text rewriting. our main contributions are twofold. first, this demo paper presents dp-rewrite, an opensource framework for differentially private text rewriting experiments. it includes a correct reimplementation of adept as a baseline, integrates pretraining on several datasets, and allows us to easily perform downstream experiments with varying privacy guarantees by adjusting the privacy budget ε. second, dp-rewrite allows us to easily detect another privacy leak in the approach proposed in adept, namely in the pre-training strategy of the autoencoder, with the system memorizing the input data. we demonstrate this in detail as a use case of dp-rewrite in section 4. foot_0 ## related work although the problem of simple data redaction is a widely researched field with several promising approaches @xcite , the related problem of private text transformation is still largely unexplored. we only briefly sketch the main ideas of local differentially private algorithms in text rewriting. for a longer introduction to dp see, e.g., habernal (2022); @xcite ; @xcite . let x, x ′ ∈ x be two data points such as texts or vectors, each belonging to a different person. in dp terminology, x and x ′ are neighboring datasets, as they differ by one person @xcite . a (local) dp algorithm m : @xmath0, such as a text or a vector. privatization is achieved by introducing randomness in m. as a result, (ε, 0)-local dp guarantees that for any two neighboring datasets x, x ′ and any output y even more current, @xcite proposed an end-to-end approach to text anonymization using a dp autoencoder, claiming to produce coherent texts of high privacy standards. however, several key aspects of the experiments lack a detailed description, while their results look surprisingly good. since the code base is not public, we cannot reproduce or reimplement their approach, and we cannot prove or refute our suspicions. ## description of software the goal of our system is to provide a seamless way to perform differentially private text rewriting, both on existing and custom datasets. a user can either load a dataset that we provide out-of-the-box, or use a custom one. in addition, we want to make it fast and convenient to run experiments for existing methodologies in dp text rewriting (e.g. adept), as well as the ability to integrate novel approaches. for this, we have a general autoencoder class based on which out-of-the-box and custom models are built. in this sense, our software is designed to be open and modular, where the researcher can swap out existing components to run a variety of experiments, as well as use the software for one's own privatized text rewriting needs. the core architecture of our system can be seen in figure 1 . we divide the experiments into three distinct modes, pre-training, rewriting, and downstream. for all three, the pipeline begins with a dataloader which can either be a dataset provided in the framework, or a custom dataset specified by the user. additionally, a rewritten dataset can be loaded for downstream experiments. the loaded dataset is then preprocessed according to user-specified parameters and the user's selected model, split into different procedures depending on the model type (e.g. rnn-based, transformer-based). the model is then initialized, optionally from an existing checkpoint. at this point, the main experiment is run based on the specified mode, either (1) pre-training the autoencoder, (2) using an existing checkpoint to rewrite the dataset, or (3) running a downstream model on an original or rewritten dataset. for each mode, a variety of evaluations are available, such as bleu @xcite and bertscore @xcite for pre-training and rewriting, and various classification metrics (e.g. f 1 score) for downstream experiments. the differential privacy component is incorporated during the rewriting phase for systems such as adept, although our framework also allows to incorporate it during the pre-training stage. ## case study we present here a case study that demonstrates the process of using our framework and provides insights into the adept system, for which we provide an implementation in the software. our goal is to investigate the difference in rewritten texts and downstream evaluations when we pre- pretrain dataloader preprocessor autoencoder model model pretraining pretrained model experiment configs, stats, evaluations load checkpoint save checkpoint selected model (e.g. adept) original dataset (prepared/custom) different preprocessing procedures depending on model type initialize model for experiment dp module rewrite dataloader preprocessor autoencoder model dataset rewriting pretrained model experiment configs, stats, evaluations load checkpoint selected model (e.g. adept) original dataset (prepared/custom) different preprocessing procedures depending on model type initialize model for experiment rewritten dataset downstr. dataloader preprocessor downstream model downstream training/eval downstream results, configs, stats rewritten dataset original dataset (prepared/custom) different preprocessing procedures depending on model type selected model (e.g. bert-downstream) initialize model for experiment train an autoencoder on one dataset and use this to rewrite another dataset. if we notice a lot of tokens from the dataset used for pre-training in the rewritten dataset, as well as comparatively higher downstream scores when pre-training and rewriting on the same dataset, then we can be certain of another form of privacy leakage in adept. ## conclusion we introduced dp-rewrite, an open-source framework for differentially private text rewriting experiments. we have demonstrated a sample usecase for our framework, which allows us to detect privacy leakage in the pre-training procedure of the adept system, an example of how the modular and customizable nature of the software allows for transparency and reproducibility in dp textrewriting research. dp-rewrite is continuing to be under active development, and we are incorporating new datasets and private text rewriting methodologies as they are released. we welcome feedback from the community.