SlowGuess/NAACL_2022_OCR huggingface dataset:
$$
\begin{array}{l} \mathbb {P} [ \mathcal {A} (D) \in S ] \leq \mathbb {P} [ \mathcal {A} \circ \operatorname {R e d a c t} _ {\pi} (D) \in S \subset E ^ {c} ] + \delta \\ \leq e ^ {\epsilon} \mathbb {P} [ \mathcal {A} (D ^ {\prime}) \in S \subset E ^ {c} ] + \delta_ {1} + \delta_ {2} \\ \leq e ^ {\epsilon} \mathbb {P} [ \mathcal {A} (D ^ {\prime}) \in S ] + \delta_ {1} + \delta_ {2}. \\ \end{array}
$$
A.3 More details on experiments
We choose the one-layer LSTM with an embedding size of 200 and a hidden size of 200. We choose distill-gpt2 as the GPT-2 model, which has 6 layers, 768 dimension and 12 heads. Vocabulary size for GPT-2 is 50257. Our experiments are conducted on NVIDIA TITAN-Xp GPU. For LSTM models, we tune the hyperparameters of the learning rate (lr) among \{20, 10, 5, 1, 0.1, 0.05, 0.01\}, batch size (bs) and the epochs among \{5, 10, 30, 50, 100\}. We finally choose  $\{\mathrm{lr} = 20, \mathrm{bs} = 256, \mathrm{epochs} = 50\}$  for Non-private-LSTM,  $\{\mathrm{lr} = 0.1, \mathrm{bs} = 5, \mathrm{epochs} = 50\}$  for S-DPSGD-LSTM and  $\{\mathrm{lr} = 0.05, \mathrm{bs} = 10, \mathrm{epochs} = 100\}$  for CRT-LSTM. The same set of hyperparameters are tuned for GPT model as well. Our final choice for DPSGD-GPT/CRT-GPT model is  $\{\mathrm{lr} = 5\mathrm{e} - 4, \mathrm{bs} = 256, \mathrm{epochs} = 10\}$ . The actual run-time of algorithms depends on implementation details. Here, we outline estimates of the run-time for training. Running one epoch on CRT-LSTM takes 2 hours whereas the same task on CRT-GPT only takes 30 minutes since the implementation of Li et al. (2021) is highly efficient. We use autodp<sup>8</sup>, an automating differential privacy computation for the privacy analysis. Noise scale  $\sigma$  is calculated numerically so that a DP budget of  $(\epsilon, \delta)$  is spent after  $T$  epochs.
A.4 Redaction policy details
We build the sequence labeling policy based on trimming one NER model trained on OntoNotes-5.0 (Weischedel et al., 2013) dataset. We modify the last layer of the NER model and set the threshold for the output scores to enable abnormal/sensitive data detection. For the screen policy  $\pi$ , we set the threshold to be 0.3 for all predictions with OntoNotes tags. For the conservative policy  $\pi_c$ , we select all predictions with tags and all plain texts with scores smaller than 0.9 to be sensitive data. We manually label 200 data points and find that the conservative policy  $\pi_c$  can achieve  $100\%$  recall with lots of false positives and that  $\pi$  can achieve  $90\%$  recall with few false positives.
<sup>7</sup>https://huggingface.co/distilgpt2
$^{8}$ https://github.com/yuxiangw/autodp
<sup>9</sup>https://huggingface.co/flair/ner-english-ontonotes-fast
A.5 Membership inference attack details
In our experiments, we manually construct a dataset with 2000 sequences. We select 1000 sequences from the protected secrets used in the training data. And we randomly generate 1000 samples of similar format which are not used in the training data. In this way, a random guess generates an accuracy of  $50\%$ . For MultiWoz 2.2, we use sentences with reference numbers as the secrets. For CustomerSim, we choose customer addresses as the secrets.
In order to show group confidentiality guarantees, we also conduct group membership inference attack. In this setting, we construct a dataset with 2000 groups, each of which includes 20 sentences. One half of the groups are "sensitive groups" with all 20 sentences drawn from protected secrets and the other half are "insensitive groups" with all 20 sentences being random. We build the classifier based on the sum of the perplexities in one group.
A.6 “The devil is in the details” – how things could go wrong with seemingly innocuous changes to the algorithm.
In this section, we highlight various aspects of our algorithms and why certain choices in the pre-processing steps need to be done in the specific way we recommend for our results to hold for them.
1. It is important that the definition of confidentiality is defined with respect to a perfectly redacted version of the dataset. If we define it as in selective differential privacy, then there will not be an amplification effect from redaction. This is because if we replace a secret  $x$  that can be detected by  $\pi$  with another  $x'$  that cannot be detected by  $\pi$ , then even if  $x$  is replaced with  $<\text{MASK}>$ ,  $x'$  will not be and the two datasets are still different after redaction. In addition, the S-DP definition will not be useful for us we do not know how to define a confidentiality parameter specific for each  $x$  or Bayesian confidentiality parameter for each  $\mu$
2. Tokenization and splitting into individual "sentences" (data points) should go before redaction / deduplication. Otherwise redaction with an approximate screening policy and with an ideal screening policy, or dedduplication may cause misalignments, resulting in almost all data points being different in the preprocessed version of  $D$  and  $D^{\prime}$ .
3. Each data point should contain only "whole" natural sentences, otherwise the sensitive part of a natural sentence could split into two data points.
4. Dedduplication steps should replace duplicate text with the same <MASK>, otherwise <MASK_Dedup> and <MASK_Reduct> are not the same so even if all secrets are masked, there will be a difference between the pre-processed versions of  $D$  and its neighbor, while in our approach there are no differences and we achieve perfect confidentiality (with  $\epsilon = 0$ ).
5. Any data point containing  $\langle \text{MASK} \rangle$  needs to be put in  $D^{pri}$ . This is because otherwise our algorithm that works on  $D'$  will be a deterministic algorithm that is perfectly distinguishable from the alternative world where the algorithm is random because the approximate policy  $\pi$  fails to redact certain secrets  $x$ .
6. In the DP-SGD algorithm, the sampled minibatches should contain the whole minibatch from  $D^{pri}$  or the whole minibatch from  $D^{pub}$ . Otherwise the noise always need to be added and the algorithm is identical to the vanilla DP-SGD, and there is no benefit of having a portion of the data being public comparing to all of the data are private.