Privacy-Preserving Domain Adaptation of Semantic Parsers
Year: 2023

Abstract:
Task-oriented dialogue systems often assist users with personal or confidential matters. For this reason, the developers of such a system are generally prohibited from observing actual usage. So how can they know where the system is failing and needs more training data or new functionality? In this work, we study ways in which realistic user utterances can be generated synthetically, to help increase the linguistic and functional coverage of the system, without compromising the privacy of actual users. To this end, we propose a two-stage Differentially Private (DP) generation method which first generates latent semantic parses, and then generates utterances based on the parses. Our proposed approach improves MAUVE by 2.5× and parse tree function type overlap by 1.3× relative to current approaches for private synthetic data generation, improving both on fluency and semantic coverage. We further validate our approach on a realistic domain adaptation task of adding new functionality from private user data to a semantic parser, and show overall gains of 8.5% points in accuracy with the new feature.

heading: 1 Introduction
text: In task-oriented dialogue systems, such as Siri and Alexa, a software agent parses a user’s intent into a program, executes it and then communicates the results back to the user (Andreas et al., 2020; Li et al., 2022; Cheng et al., 2020; Gupta et al., 2018; Young et al., 2013). As a result of their growing popularity, these systems face an increasing demand to improve their linguistic coverage (How do users talk?) as well as functional coverage (What are users trying to do?). An input utterance to such a system could look like this: “Could you tell me what the weather is gonna be like today in
∗ Work done as part of a Microsoft Semantic Machines internship. Corresponding author email: fatemeh@ucsd.edu
New York?” and the agent answers it by predicting and executing a program, or semantic parse.
In many cases, due to privacy controls, system developers can only use datasets that are limited and contrived, e.g., dialogues created by crowd workers pretending to be users. This is a significant domain shift from real private user data. Unlabeled data from real user interactions with dialogue systems has abundant signals that could be used to improve the linguistic and functional coverage of semantic parsers. For instance, practitioners could detect gaps in coverage by examining interactions where the system was unsure about what to do or responded that it lacked the requested functionality.
Note that training on real user interactions would be problematic for privacy even if automated and unsupervised. Trained models can “memorize” details of their training data (Tirumala et al., 2022; Feldman, 2020; Mireshghallah et al., 2022b), and this can be exploited through different types of attacks that either extract full training sequences from models (Carlini et al., 2021, 2019) or infer the presence of a given sequence of interest in the training data (Mireshghallah et al., 2022a).
To mitigate that problem, Differentially Private (DP) training algorithms, such as DP-SGD (Abadi et al., 2016; Dwork et al., 2006), can be used to provide worst-case guarantees on the information leakage of a trained model. This guarantee is controlled by the privacy budget ϵ, where lower epsilon means higher privacy. But while DP-SGD could be used to adapt (fine-tune) a semantic parser on unannotated private data, there is a limit to what can be done in this way. Even if some users are asking the system to hop up and down, fine-tuning is unlikely to make it grow legs. Thus, our goal in this paper is to use DP-SGD to produce realistic data that can be inspected (so that the developers know to build legs) and expertly annotated (to
4950
rapidly teach the semantic parser that words like “hop” and ”jump” should invoke the leg API).
Recent concurrent work (Yue et al., 2022; Mattern et al., 2022) has attempted to generate synthetic labeled data to train a text classifier. They sample labels from a given distribution and sample corresponding text strings from a differentially private language model, conditioning each string on its label. This approach does not directly apply to our setting, as our distribution over labels is private and one of our goals is to learn it. We cannot even substitute a uniform distribution for the sake of generation, since no such distribution exists over the infinitely many labels of the semantic parsing task.
To meet the unique requirements of our setting, a simple baseline is to ignore the classes and simply train a differentially private language model on all the private utterances, applying DP-SGD to the usual log-likelihood objective (Figure 1). We could then sample synthetic utterances for inspection and annotation. However, we find that when we enforce privacy with budget ϵ = 3, this baseline’s top-25 most common function types1 only have 64% overlap with the top-25 most common function types in the private utterances (see Section 5.1 and Table 6 for full experimental details and results). In other words, the baseline model does not accurately capture the distribution of the private training data, over a limited number of synthesized samples.
To ensure sufficient coverage of how users invoke different function types, we propose a 2-stage method (Figure 2) that exploits the structure of the output space, by privately (using DP-SGD) modeling the parse trees (bottom of the figure) and the conditional distribution of the utterances given a parse tree (top of the figure), separately. These models can then be used to generate as many samples as desired, by first sampling parse trees from the parse generation model and then prompting
1Function types are the set of unique, discrete node labels in the semantic parse tree, corresponding to API methods.
the parse2utterance model with these parse trees. Using the proposed method, we observe a 80% coverage of the top-25 most common function types in private utterances, which is a significant improvement over the baseline (64%). To further evaluate the efficacy of our method in improving a downstream system, we annotate DP-generated utterances to add a missing functionality to a lowresource semantic parser, and show that the parser’s accuracy on this missing functionality is 13.4% higher if the DP-generated utterances come from our 2-stage method rather than the 1-stage baseline.

heading: 2 Background
text: Definition 2.1 (Differential Privacy (DP) (Dwork et al., 2006)). A randomized algorithmA is (ϵ,δ)differentially private if for any two neighboring inputsD andD′, which differ in exactly the data pertaining to a single record, and for any set S of possible outputs: Pr[A(D)∈S]≤eϵPr[A(D′)∈S]+δ.
To train a neural network with differential privacy, the most widely used algorithm is the DP variant of stochastic gradient descent, DP-SGD (Abadi et al., 2016). It resembles ordinary SGD, but at each gradient update step, it first clips the per-example gradient to a maximum norm of C, then obfuscates it by adding Gaussian noise with mean 0 and standard deviation σC. Intuitively, this limits the contribution that any single example makes to the final model parameters returned by the training algorithmA. The privacy expenditure of DP-SGD, (ϵ,δ), is a function of C, σ, |B| (batch size), |D| (dataset size), and the total number of epochs T (which controls the total number of gradient updates). It is determined based on the Rényi DP (Mironov, 2017) privacy accounting method. In practice, following prior work, we fine-tune our models using DP-Adam (Abadi et al., 2016; Li et al., 2021; Yu et al., 2021). We elaborate more on DP training in Appendix C. Post-processing property. This property of DP ensures that if an algorithm A satisfies (ϵ,δ)-DP,
2
then so does F ◦ A for any function F (Dwork et al., 2006). This means that we can take as many samples as we want from the DP-trained models, without changing the privacy expenditure.
Algorithm 1 Differentially Private Training and Sampling: 1-stage Baseline Input: Utterance generation model pθx(x), private
utterances Dpriv, privacy budget (ϵ,δ), batch size |B|, epochs T , clipping threshold C, privacy accountantA 1: Feed the parameters ϵ, δ, C, T , and |B| to the accountantA to get noise multiplier σ 2: Fine-tune pθx(x) on Dpriv with DP-SGD and parameters σ and C for T epochs with batch size |B| 3: Populate Ds with samples from pθx 4: return Ds

heading: 3 Method
text: Setting. We are given an unlabeled private dataset Dpriv={xi} drawn from the distribution of private user utterances, ppriv(x). We are also given a labeled public dataset Dpub = {(xj , yj)}, and a semantic parser pϕ0(y |x) already trained on Dpub. Goal. Our goal is to find an (ϵ,δ)-DP model ϕ, such that pϕ(y |x) has a lower loss than pϕ0(y |x) on the task of semantically parsing utterances from ppriv. We will achieve this by proposing different methods of using Dpriv with DP to train a model pθ of ppriv, and using it to synthesize an unlabeled dataset, Ds, that can then be manually annotated and used to augment the training set Dpub for learning ϕ.
In the rest of this section, we first introduce the baseline method for pθ(x), then we propose our 2-stage method. In both cases, the output of the pro-
cess is a dataset of unlabeled synthetic utterances, Ds, which is similar to Dpriv and will later be annotated and used to augment Dpub when training ϕ. Due to the post-processing property (see Section 2), any model trained on Ds still satisfies (ϵ,δ)-DP.
Algorithm 2 Differentially Private Training and Sampling: Proposed 2-stage Technique Input: Parse tree generation pθy(y) and
parse2utterance pθyx(x | y) models, private utterances Dpriv, privacy budget (ϵ, δ), batch size |B|, T1 and T2 as epochs allocated to stages 1 and 2, clipping threshold C, privacy accountant A, low-resource parser pϕ0(y | x) trained on Dpub 1: Feed the parameters ϵ, δ, C, T =T1+T2, and |B| to the accountantA to get noise multiplierσ 2: FeedDpriv topϕ0 to sample a parse tree for each utterance, and augment Dpriv with the trees 3: Fine-tune pθy(y) on only parse trees from Dpriv with DP-SGD and parameters σ and C for T1 epochs with batch size |B| 4: Fine-tune pθyx(x | y) on Dpriv with DP-SGD and parameters σ and C for T2 epochs 5: Take samples from pθy(y) with batch size |B| 6: Prompt pθyx(x |y) with the sampled parse trees
from the previous step, and populate Ds with the output samples.
7: return Ds

heading: 3.1 Baseline: Vanilla DP Language Model
text: Figure 1 shows the 1-stage baseline approach of fine-tuning a pre-trained generative auto-regressive language model on private user utterances using DP-SGD (Abadi et al., 2016; Li et al., 2021; Yu et al., 2021). Algorithm 1 details this process: To
3
create the synthesized dataset, we first fine-tune the initial utterance model pθx on Dpriv, using DP-SGD with noise multiplier σ, batch size |B| and clipping threshold C, for T epochs. The noise multiplier is given to us by the privacy accountant (see Section C.1). Then, we take samples from this fine-tuned model. Due to the post-processing property of DP, any sample from this model satisfies the guarantees and the synthesized dataset size does not affect the privacy guarantees.

heading: 3.2 Proposed Method: 2-stage Generation
text: As we discuss in the introduction and the results sections, the baseline fails to accurately capture the distribution of different word-types (vocabulary) and parse tree function-types in the private utterances. To increase the linguistic and functional coverage of the synthesized data, we propose a 2-stage method to exploit the inherent structure in the private user-utterance parse trees. We assume that the parse trees for the private utterances are also private, since they are almost unique to each utterance, and cannot be released/used without privacy measures. We also assume that the parse trees are noisy, e.g., generated by low-resource parsers rather than expert annotators, and we use such trees in our end-to-end experiments. Figure 2 shows an overview of our proposed 2-stage generation model, with the bottom part showing (in blue) one stage of the process, training a parse tree generation model, with DP-SGD, to model the parse trees as intermediate (latent) variables. A model like p(x) = ∑ y p(y)p(x|y) is a latent variable model (for example, a variational autoencoder) of x, where we can sample x, by first drawing y, then drawing x given y, then discarding y. That is what we have done, with y being a parse-tree and x being an utterance. The other stage is training a parse2utterance model that would take the intermediate variables (parse trees, y) as input prompts and produce the utterance corresponding to them. It is noteworthy that the training of these two stages is completely independent and can be parallelized.
Algorithm 2 shows the details of the training and sampling process. An important design choice is how to split T , the number of overall training epochs, between the two stages as T = T1 + T2. This effectively splits the privacy budget: the stage that gets more epochs consumes more of the privacy
budget. We discuss this further and run ablation studies in Section 5.3.
We choose to use the same privacy parameters (σ and C) determined from T for training of both stages, as this enables us to use the sophisticated privacy accountant of Rényi DP (RDP) (Mironov, 2017) and achieve a tighter bound on the privacy parameter ϵ, compared to directly splitting ϵ and using different privacy parameters for each stage. The privacy accountant keeps track of how much privacy budget ϵ (information that an adversary can recover about a training sample) has been spent so far during the training process. We elaborate more on this choice in Appendix C.2.

heading: 4 Experimental Setup
text: In this section we briefly describe our experimental setup. For full experimental details see App. A.

heading: 4.1 Datasets
text: We use two large-scale conversational semantic parsing datasets, SMCalFlow v2.0 (Andreas et al., 2020) and TreeDST (Cheng et al., 2020). We pre-process them to break the conversations with the agents into single turns, each consisting of an (utterance, parse tree) pair, and we only use the human turns. This yields a training/test dataset with size of 133,584/14,571 and 121,652/22,897 for SMCalFlow and TreeDST respectively.

heading: 4.2 Metrics
text: We compare the synthesized datasets to human utterances on a distribution level, as there is no one-to-one mapping between them. We report two sets of metrics here: (1) Language and (2) Parse metrics. Language metrics (MAUVE and word overlap) are measured from the generated utterances themselves, whereas parse metrics (chi-square distance and function overlap) are measured from parse trees corresponding to the synthesized utterances, produced using a high-resource parser.2 All the results are reported over 14, 751 and 22, 897 synthesized utterances, compared to the same number of human utterances, for SMCalFlow and TreeDST datasets respectively. MAUVE (Pillutla et al., 2021) is a comparison measure for open-ended text generation, which directly compares the learned distribution from
2A parser trained on a large amount of data, close to SotA accuracy.
4
a text generation model to the distribution of human-written text using divergence frontiers.3 Word-type Overlap (W. Overlap) measures the word-type (vocabulary) coverage of generated text, as the ratio of overlapping types between the generated text and human utterances, against the human utterances, when the text is tokenized using spaces. Function Type Overlap (F. Overlap) measures the function type coverage of the parse trees of the synthesized text, as the ratio of overlapping parse tree API function types between the generated text and human utterances, against the human utterances. Chi-square Distance (Dist.) measures the χ2 distance between the distribution of API function types from the parse trees of the synthesized utterances against the parse trees of human utterances.

heading: 4.3 Model Architectures and Decoding
text: Semantic Parser. We train a Transformer-based semantic parser from Zhou et al. (2022) on only the human turns from the dialogues (without any context). Details are provided in Appendix A.3. 1-stage and 2-stage models. We fine-tune a pretrained GPT-2 (small) from Hugging Face (Wolf et al., 2019) for all three of the utterance generation, parse tree generation, and parse2utterance models. We also provide results for GPT2-Large in the Appendix B.5. Decoding. We use Hugging Face’s multinomial beam search (beam_sample) with beam width of 1 for decoding from the parse-generation model and beam width of 5 for decoding from parse2utterance and utterance generation models, as found in our hyperparameter search.

heading: 4.4 Baselines
text: 1-stage. This baseline denotes the method explained in Section 3.1, where we fine-tune a pre-trained language model (GPT2-Small) with DPSGD (Abadi et al., 2016), on the utterances. This baseline is essentially equivalent to the methods proposed in Li et al. (2021); Yu et al. (2021). 1-stage + domain (1.5-stage). For further evaluation of our method, we devise a more sophisticated baseline, inspired by Yue et al. (2022), where we augment the 1-stage model with a constrained set of prompts that reflect the domain of the modeled utterance. As such, we fine-tune GPT2-Small with DP-SGD to create a domain2utterance model on the
3 https://github.com/krishnap25/mauve
TreeDST dataset, which uses the domain label (10 domains: flight, hotel, etc.) instead of the parse tree. We sample domains from the true domain distribution. This is also similar to the setup in Mattern et al. (2022), though they only target classification tasks.

heading: 5 Experimental Results
text: In this section we first compare the baseline and our proposed method. We then study possible reasons for the observed superiority of the proposed 2-stage technique, and analyze hyperparameter sensitivity for the generation process. Finally, we compare the performance of the baseline and the proposed method on improving the performance of a downstream semantic parser. We provide additional experiments and ablations (such as hyperparameter sensitivity and detailed result break-downs) in Appendix B alongside sample synthesized utterances and parse-trees in Tables 9 and 10, as a reference.

heading: 5.1 Comparison with Baselines
text: Table 1 shows a comparison of the 1-stage and 1-stage + domain (1.5 stage) baselines (Section 4.4) with our proposed 2-stage method, for three different privacy budgets of ϵ= {3,8,16}, for the two datasets SMCalFlow and TreeDST. We present results for the 1-stage + domain baseline only on the TreeDST dataset, since we have the domain annotations only for this dataset. The ground-truth row reports the metrics for the utterances in the test set, which is why the language metrics are both at the maximum value 1.0. However, the parse metrics are not perfect since the high-resource parser used for evaluations does not achieve 100% accuracy on the test set.
We can see that the proposed 2-stage method outperforms both the 1-stage baselines, at all levels of privacy budget, even when the privacy budget is ∞ (i.e., the No DP row in the table), for both datasets. The 1-stage + domain baseline has a performance that is on average better than the 1-stage baseline (hypothethically due to the guidance that the domain prompts provide) but inferior to the proposed 2-stage method. We can also observe that on average, as the privacy budget increases (lower privacy), the performance of all methods increases, which makes sense as the added noise is decreasing.
Both single and 2-stage methods perform better overall on the TreeDST dataset in terms of the parse metrics. This could be due to the smaller set
5
of function types for TreeDST (303 for TreeDST vs. 524 for SMCalFlow), making it easier for both methods to capture these types. One counterintuitive observation is that for some metrics, for the 1-stage baseline, the performance with ϵ=16 is actually higher than ϵ=∞. We hypothesize that this could be due to the regularization effect that small amounts of noise has on the training (Smith et al., 2020), therefore the DP model with a high budget can generalize better (overfits less). We also observe that for the SMCalFlow dataset, MAUVE doesn’t improve much as we increase the privacy budget. We relate this to the complexity of the hyperparameter search/optimization in DP-mechanisms, and that we were not optimizing for improving MAUVE. A more extensive hyperparameter search could yield better results on ϵ=16.
We provide more fine-grained comparisons on the parse tree distribution matching with ground truth (such as the top-10, 25, 50 and 100 most common function coverage in Appendix B.4). We explore the reasons behind the superior performance of the proposed method in the next section.

heading: 5.2 Ablation Studies
text: We hypothesize that the superiority of the 2-stage method, which models the parse trees as intermediate variables, is because it (1) improves the language modeling within each utterance and (2) helps the model learn the different semantic modes in the data.
Disrupting the Correlation between Parse-trees and Utterances. We first disrupt the correlation between parse trees and utterances by shuffling them (i.e., each utterance is now paired with a random parse tree). We discuss the full details and results of this experiment in Appendix B.2, but in short we observe that in this setup, the parse-related metric (function overlap) for 2-stage synthetic data falls from 63.9% to 23.2%, which is below that of the 1-stage baseline (47.5%), supporting the hypothesis that the structure in the parse trees and the correlation to utterances are important. Based on this, in the rest of this section, we test our multi-modality hypothesis by limiting the data to fewer modes. Changing the Modes in the Data. Our conjecture is that part of why the 2-stage model benefits from explicitly learning a distribution over semantic parses is that this helps it capture the different
semantic modes in the data—that is, the various types of functionality invoked by the utterances.
To test this hypothesis, we create a subsample of the original dataset, consisting only of (utterance, parse tree) pairs where the parse tree contains the Weather function. This “single-mode” dataset focuses on weather-related queries. We compare the pattern of results to that in the original experiment, where the dataset had greater diversity of function types. Note that, due to the high compositionality of the parse trees (e.g., a parse tree that contains the Weather function may also contain many other functions for, e.g., datetimes and locations), the restricted dataset still contains 158 function types, compared to the 524 in the original data.
Table 2 shows the results for this experiment, and compares the performance of the 1-stage and 2-stage methods (the numbers don’t match those of Table 1 for the same ϵ value, as for this experiment we use a smaller batch size of 1024 for the sake of run-time). As we can see, the improvement achieved by the 2-stage method shrinks on the restricted dataset, which supports our conjecture. It is noteworthy that although the improvement shrinks, it remains relatively high for the metrics that consider parses, showing that the 2-stage method retains an advantage in capturing the remaining functional diversity in the restricted dataset.

heading: 5.3 Hyperparameter Sensitivity Analysis
text: We run extensive analysis to study the effect of different hyperparameters (batchsize, learning rate, clipping threshold and the total number of the epochs for the 2-stage method) on the quality of the synthesized text. For the sake of space we present all these results in Appendix A.3. In short, we find that as the batch size increases, the quality of the generated text also improves, which has been observed by prior work in DP generation of text (Li et al., 2021). For splitting of the privacy budget between the training of the parse-generation and parse2utterance models, we find that most of the epochs should be allocated to the latter: increasing T2 at the expense of T1 steadily improves the quality of the generated text (under both text-based and parse-based metrics), until a tipping point is reached. We findT1=2 and T2=8 epochs to be the best setup.
6

heading: 5.4 Downstream Parser Improvement
text: In this section, we demonstrate a major application of our privacy-preserving data synthesis, through an end-to-end experiment: improving the performance of a low-resource semantic parser and adding new functionality to it based on private user data. Building on the notation described in Section 3, we assume access to a small “eyes-on” (non-private) dataset Dpub of (utterance, parse tree) pairs, and a semantic parser pϕ0(y | x) trained on Dpub. We also assume we have “eyes-off” access to a much larger unlabeled private utterance dataset Dpriv, which we can only access through a DP mechanism. Our goal is to synthesize dataset Ds such that a parser trained on Dpub ∪ Ds performs better on dataset Dpriv than the original parser pϕ0(y |x).
We devise this experiment such that Dpub is
missing function types that Dpriv has, and we aim to capture this missing functionality through the DP data synthesis and augmentation. Specifically, we will construct Dpub by removing one class of function from SMCalFlow: either Weather or event on date (EoD). We chose these two to be distinct, as weather-related queries comprise only 3.4% of the samples in the dataset, whereas EoD appears in 10.7%. Comparing these two helps us study the effect of a function’s commonness on the ability to add it through this procedure. Dataset Partitioning. We uniformly subsample 1 10 of the pre-processed SMCalFlow training set (from Section 4.1) to form Dpub. Each entry contains a human-generated utterance and the corresponding expert-annotated parse tree. We drop all the samples with weather or EoD from Dpub. We use the remaining 910 th of the dataset to form Dpriv. We test on a uniformly sampled subset of the test data (with the same 110 ratio). Parser Metrics. We measure the performance of the initial low-resource semantic parser and the augmented ones using the following three metrics, adopted from Zhou et al. (2022): (1) Exact Anonymized Graph Match, which reports the percentage of test samples for which the anonymized generated parse tree is an exact match to the ground truth expert annotations from SMCalFlow; (2&3) API Match Precision and Recall, which measures the precision and recall of the generated parse tree nodes (API functions) from the parser, treating the tree as a bag of nodes, against those of the ground truth.
7

heading: Experimental Procedure.
text: Starting with a low-resource parser pϕ0(y | x) trained on Dpub, and we aim to improve it using Dpriv. We first obtain a predicted parse tree for each utterance in Dpriv by running it through pϕ0(y |x). We then train the 1-stage baseline and our 2-stage method on Dpriv with DP-SGD (we set ϵ=3), and then take samples from them to formDs, as outlined in Algorithms 1 and 2. For the purpose of this experiment, we set the size of Ds to 90,000 samples 4. At this point, when applied in practice, we would have experts annotate the utterances in Ds with their parse trees. For this experiment, lacking human annotators, we use a high-resource parser (Appendix A.3) to predict a parse tree as a reasonably good approximation to the ground truth. Finally, we augment Dpub with Ds, re-train the semantic parser, and compare its performance to pϕ0(y |x). Results. Table 3 shows the results for the different compositions of Dpub, described earlier in this section. We provide the language and parse metrics (from the previous sections) for these generations in Appendix B.3, alongside a more fine-grained breakdown of the results in Appendix Table 8. As expected, the exact graph match performance of the non-augmented parser when Dpub is missing Weather or EoD is 0.0 on utterances containing those functions (the precision and recall are not exactly zero since there are queries that contain Weather alongside other function types and those other types are correctly identified). After augmentation, we see that both methods for synthesizing ut-
4As we want to makes Dpub ∪Ds have about the same size as the original SMCalFlow training set). The future work can explore different ways to vary exactly how we sample Ds (such as focusing on the fenced utterances, or changing the size of Ds).
terancesDs lead to parser improvements, with the 2- stage method providing more overall improvement.
It is on EoD that we observe the most improvement (and the most pronounced gap between the two methods), especially from the 2-stage method. Presumably this is due to the higher prevalence of EoD functions in Dpriv, so there are more training samples (10.7% vs. the 3.4% of weather) for it than the private training could use. We conjecture that this is also the reason behind the bigger performance gap between the single and two stage models for this function, as there are more samples wrongly annotated by the low-resource parser, and the 2-stage method is picking up on this through its use of the parse trees.
Another observation is that the gap between all the augmentations and using the full dataset is still quite significant. We believe this is due to the fact that we used the low-resource parser to provide annotations for the private data, which means inaccurate annotations are being fed to the 2-stage method for training. Therefore Dpriv is still far from the fully expert annotated data used to train the parser in the first row of the table. Using a better parser, or iteratively augmenting and re-annotating, might help close this gap by providing more accurate parse trees to the 2-stage method.

heading: 6 Related Work
text: We offer a brief summary of related work here. For a detailed discussion see Appendix D. PII scrubbing. Techniques such as automated removal of personally identifiable information (PII) (Lison et al., 2021; Aura et al., 2006) and training with redacted data (Zhao et al., 2022) are often used to protect user privacy, especially in
8
medical settings (Kayaalp et al., 2014). However, alone, they do not provide stringent guarantees or bounds on information leakage (Brown et al., 2022), as there are many forms of private information not captured in PII removal. Rather, DP is used as a gold standard for limiting leakage. Differentially Private Data Synthesis. Recent concurrent work (Yue et al., 2022; Mattern et al., 2022) has attempted different variations of taking samples from a DP-trained model (Abadi et al., 2016; Li et al., 2021; Yu et al., 2021; Kerrigan et al., 2020; Shi et al., 2021; Anil et al., 2021; Tian et al., 2021) to synthesize labeled data for classification tasks, by conditioning the generation on the label of each sample, and assuming that the prior distribution over labels is known and public. They then use this data to augment and improve classification models. In our case, however, we are not dealing with finite labels or classification, we want to improve the performance of a semantic parser, which is a structured (hierarchical) task, where there are infinite possible parses that we cannot enumerate, and the labels are also private (since parse trees are almost unique we consider them private). Therefore, existing methods cannot be applied to this problem, as mentioned in the introduction.
Another difference in this concurrent line of work is their reliance on gold (ground truth) labels. In our case, as we show in the final experiment, we can instead train our DP models on imperfect parse trees that are generated by low-resource parsers. Semantic Parsing. The closest work to ours is Yang et al. (2022), which deals with the problem of safely learning from private user utterances, starting with a low-resource semantic parser. Unlike our setup, however, they do not rely on DP and are only concerned with the removal of PII. While they also consider a distribution over parse trees and employ a parse-tree-to-utterance process, they implement the latter using only the original supervised data, which forecloses the possibility of adapting to distributional shifts as in our experiments. Another line of work advocates for attenuating privacy risks by enabling semantic parsers to autonomously learn from interacting with users (Yao et al., 2020; Karamcheti et al., 2020; Yao et al., 2019). Through privacy-preserving data synthesis, our method supports diagnostics of private user traffic and better control over the associated learning process.
Data Augmentation Data augmentation for performance improvement is also relevant to our work. Malandrakis et al. (2019); Cho et al. (2019); Jolly et al. (2020); Okur et al. (2022); Zhang et al. (2022) propose data generation methods designed for the “intents and slots” model where each utterance is considered to have one of a fixed set of intents, and each intent has a fixed set of slots, each of which needs to be filled with a value. We, in contrast, use the SMCalFlow and TreeDST datasets which use compositional semantic representations of arbitrary complexity. Cho et al. (2019) and Okur et al. (2022) propose to use automated paraphrasing to create semantically equivalent variants of an existing utterance. Each utterance is paraphrased independently of the others. Unfortunately, paraphrases of a private utterance cannot be made public as they leak information about the private utterance; obtaining differentially private paraphrases would require new research. Training the paraphrasing model by DP-SGD will not help (and is generally unnecessary as such a model can be trained on public data).

heading: 7 Conclusions
text: In this paper, we studied the problem of using private user data to improve semantic parser performance in task-oriented dialogue systems, without violating user privacy. We proposed a two-stage method for differentially private utterance synthesis that exploits the inherent structure in the parse trees to better fit the private distribution. We showed that this method outperforms a baseline DP generative language model on a variety of datasets and metrics. We also demonstrated the effectiveness of our method in an end-to-end application scenario where we improved the performance of a low-resource parser by adding new functionality that was motivated by private user data. We showed that our method provided overall gains of 8.5% points in accuracy with the new feature.

heading: Limitations
text: DP training of large models is compute-intensive, requiring per-example gradients and large batch sizes (Li et al., 2021; Subramani et al., 2021). This renders the training of such models difficult and not easily accessible to everyone.
DP-SGD takes records to be single training examples, which in this paper’s experiments
9
correspond to single user utterances. That setup prevents the trained model from revealing much information about any given single utterance, but it may still allow information to leak that is repeated across multiple utterances (Brown et al., 2022).
For both the baseline method and our two-stage method, we trained our model to approximately match the true distribution of private user utterances, ppriv(x), to the extent that this was possible under a differential privacy guarantee. Of course, there are many ways to measure the quality of an approximation, and different approximations are appropriate for different tasks where it might be important to preserve different properties of ppriv(x). The one-stage baseline approach implicitly aims to achieve a low cross-entropy, by applying DP-SGD to the log-likelihood function. In contrast, our two-stage approach aims to encourage an approximation that also roughly preserves the marginal distribution over semantic function types. We did not investigate more direct ways of encouraging such an approximation, for example, one-stage DP-SGD with a modified objective function that explicitly evaluates the marginal distribution in addition to the log-likelihood.
Finally, we trained an approximate model of ppriv from which we can draw utterances to inspect and annotate. But we must acknowledge that ppriv is not the ideal distribution to approximate. Even if we were able to actually use private utterances to improve the system, we would not necessarily want to draw them directly from ppriv. Rather, we would want to select them by active learning—selecting the private user utterances that would be most useful to inspect or to include in the annotated training data. Thus, when training our model by DP-SGD (using either the one-stage or two-stage procedure), we could upweight or upselect the private utterances that appear useful in this way—resulting in a differentially private model that generates useful synthetic utterances. Specifically, traditional active learning by uncertainty sampling (Settles, 2012) would select utterances where the semantic parser was uncertain what to do. We would also want to select utterances where the system suspected for other reasons that it did not do the right thing—because it classified the user’s request as a functionality that the system did not yet support, or because the user objected in some way to the system’s response. We
have left experiments on this setup to future work.

heading: Ethics Statement
text: The over-arching goal of our work is to improve semantic parsers and dialogue systems while protecting the privacy rights of users who contribute their data to this goal. While we train our models by applying DP mechanisms with worst-case guarantees, deploying these models in real-world setups and using these synthesized data-sets requires further verification that users’ privacy is preserved, by setting the right definition of “record” (i.e., training example) and the right (ϵ,δ) budget based on privacy policy guidelines. Further studies are needed on what the reported privacy budgets actually mean in practice for users, how users perceive these privacy mechanisms, and how they can provide informed consent to have their data used to improve the systems (Brown et al., 2022).

heading: A Experimental Setup Details

heading: A.1 DP Training Implementation
text: We build our methods on top of the setup introduced by Li et al. (2021), and we use their repository https://github.com/lxueche n/private-transformers.

heading: A.2 Software, Hardware, and Data Specifications
text: We use Opacus 0.15.0, HuggingFace Transformers 4.10.3, PyTorch 1.9.1 with Cuda 10.2, and Python 3.8.8. We run our experiments on an Azure ML Nvidia DGX-2 system, which has 16 Tesla V100 GPUs with 512GB memory in total. (We estimate that the experiments took an total of 4 weeks of GPU-hours.) We use SMCalFlow 2.0, and TreeDST, which are both publicly available datasets. For all the datasets and software used, we abide by the usage agreement.

heading: A.3 Training Hyperparameter Details
text: Based on the hyperparameter analysis shown in Section B.1 below, we find the best clipping threshold to be C =0.1 in all experiments (SMCalFlow and TreeDST, and the learning rate to be 2×10−3. We also set the privacy parameter δ=8×10−6, and T =10 epochs for the 2-stage setup. Below we list the dataset-specific parameters.
SMCalFlow. For Table 1, we use batch size of 2048, by setting actual batch size to 32 and gradient accumulation to 64. For the rest of the tables we
use batch size of 1024 for the sake of speed, as we do gradient accumulation.
We use T =10 for ϵ=16 and T =7 for ϵ values of 8 and 3 in the single stage scenario, and T1 = 2 and T2 = 8 in the 2-stage. For the non-private experiments, T =6 for single stage and T =10 for two stage, where we split the epochs equally. For the numbers in Tables 2 and 4 we set T1=3 and T2=7.
TreeDST. For Table 1, we use batch size of 1024, by setting actual batch size to 32 and gradient accumulation to 32. We use T =10 for all epsilon values in the single stage scenario, and T1=2 and T2 = 8 in the 2-stage. We set the learning rate to 0.002. For the non-private experiments, T =6 for 1-stage and T =20 for 2-stage, where we split the epochs equally. We set the learning rate to 0.001.
High-Resource Parser Hyperparameters. We use the parser architecture from Zhou et al. (2022), where we train the model with no context (as in we train on single utterances and not conversations), and we only use the human utterances and not the agent’s. We train two parsers, one on SMCalFlow and one on TreeDST, for their corresponding evaluations. All of our parsers are based on the Transformer architecture, adapted to the graph action sequence. Implementation details for the parser are provided in Appendix E of Zhou et al. (2022). We trained this model with learning rate of 0.001 for 50 epochs.
End-to-end (Parser Improvement) Experiment Hyperparameters. For the experiments in Section 5.4, for training the DP models we use the same hyperparameters as before. For training the low-resource parsers, we find that for the one missing the Weather function the best set learning rate and epoch numbers are 0.003 and 150 epochs. For event on date we use the same learning rate, but train for 180 epochs.
Choosing the best setup to report. To select the best hyperparameter setup for the baselines and our method, we relied on MAUVE and Function overlap metrics, as in chose the setup which had a higher combination of these two values (in almost all cases if one was highest the other was also highest). However, we did discard some setups with high GPT-2 loss (higher than 2.0 on a GPT-2 fine-tuned on SMcalflow 2.0), as we observe that
14
MAUVE and Function overlap tend to be very high on some grammatically incorrect sentences with diverse vocabulary. Therefore, we use GPT-2 loss as an auxiliary metric to help us sift.

heading: B Additional Experimental Results

heading: B.1 Hyperparameter Sensitivity Analysis
text: In this section we compare different hyperparameter settings in the generation process, and analyze their effect on the quality of the synthesized text. First, we vary the privacy budget split between the first and second stages, in the 2-stage setup. Then, we vary the training hyperparameters such as learning rate, clipping threshold, and batch size, to measure the sensitivity of the results. We use the test set portion of the dataset for this.

heading: B.1.1 Training Epoch Split
text: How the number of training epochs is split between the two stages (i.e., how the privacy budget is split) has a significant impact on the quality of the synthesized text. Figure 3 shows the results for this experiment, where we experiment with all 9 possible ways of splitting 10 total training epochs between the two stages. A row label such as “9–1” denotes a model trained for9 epochs on stage 1 and1 epoch on stage 2. The more epochs spent on a stage, the more the privacy expenditure of that stage.
Based on the 4 measured metrics, we can see that the setup 2–8 is the best one. We also see that as we go from the bottom of the graph to the top (i.e. as we spend less epochs on training the parse
generation model and more epochs on training the parse2utterance) the overall quality improves. However, we also see that this trend breaks when we go from 2–8 to 1–9, where the metrics plummet, which is probably due to 1 training epoch being too small for the parse generation model.

heading: B.1.2 Clipping, Batch size and Learning Rate
text: Finally, we run a hyperparameter sensitivity analysis on the gradient clipping threshold (for the DP optimization which requires gradient clipping and addition of noise, see Section C.1), batch size and learning rate. Figure 4 shows the results for the 1-stage and 2-stage techniques, on the top and bottom rows of the figure, respectively. Here, we only look at the token type overlap and function overlap metrics, for simplicity. The first two graphs in each row are controlled for batch size (set to 1024 and change clipping and learning rate), whereas the last two graphs are controlled for clipping threshold (set to 0.1) and change batch size and learning rate. We can see that for both the baseline and the proposed method, the best set of parameters is 0.1, 0.002 and 2048 for clipping, learning rate and batch size, respectively.

heading: B.2 Disrupting the Correlation Between Parse Trees and Utterances
text: For this experiment we take the parse trees and utterances in the dataset and randomly pair them up, so that the utterance in each pair has no relationship to the parse tree. In this setting, we expect the
15
parse2utterance model to ignore the uninformative parse or, by fitting to spurious correlations, use it as a source of randomness.
Table 4 shows the results for this experiment, both with and without DP. We can see that the 2-stage shuffle model performs worse than the baseline on the parse-related metrics, which supports the hypothesis that the structure in the parse trees and the correlation to utterances are important for these metrics. That is, the benefit of the original 2-stage model does not arise simply from the fact that it has 2-stages or more parameters. We also see the same pattern for the word-type (vocabulary) overlap metric, where once shuffled the model fails to capture the diversity in the vocabulary. However, the baseline exhibits higher MAUVE than the shuffled model. Manual investigation revealed that the shuffled model’s generations are grammatically correct but are more repetitive across utterances, compared to other models, and MAUVE fails to sufficiently penalize these issues.

heading: B.3 Metrics for the Samples Used in the Parser improvement Experiment
text: Table 5 shows the detailed language and parse metrics for the checkpoints used for augmenting the parser in Section 5.4. These results are obtained for ϵ=3, and for the 2-stage method they are overall worse than the numbers shown in the main results, Table 1, for the same ϵ value. The reason behind this discrepancy is that for the parser improvement experiment we annotate the training samples using the low-resource parser which is much less accurate that the ground truth parse trees used in Table 1. However, we see that for the 1-stage baseline the overlap metrics are now much higher. The reason behind this discrepancy is that in Table 5 we report results over the 90k samples, whereas the results in the body of the paper are over 13k samples, and the more we generate the more vocabulary/functions get covered.

heading: B.4 Function Distribution Coverage Break-down Results
text: To provide a better, more detailed depiction of how well each method matches the distribution of the functions to that of the ground truth, we report the overlap between the top-10, 25, 50 and 100 most common functions in the ground truth, with those for each generation method. We report those results
in Table 6.

heading: B.5 Results For Larger Models
text: The results discussed in the body of the paper are all reported on pre-trained GPT2-small. To further explore with other models, we present results on GPT2-Large, on the SMCalFlow dataset and present the results in Table 7. We do not observe improvements over GPT2-small in the synthesized data’s quality. We speculate this is due to insufficient hyperparameter searches on the larger model, or the small size of our training dataset.

heading: B.6 Effect of Data-augmentation for Adding New Functionality on Existing Functions
text: In this section we want to see whether our data augmentation we implement to add new functionality ends up hurting the performance of the existing functions. Table 8 shows this. This table corresponds to Table 3 in the main body of the paper, however it presents results for every functionality, except the augmented one.

heading: B.7 Examples of Generated Utterances and Parse-trees
text: Table 9 shows some examples of generated utterances for the 1-stage baseline, and the 2-stage proposed method. Table 10 shows pairs of generated trees and then utterances that are conditioned on those trees, for the second stage model. We observe some syntactically invalid generated parse trees from the parse generation model in the 2-stage setup. Nonetheless, the second stage model can still generate coherent text from such trees, and benefit from them, as we see the 2-stage model captures the distribution better.

heading: C DP and Privacy accounting

heading: C.1 Training via DP-SGD
text: To train a neural network with differential privacy, the most widely used algorithm is the DP variant of stochastic gradient descent (DP-SGD) (Abadi et al., 2016). DP-SGD resembles ordinary SGD, but at each gradient update step (derived from a minibatch of training examples), it first clips the per-example gradient by its norm, then obfuscates the gradient by adding Gaussian noise. Intuitively, this limits the contribution that a single example makes to the final model parameters.
Clipping the gradient to a maximum norm of C
16
(a hyperparameter) is done as follows:
g̃← ∑
x∈B clip(∇ℓ(x)) (1)
where B is the current minibatch and clip(v) = v · min { 1, C∥v∥2 } . The clipped gradient is small enough for us to obfuscate it (without further changing its mean) by adding Gaussian noise with mean 0 and standard deviation σC:
g← g̃+N ( 0,σ2C2 )
|B| (2)
The privacy expenditure (ϵ,δ) is a function of C, σ, |B|, |D|, and the total number of epochsT (which controls the total number of gradient updates). It is determined based on the Rényi DP (Mironov, 2017) privacy accounting method. In practice, following prior work, we fine-tune our models using DP-Adam (Abadi et al., 2016; Li et al., 2021).
Post-processing property. The post-processing property of DP (Dwork et al., 2006) ensures that if an algorithm A satisfies (ϵ,δ)-DP, then so does
F ◦A for any function F , which means that we can run as many inferences (i.e., take as many samples) as we want from the DP-trained models, without changing the privacy expenditure.

heading: C.2 Privacy Accounting
text: We chose in Algorithm 2 (§3.2) to split T into T1 and T2 and share the other DP parameters across the two stages. This lets us use a single shared moments accountant (Abadi et al., 2016) and thus benefit from sub-linear composition.
To be precise, if our method for ensuring (ϵ,δ)DP happens to guarantee (ϵi,δ)-DP at each stage i (as a result of training for Ti epochs with clipping threshold C, batch size |B|, and noise multiplier σ), then we have ϵ≤ϵ1+ϵ2. We may enjoy ϵ<ϵ1+ϵ2 (Kairouz et al., 2015; Mironov, 2017).
Thus, we are in general able to train for more total epochs, or with lower noise multipliers, than if we had directly divided the privacy budget as ϵ= ϵ1+ ϵ2, enforced (ϵi,δ)-DP at each stage i to guarantee (ϵ,δ)-DP overall by linear composition, and used that commitment to determine the maximum allowed Ti (for a given σi) or the minimum allowed σi (for a given Ti) at each stage i.

heading: D Additional Related Work
text: In this section, we discuss additional related work beyond Section 6.

heading: D.1 Differentially Private Training and Synthesis
text: In our work, we took samples from a generative language model trained with differential privacy (Abadi et al., 2016; Li et al., 2021; Yu et al., 2021; Kerrigan et al., 2020; Shi et al., 2021; Anil et al., 2021; Tian et al., 2021), to build a synthesized
17
dataset which would then be used to improve different down-stream tasks.
As an alternative, it is also possible to decode in a differentially private way from a non-DP model (Ginart et al., 2022) Other work has proposed DP n-grams (Kim et al., 2021a), which helps extract common n-grams from the data privately. n-grams, however, are not much help in improving performance on downstream tasks (parsing or NLU classification), unless n is quite large.

heading: D.2 Semantic Parsing
text: Other than Yang et al. (2022), we are not aware of other works that specifically address learning from private data for semantic parsing. Nevertheless, many papers have explored ways to construct, improve, and adapt semantic parsing systems with minimal amounts of supervision. Wang et al. (2015) construct semantic parsing datasets by first enumerating potential parse trees (as “canonical utterances”) and then asking crowd workers to convert them into utterances. Su and Yan (2017) leverage the canonical utterances for cross-domain generalization in semantic parsing by reformulating semantic parsing as paraphrasing from input utterances to canonical utterances. Yin et al. (2022) is a later instantiation of a similar idea, with automated paraphrasing of canonical utterances into natural utterances and a few other components. Zhao et al. (2019); Zhong et al. (2020); Cao et al. (2020); Burnyshev et al. (2021); Kim et al. (2021b); Tseng
et al. (2021) are other works in a similar vein. Since these tend to use similar primitives as our 2-stage approach, but trained without DP on different sources of data, our approach is largely complementary and can be used to augment the prior approaches.
18
19
ACL 2023 Responsible NLP Checklist
A For every submission: 3 A1. Did you describe the limitations of your work?
Limitations section after section 7
3 A2. Did you discuss any potential risks of your work? Limitations and ethics statement
3 A3. Do the abstract and introduction summarize the paper’s main claims? Abstract
7 A4. Have you used AI writing assistants when working on this paper? Left blank.
B 3 Did you use or create scientific artifacts? we use existing artifact, section 4
3 B1. Did you cite the creators of artifacts you used? section 4
3 B2. Did you discuss the license or terms for use and / or distribution of any artifacts? Appendix A.1
3 B3. Did you discuss if your use of existing artifact(s) was consistent with their intended use, provided that it was specified? For the artifacts you create, do you specify intended use and whether that is compatible with the original access conditions (in particular, derivatives of data accessed for research purposes should not be used outside of research contexts)? A.1 and 4
3 B4. Did you discuss the steps taken to check whether the data that was collected / used contains any information that names or uniquely identifies individual people or offensive content, and the steps taken to protect / anonymize it? Section 4 and 5.4, for reporting metrics on the data we anonymize the strings. Also, this paper itself is on privacy and it builds on how we can have bounded data leakage using DP, as removal of PII is not enough.
3 B5. Did you provide documentation of the artifacts, e.g., coverage of domains, languages, and linguistic phenomena, demographic groups represented, etc.? 4 and Appendix A
3 B6. Did you report relevant statistics like the number of examples, details of train / test / dev splits, etc. for the data that you used / created? Even for commonly-used benchmark datasets, include the number of examples in train / validation / test splits, as these provide necessary context for a reader to understand experimental results. For example, small differences in accuracy on large test sets may be significant, while on small test sets they may not be. 4 and appendix A
C 3 Did you run computational experiments? 5
3 C1. Did you report the number of parameters in the models used, the total computational budget (e.g., GPU hours), and computing infrastructure used? 5 and appendix A
The Responsible NLP Checklist used at ACL 2023 is adopted from NAACL 2022, with the addition of a question on AI writing assistance.
3 C2. Did you discuss the experimental setup, including hyperparameter search and best-found hyperparameter values? 4, and Appendix A
3 C3. Did you report descriptive statistics about your results (e.g., error bars around results, summary statistics from sets of experiments), and is it transparent whether you are reporting the max, mean, etc. or just a single run? 4,5 and appendix B
3 C4. If you used existing packages (e.g., for preprocessing, for normalization, or for evaluation), did you report the implementation, model, and parameter settings used (e.g., NLTK, Spacy, ROUGE, etc.)? 4, 5 and appendix A
D 7 Did you use human annotators (e.g., crowdworkers) or research with human participants? Left blank.
D1. Did you report the full text of instructions given to participants, including e.g., screenshots, disclaimers of any risks to participants or annotators, etc.? No response.
D2. Did you report information about how you recruited (e.g., crowdsourcing platform, students) and paid participants, and discuss if such payment is adequate given the participants’ demographic (e.g., country of residence)? No response.
D3. Did you discuss whether and how consent was obtained from people whose data you’re using/curating? For example, if you collected data via crowdsourcing, did your instructions to crowdworkers explain how the data would be used? No response.
D4. Was the data collection protocol approved (or determined exempt) by an ethics review board? No response.
D5. Did you report the basic demographic and geographic characteristics of the annotator population that is the source of the data? No response.

heading: Future Work
text: We
have left experiments on this setup to future work