Synthetic Query Generation for Privacy-Preserving Deep Retrieval Systems using Differentially Private Language Models (2024)

Abstract: We address the challenge of ensuring differential privacy (DP) guarantees in training deep retrieval systems. Training these systems often involves the use of contrastive-style losses, which are typically non-per-example decomposable, making them difficult to directly DP-train with since common techniques require per-example gradients. To address this issue, we propose an approach that prioritizes ensuring query privacy prior to training a deep retrieval system. Our method employs DP language models (LMs) to generate private synthetic queries representative of the original data. These synthetic queries can be used in downstream retrieval system training without compromising privacy. Our approach demonstrates a significant enhancement in retrieval quality compared to direct DP-training, all while maintaining query-level privacy guarantees. This work highlights the potential of harnessing LMs to overcome limitations in standard DP-training methods.

## introduction deep retrieval systems have been widely adopted in many online services, from search to advertising, to match user queries to relevant recommendations @xcite . in many applications, candidate items for retrieval are often publicly available non-personal information in the sense that they do not contain any specific information related to any single user (e.g., articles, products, movies, ads). however, the input queries to retrieval systems can often contain user personal information. therefore, training deep retrieval systems on user data may enhance user experience through timely relevance, but it may also unintentionally compromise user privacy since neural network models have been demonstrated to implicitly memorize and leak sensitive user information in the training data @xcite . this raises privacy sensitivities around each stage of data collection, training, inference, and hosting these systems. in this work, we seek to address the problem of ensuring user query privacy guarantees in deep retrieval systems without significantly hindering their utility. the standard approach to ensure the privacy of training data in many large-scale machine learning models is to directly introduce differential privacy (dp) @xcite guarantees during training @xcite . these dp-training strategies provide guarantees by limiting the impact each individual data instance has on the overall model. however, some models contain design elements that inherently hinder the ability to limit per-example contributions, and thus are more difficult to directly dp-train. these include models with components that calculate batch statistics such as batch normalization layers @xcite and models with losses that cannot be decomposed into per-example losses such as pairwise and contrastive-style losses @xcite . this limits the application of dp-training on deep retrieval systems since these systems typically use non-per-example decomposable contrastivestyle losses to train semantic neural representations of user queries and candidate items in order to facilitate efficient vector-based retrieval strategies. the injected noise needed to achieve dp guarantees for these losses can scale with the number of candidate items that appear in the example-level loss computations, which can result in excessive retrieval quality degradation. hence, additional considerations are often needed to adapt dp-training to deep retrieval models for achieving an adequate privacyperformance tradeoff. in this work, we take an approach that ensures user query privacy prior to training a deep retrieval system in order to circumvent the various issues with directly dp-training deep retrieval systems with a non-per-example decomposable contrastivestyle loss. we build on the framework of synthetic data generation using dp language models (lms) @xcite to develop an approach for private query sharing to train any downstream deep retrieval system with query-level privacy guarantees with respect to the original training data. we empirically demonstrate considerable improvements in our approach on retrieval quality without compromising privacy guarantees compared to direct dp-training methods. more generally, our work presents an initial study into a nascent opportunity to leverage exciting breakthroughs in lms to overcome crucial limitations in directly dp-training machine learning systems. ## approach we describe our general-purpose approach to obtain dp synthetic data for training a downstream deep retrieval system while ensuring query-level privacy on the original training data. ## evaluation 6.1 evaluation on retrieval tasks we evaluate the retrieval models on the ms-marco test data set and various other beir re-trieval data sets for zero-shot evaluation. we evaluate on the normalized discounted cumulative gain score over the top 10 predictions (ndcg@10) which measures the relevance and ranking quality of items in a recommendation list, considering both quality and position. we also report the recall score over the top 10 predictions (recall@10), which measures the percentage of times the ground truth recommendation appears in the top 10 predictions. we report the evaluation results of a single training run. ## conclusion our work focused on ensuring dp guarantees in training deep retrieval systems. we discussed the limitations of dp-training such systems with the often used in-batch softmax loss function, which is non-per-example decomposable. we introduce an approach of using dp lms to generate private synthetic queries for downstream deep retrieval training. this approach ensures theoretical guarantees on query-level privacy prior to downstream training, thereby bypassing some of the limitations of dp-training. furthermore, we empirically demonstrate that our approach improves retrieval quality compared to direct dp-training, without compromising query privacy. our work highlights the potential of lms to overcome crucial limitations in dp-training ml systems. limitations there are a few limitations to our approach. firstly, while we observed that larger lms generate higher quality synthetic queries, it is worth noting that training such large models may too computationally expensive. exploring more parameter-efficient finetuning methods tailored to dp-training could mitigate the computational burden associated with training such larger models. secondly, it is necessary for the publicly pretrained lm utilized for generating synthetic queries to not have been pretrained on the original queries we aim to privatize. this imposes a constraint on the choice of pretrained lms suitable for generating private synthetic queries. next, as observed in the synthetic example in risks & ethical considerations data privacy is a crucial consideration in the responsible development of personalized machine learning systems. our work directly offers potential solutions to address issues of data privacy in deep retrieval systems. however, it is important to acknowledge the above limitations on privacy guarantees of our approach to prevent undesired privacy risks.