Can LLM s get help from other LLM s without revealing private information? (2024)

Abstract: Cascades are a common type of machine learning systems in which a large, remote model can be queried if a local model is not able to accurately label a user’s data by itself. Serving stacks for large language models (LLMs) increasingly use cascades due to their ability to preserve task performance while dramatically reducing inference costs. However, applying cascade systems in situations where the local model has access to sensitive data constitutes a significant privacy risk for users since such data could be forwarded to the remote model. In this work, we show the feasibility of applying cascade systems in such setups by equipping the local model with privacy-preserving techniques that reduce the risk of leaking private information when querying the remote model. To quantify information leakage in such setups, we introduce two privacy measures. We then propose a system that leverages the recently introduced social learning paradigm in which LLMs collaboratively learn from each other by exchanging natural language. Using this paradigm, we demonstrate on several datasets that our methods minimize the privacy loss while at the same time improving task performance compared to a non-cascade baseline.

## introduction large language models (llms) such as gemini ultra by @xcite and gpt-4 by @xcite are reporting remarkable performance on many tasks. these models, however, not only come with high inference costs, but they also have to run in data centers far from the local contexts where private data is available. conversely, models that can run in private contexts, such as gemini nano, have more limited capabilities since they run on the user's device. to unlock state-of-the-art performance in private contexts, local models with access to sensitive data need to be equipped with a privacy-preserving mechanism that enables querying a remote model without sharing any sensitive data. although standard cascade systems in which a smaller, less capable model, queries a larger, much more capable one in order to solve a task have previously been studied @xcite , privacypreserving ones have not yet been explored. in today's cascade systems, the decision of whether a larger model should be leveraged or not is usually done through an additional mechanism that determines whether the query can be handled by the smaller model independently @xcite . if determined to be handled by the larger model, the query is simply forwarded without consideration for the private data it may contain. this poses privacy threats for users, ranging from leaking sensitive data to the forwarded sample even being ingested in training datasets of the remote system. we introduce the first privacy-preserving approach to cascade systems. contrasting to standard cascade systems, our local model always assumes its data is private. as such, the local model should not share anything private with the remote model. going one step further, even if the local model does not verbatim share private information, we aim to prevent a curious remote model operator from reconstructing private data by utilizing auxiliary information it might have. to focus on these challenges, we assume there are no efficiency constraints and that the local model can always ask for help from the remote model, as shown in figure 1 . therefore our optimal cascade setup consists of minimizing the privacy loss while maximizing task performance, where an upper bound is given by querying the teacher model with the actual data, although private. to succeed at this task, the local model, typically smaller and less capable, needs to find the right balance between revealing sufficient information about the problem to receive useful signals from the more capable, remote model while keeping details private. to enable learning from the remote model, the local model makes use of gradient-free learning capabilities through natural language that in-context learning (icl) capabilities of llms enable @xcite . throughout, we leverage the recently introduced social learning paradigm by @xcite in which llms learn through natural language from other llms. ## problem setting our paper considers a variant of social learning @xcite where neither of the participants has any labeled data. a local model, called the student, has private data that it cannot label well by itself. a larger, remote model, called the teacher, can do a better job at labeling the data. these two models form a cascade, in which the student can improve its performance by communicating with the teacher. we call what the student sends to the teacher a query. figure 1 shows a visualization of this setup. constraints there are two constraints on the queries from student to teacher. (i) the communication must be privacy-preserving, i.e. the student may not copy over its data and must not reveal anything private. (ii) there is only a single round of communication between student and teacher, meaning neither of them can maintain any state or update a model of the other's capabilities. to this end, all algorithms follow the same structure. given (0) that the student needs help, it then (1) uses it's private data to generate a query to the teacher. in turn, (2) the teacher uses the query to generate icl examples for the student. finally, (3) the student uses the icl examples to go back to solving its original problem. ## privacy measures the student's data may often contain sensitive personal information that should be kept hidden from an untrusted, or partially trusted, teacher. for example, consider a query that tries to figure out what disease could best explain a set of health symptoms that are experienced by a user after they have engaged in a specific sequence of activities. here, being able to associate the set of activities and/or symptoms with a specific user is a privacy violation that we would like to eliminate. to address such privacy violations, one might be tempted to resort to data anonymization techniques, such as differential privacy (dp) @xcite . however, these techniques are most useful when computing aggregates across many users (e.g. average of gradient vectors computed on a batch of sensitive training examples). using the local model of dp @xcite @xcite as a mechanism to mask private information in the query will end up masking both private and non-private information in the query, rendering the masked query useless for the task at hand. alternatively, using the icl model of dp @xcite to privatize the sensitive parts of the student's data suffers a major hurdle: it assumes the student has many private examples it can jointly consider when creating a query to the teacher. while we do look into grouping examples when generating a query in section 4.4, we expect the student to have very few private examples, and want it to be able to generate privacy-preserving queries even when only having a single, private example. dp-icl cannot work in such a setting. instead, we leverage data minimization privacy techniques, specifically contextual integrity (nissenbaum, 2004) which describes privacy as an appropriate flow of information. under this technique, the student would keep information that is useful for the task (e.g. the activities & symptoms in the above-mentioned example) but remove any personally identifying information that is not relevant to the query context. we note that even under perfect masking, this approach could still leak sensitive information, should the teacher model have access to auxiliary information that can be used to identify certain unique features that are strongly correlated with the "perfectly masked" prompts @xcite . thus, an important contribution of our work is a methodology for measuring and assessing leakage under auxiliary information. the success of our approach hinges on correctly identifying and masking the sensitive parts of the query without tampering with the description of the task. to this end, we propose various techniques that can analyze information in queries to produce safe queries that can be shared with the teacher model. to assess the privacy of the queries, we consider two concrete metrics, the entity leak metric that counts entities that exist in both original examples and the student's queries, and the mapping leak metric that considers a setting with auxiliary information. entity leak metric contextual integrity states that privacy is the appropriate flow of information. for most production applications, it is hard to say what is appropriate to share. as a proxy for this, we consider the interpretable metric of leaked entities. all entities, such as names, locations, email addresses, or numbers, in the dataset, are considered to be private. we measure how many of the entities in the original example are still part of the student's query upon masking. ## methods we consider three algorithms for how the student could privately learn from the teacher, as shown in figure 2 . the first of these methods is based on the student describing the problem it is facing while the latter two methods generate similar, non-private examples that the teacher can label. as a hyperparameter for all these methods, we consider the expansion size to denote how many labeled icl examples the student will receive from the teacher. ## experiments in order to evaluate the effectiveness of the methods introduced in section 4, we evaluate them in terms of accuracy and privacy on a diverse group of datasets and compare them against two baselines. models we use the gemini 1.0 family of models @xcite for all of our experiments. as the teacher, we utilize ultra, the most powerful model of the family. in most of our experiments, nano-2, a 3.5b parameter model that can be deployed on mobile phones, is the student. the student model capabilities naturally influence the performance of our method and hence we also run experiments when pro is the student. in line with previous reports on nano's performance @xcite , we normalize task success in all our experiments by the teacher's performance since it is an upper bound for what we can hope to achieve. ## conclusion in this paper, we investigated whether llms can privately query external llms to improve their performance. indeed, we find that our methods comfortably beat strong baselines that have privacy constraints in place, even with gemini 1.0 nano-2 as the student, a 3.5b model that fits on phones. to evaluate the privacy performance of our methods, we look at two metrics, a simple to interpret count of entities leaked, and another, novel, metric that measures an upper bound of what a curious teacher with auxiliary information could hope to recover from the student's queries. for the first metric, we find masking problems (method 3) to work well, while generating new problems (method 2) with grouping does well in cases where the teacher can be expected to have auxiliary information. ultimately, we note that the choice of methods depends on the concrete threat model considered. for either threat model, we present a compelling system and analysis, which show that leakage can be low while beating strong quality baselines. additionally, we show how grouping examples improves the privacy metrics, and can, under a given labeling budget constraint, even improve model quality. future work in this space could consider more complex forms of student-teacher interactions, further improve the privacy metrics established, and look into modalities other than text.