R emote RAG : A Privacy-Preserving LLM Cloud RAG Service (2025)

Abstract: Retrieval-augmented generation (RAG) improves the service quality of large language models by retrieving relevant documents from credible literature and integrating them into the context of the user query.Recently, the rise of the cloud RAG service has made it possible for users to query relevant documents conveniently.However, directly sending queries to the cloud brings potential privacy leakage.In this paper, we are the first to formally define the privacy-preserving cloud RAG service to protect the user query and propose RemoteRAG as a solution regarding privacy, efficiency, and accuracy.For privacy, we introduce (n,\epsilon) -DistanceDP to characterize privacy leakage of the user query and the leakage inferred from relevant documents.For efficiency, we limit the search range from the total documents to a small number of selected documents related to a perturbed embedding generated from (n,\epsilon) -DistanceDP, so that computation and communication costs required for privacy protection significantly decrease.For accuracy, we ensure that the small range includes target documents related to the user query with detailed theoretical analysis.Experimental results also demonstrate that RemoteRAG can resist existing embedding inversion attack methods while achieving no loss in retrieval under various settings.Moreover, RemoteRAG is efficient, incurring only 0.67 seconds and 46.66KB of data transmission (2.72 hours and 1.43 GB with the non-optimized privacy-preserving scheme) when retrieving from a total of 10^5 documents.

## introduction large language models (llms) have attracted widespread attention since the release of chatgpt @xcite . however, llm is not without its flaws. one major issue is its tendency to generate factually incorrect or purely fictional responses, a phenomenon known as hallucination @xcite . to mitigate this problem, retrieval-augmented generation (rag) @xcite ) has been proposed to offer credible external knowledge, providing significant convenience for numerous tasks. rag aims to understand the input query, extract relevant information from external data sources, and enhance the quality of the generated answers @xcite @xcite . specifically, rag allows the retrieval of relevant documents which can help understand or answer the input query and inserts them into the context of prompts to improve the output of llm @xcite . its ability to enable llm to provide answers with credible literature makes rag an important technique in the application of llm, leading to the development of many excellent and user-friendly open-source rag projects @xcite @xcite . to leverage the power of rag, a new concept rag-as-a-service (raas) has been proposed, gathering significant attention @xcite . in raas, the rag service is entirely hosted online in the cloud. the user submits requests to the cloud with input queries to receive responses from raas. in this scenario, the cloud serves as the maintainer of the rag service. while the current solution facilitates the wide adoption of raas, it raises serious privacy concerns. the input query may contain sensitive information, such as health conditions and financial status. unfortunately, this data is not protected and must be uploaded in plaintext to the cloud in order to retrieve relevant documents. in this study, we aim to tackle a challenging question: how can we minimize privacy leakage in queries for raas while ensuring the accuracy of the responses, all with minimal additional costs? targeting the question above, we design a novel solution remoterag. for privacy, we propose (𝑛, 𝜖)-distancedp inspired by differential privacy and an embedding perturbation mechanism, so that the user can control the privacy leakage with a privacy budget 𝜖 in 𝑛-dimensional space of embeddings. we further study the potential privacy leakage in averaging the most relevant embeddings and find it within the constraint of (𝑛, 𝜖)-distancedp most of the time. for efficiency, we limit the search range from the total documents to a small number of selected documents, which are the relevant documents to a perturbed embedding generated from (𝑛, 𝜖)-distancedp. this small search range can save a significant amount of computation and communication costs used for privacy protection. for accuracy, we theoretically analyze the minimum size of the relevant documents to the perturbed embedding to ensure that they do contain target documents for the original query. contributions of this paper are listed as follows: • to the best of our knowledge, we are the first to address the privacy-preserving cloud rag service problem. we formally define the privacypreserving cloud rag service and characterize its corresponding threat model. ## problem formulation we first formally define the problem in developing a privacy-preserving llm cloud rag service. the main notations used in this paper are listed in ## conclusion in this paper, we are the first to address and formally define the privacy-preserving cloud rag service problem. we propose remoterag as a solution regarding privacy, efficiency, and accuracy. (𝑛, 𝜖)-distancedp is introduced to characterize the privacy leakage of the user query. the perturbation limits the search range, significantly saving computation and communication costs. theoretical analysis ensures the accuracy. experimental results also demonstrate the superiority of remoterag in privacy, efficiency, and accuracy, compared to privacy-ignorant and privacy-conscious services.