Text Embeddings Reveal (Almost) As Much As Text (2023)

Abstract: How much private information do text embeddings reveal about the original text? We investigate the problem of embedding inversion , reconstructing the full text represented in dense text embeddings. We frame the problem as controlled generation: generating text that, when reembedded, is close to a fixed point in latent space. We find that although a naive model conditioned on the embedding performs poorly, a multi-step method that iteratively corrects and re-embeds text is able to recover 92% of 32-token text inputs exactly. We train our model to decode text embeddings from two state-of-the-art embedding models, and also show that our model can recover important personal information (full names) from a dataset of clinical notes.

## introduction systems that utilize large language models (llms) often store auxiliary data in a vector database of dense embeddings @xcite . users of these systems infuse knowledge into llms by inserting retrieved documents into the language model's prompt. practitioners are turning to hosted vector database services to execute embedding search efficiently at scale @xcite . in these databases, the data owner only sends embeddings of text data @xcite to the third party service, and never the text itself. the database server returns a search result as the index of the matching document on the client side. vector databases are increasingly popular, but privacy threats within them have not been comprehensively explored. can the third party service to reproduce the initial text, given its embedding? neural networks are in general non-trivial or even 1 our code is available on github: github.com/jxmorris12/vec2text. impossible to invert exactly. furthermore, when querying a neural network through the internet, we may not have access to the model weights or gradients at all. still, given input-output pairs from a network, it is often possible to approximate the network's inverse. work on inversion in computer vision @xcite has shown that it is possible to learn to recover the input image (with some loss) given the logits of the final layer. preliminary work has explored this question for text @xcite , but only been able to recover an approximate bag of words given embeddings from shallow networks. in this work, we target full reconstruction of input text from its embedding. if text is recoverable, there is a threat to privacy: a malicious user with access to a vector database, and text-embedding pairs from the model used to produce the data, could learn a function that reproduces text from embeddings. we frame this problem of recovering textual embeddings as a controlled generation problem, where we seek to generate text such that the text is as close as possible to a given embedding. our method, vec2text, uses the difference between a hypothesis embedding and a ground-truth embedding to make discrete updates to the text hypothesis. when we embed web documents using a state-ofthe-art black-box encoder, our method can recover 32-token inputs with a near-perfect bleu score of 97.3, and can recover 92% of the examples exactly. we then evaluate on embeddings generated from a variety of common retrieval corpuses from the beir benchmark. even though these texts were not seen during training, our method is able to perfectly recover the inputs for a number of datapoints across a variety of domains. we evaluate on embeddings of clinical notes from mimic and are able to recover 89% of full names from embedded ## overview: embedding inversion text embedding models learn to map text sequences to embedding vectors. embedding vectors are useful because they encode some notion of semantic similarity: inputs that are similar in meaning should have embeddings that are close in vector space @xcite . embeddings are commonly used for many tasks such as search, clustering, and classification @xcite @xcite . given a text sequence of tokens @xmath0, a text encoder ϕ : v @xmath1. now consider the problem of inverting textual embeddings: given some unknown encoder ϕ, we seek to recover the text x given its embedding @xmath2. text embedding models are typically trained to encourage similarity between related inputs @xcite . thus, we can write the problem as recovering text that has a maximally similar embedding to the ground-truth. we can formalize the search for text x with embedding e under encoder ϕ as optimization: assumptions of our threat model. in a practical sense, we consider the scenario where an attacker wants to invert a single embedding produced from a black-box embedder ϕ. we assume that the attacker has access to ϕ: given hypothesis text x, the attacker can query the model for ϕ(x) and compute cos(ϕ(x), e). when this term is 1 exactly, the attacker can be sure that x was the original text, i.e. collisions are rare and can be ignored. ## experimental setup embeddings. vec2text is trained to invert two state-of-the-art embedding models: gtr-base @xcite , a t5-based pre-trained transformer for text retrieval, and text-embeddings-ada-002 available via the openai api. both model families are among the highest-performing embedders on the mteb text embeddings benchmark @xcite . ## defending against inversion attacks is it easy for users of text embedding models protect their embeddings from inversion attacks? we consider a basic defense scenario as a sanity check. to implement our defense, the user addes a level of gaussian noise directly to each embedding with the goal of effectively defending against inversion attacks while preserving utility in the nearestneightbor retrieval setting. we analyze the trade-off between retrieval performance and reconstruction accuracy under varying levels of noise. formally, we define a new embedding model as: we simulate this scenario with ϕ as gtr-base using our self-corrective model with 10 steps of correction, given the noisy embedder ϕ noisy . to measure retrieval performance, we take the mean ndcg@10 (a metric of retrieval performance; higher is better) across 15 different retrieval tasks from the beir benchmark, evaluated across varying levels of noise. we graph the average retrieval performance in figure 2 (see a.2 for complete tables of results). at a noise level of λ = 10 -1 , we see retrieval performance is preserved, while bleu score drops by 10%. at a noise level of 0.01, retrieval performance is barely degraded (2%) while reconstruction performance plummets to 13% of the original bleu. adding any additional noise severely impacts both retrieval performance and reconstruction accuracy. these results indicate that adding a small amount of gaussian noise may be a straightforward way to defend against naive inversion attacks, although it is possible that training with noise could in theory help vec2text recover more accurately from ϕ noisy . note that low reconstruction bleu score is not necessarily indicative that coarser inferences, such as clinical area or treatment regimen, cannot be made from embeddings. ## analysis how much does the model rely on feedback from ϕ? figure 3 shows an ablation study of the importance of feedback, i.e. performing corrections with and without embedding the most recent hypothesis. the model trained with feedback (i.e. additional conditioning on ϕ(x (t) ) is able to make a more accurate first correction and gets better bleu score with more rounds. the model trained with no feedback can still edit the text but does not receive more information about the geometry of the embedding space, and quickly plateaus. the most startling comparison is in terms of the number of exact matches: after 50 rounds of greedy selfcorrection, our model with feedback gets 52.0% of examples correct (after only 1.5% initially); the model trained without feedback only perfectly matches 4.2% of examples after 50 rounds. during training, the model only learns to correct a single hypothesis to the ground-truth sample. given new text at test time, our model is able to correct the same text multiple times, "pushing" the text from 0.9 embedding similarity to 1.0. we plot the closeness of the first hypothesis to the groundtruth in the training data for the length-32 model in figure 4 . we see that during training the model learns to correct hypotheses across a wide range of closenesses, implying that corrections should not go 'out-of-distribution' as they approach the ground-truth. how informative are embeddings for textual recovery? we graph bleu score vs. cosine similarity from a selection of of reconstructed text inputs in figure 5 . we observe a strong correlation between the two metrics. notably, there are very few generated samples with high cosine similarity but low bleu score. this implies that better following embedding geometry will further improves systems. theoretically some embeddings might be impossible to recover. prior work @xcite has shown that two different sequences can 'collide' in text embedding space, having similar embeddings even without any word overlap. however, our experiments found no evidence that collisions are a problem; they either do not exist or our model learns during training to avoid outputting them. improved systems should be able to recover longer text. does having a strong base model matter? we ablate the impact of initialization by evaluating our 32-token wikipedia model at different initializations of x (0) , as shown in section 7. after running for 20 steps of correction, our model is able to recover from an unhelpful initialization, even when the initialization is a random sequence of tokens. this suggests that the model is able to ignore bad hypotheses and focus on the true embedding when the hypothesis is not helpful. ## related work inverting deep embeddings. the task of inverting textual embeddings is closely related to research on inverting deep visual representations in computer vision @xcite @xcite , which show that a high amount of visual detail remains in the logit vector of an image classifier, and attempt to reconstruct input images from this vector. there is also a line of work reverse-engineering the content of certain text em-input nabo gass (25 august, 1954 in ebingen, germany) is a german painter and glass artist. round 1 (0.85): nabo gass (11 august 1974 in erlangen, germany) is an artist. ✗ round 2 (0.99): nabo gass (b. 18 august 1954 in egeland, germany) is a german painter and glass artist. ✗ round 3 (0.99): nabo gass (25 august 1954 in ebingen, germany) is a german painter and glass artist. ✗ round 4 (1.00): nabo gass (25 august, 1954 in ebingen, germany) is a german painter and glass artist. ## conclusion we propose vec2text, a multi-step method that iteratively corrects and re-embeds text based on a fixed point in latent space. our approach can recover 92% of 32-token text inputs from their embeddings exactly, demonstrating that text embeddings reveal much of the original text. the model also demonstrates the ability to extract critical clinical information from clinical notes, highlighting its implications for data privacy in sensitive domains like medicine. our findings indicate a sort of equivalence between embeddings and raw data, in that both leak similar amounts of sensitive information. this equivalence puts a heavy burden on anonymization requirements for dense embeddings: embeddings should be treated as highly sensitive private data and protected, technically and perhaps legally, in the same way as one would protect raw text.