Text Embeddings Reveal (Almost) As Much As Text
Authors: John Xavier Morris
Volodymyr Kuleshov
Vitaly Shmatikov
Alexander M Rush
Venue: EMNLP 2023 Main (2023)
Keywords: text embeddings
text retrieval
privacy
inversion
leakage attack
TL;DR: We show that we can reconstruct text from text embeddings and this threat model can be a serious violation of user privacy.

Abstract: How much private information do text embeddings reveal about the original text? We investigate the problem of embedding \textit{inversion}, reconstructing the full text represented in dense text embeddings. We frame the problem as controlled generation: generating text that, when reembedded, is close to a fixed point in latent space. We find that although a naive model conditioned on the embedding performs poorly, a multi-step method that iteratively corrects and re-embeds text is able to recover 92% of 32-token text inputs exactly. We train our model to decode text embeddings from two state-of-the-art embedding models, and also show that our model can recover important personal information (full names) from a dataset of clinical notes.

Text Embeddings Reveal (Almost) As Much As Text
John X. Morris, Volodymyr Kuleshov, Vitaly Shmatikov, Alexander M. Rush Department of Computer Science Cornell University
Abstract
impossible invert exactly, Furthermore, when querying a neural network through the internet, we may not have access to the model weights or gradi- ents at all.
How much private information do text em- beddings reveal about the original text? We investigate the problem of embedding inver- sion, reconstructing the full text represented in dense text embeddings. We frame the prob- lem as controlled generation: generating text that, when reembedded, is close fixed point in latent space. We find that although a naïve model conditioned on the embedding performs poorly, a multi-step method that iteratively cor- rects and re-embeds text is able to recover 92% of 32-token text inputs exactly. We train our model to decode text embeddings from two state-of-the-art embedding models, and also show that model recover important per- sonal information (full names) from a dataset of clinical notes."
Still, given input-output pairs from network, often possible to approximate the network's inverse. Work on inversion in computer vision (Mahendran and Vedaldi, 2014; Dosovitskiy and Brox, 2016) has shown that it is possible to learn to recover the input image (with some loss) given the logits of the final layer. Preliminary work has explored this question for text (Song and Raghu- nathan, 2020), but only been able to recover approximate bag of words given embeddings from shallow networks.
In this work, we target full reconstruction of in- put text from its embedding. If text is recoverable, there is threat to privacy: malicious cess to a vector database, and text-embedding pairs from the model used to produce the data, could learn a function that reproduces text from embed- dings.
with
1 Introduction
Systems that utilize large language models (LLMs) often store auxiliary data a vector database of dense embeddings (Borgeaud et al., 2022; Yao et al., 2023). Users of these systems infuse knowl- edge into LLMs by inserting retrieved documents into the language model's prompt. Practition- ers are turning to hosted vector database services to execute embedding search efficiently scale (Pinecone; Qdrant; Vdaas; Weaviate; LangChain). In these databases, the data owner only sends em- beddings of text data (Le and Mikolov, 2014; Kiros et al., 2015) to the third party service, and never the text itself. The database server returns search result as the index of the matching document on the client side.
We frame this problem of recovering textual em- beddings as a controlled generation problem, where we seek generate text such that the text is as close as possible to given embedding. Our method, Vec2Text, uses the difference between a hypothesis embedding and a ground-truth embedding to make discrete updates to the text hypothesis.
When we embed web documents using state-of- the-art black-box encoder, our method can recover 32-token inputs with a near-perfect BLEU score of 97.3, and can recover 92% of the examples exactly. We then evaluate on embeddings generated from variety of common retrieval corpuses from the BEIR benchmark. Even though these texts were not seen during training, our method is able to per- feetly recover the inputs for a number of datapoints across a variety of domains. We evaluate on em- beddings of clinical notes from MIMIC and are able to recover 89% of full names from embedded
Vector databases increasingly popular, but privacy threats within them have not been compre- hensively explored. Can the third party service to reproduce the initial text, given its embedding? Neural networks are in general non- trivial or even
'Our code is available on Github:
github.com//xmorrst2/. 2/.ectttxt.

---

8100
Target embedding
Mage (foaled 2.2222))
American Trorouglene recehoras who
lirrty.
X
Correction
Kentucky Berty whion was Mage Amaricun Ther Thoroughteed sediiiteeedeee horan and mome
Figure I: Overview of our method, Vec2TexL. Given access to a target embedding e (blue) and query access to an embedding model ó (blue model), the system aims iteratively generate (yellow model) hypotheses è (pink) reach the target. Example input is taken from recent Wikipedia article (June 2023). Vec2Text perfectly recovers this text from embedding after 4 rounds of correction.
notes. These results imply that text embeddings present the same threats privacy the text from which they computed, and embeddings should be treated with the same precautions raw data.
Assumptions of our threat model. In practical sense, consider the scenario where attacker wants invert single embedding produced from black-box embedder á. We assume that the tacker access to &: given hypothesis text å, the attacker can query the model for (()) and compute cos( (a(())))) e). When this term
2 Overview: Embedding Inversion
exactly, the at- tacker can be sure that was the original text, i.c. collisions are rare and can be ignored.
Text embedding models learn to map text se- quences embedding vectors. Embedding vec- tors are useful because they encode some notion of semantic similarity: inputs that are similar in meaning should have embeddings that are close in vector space (Mikolov et al., 2013). Embeddings are commonly used for many tasks such as search, clustering, and classification (Aggarwal and Zhai, 2012; Neelakantan et al., 2022; Muennighoff et al., 2023).
3 Method: Vec2Text
3.1 Base Model:
Learning to Invert ø
Enumerating all possible sequences to compute Equation (1) is computationally infcasible. One way to avoid this computational constraint is by learning distribution of texts given embeddings. Given dataset of texts D {xI,...}, we learn to invert encoder á by learning distribution of texts given embeddings, p(x e; e), by learning 0 via maximum likelihood:
Given a text sequence of tokens x C V text encoder é 7 + R& maps to fixed-length embedding vector E add
Now consider the problem of inverting textual embeddings: given some unknown encoder é, we seek to recover the text given its embedding =z)). Text embedding models are typically trained encourage similarity between related in- puts (Karpukhin et al., 2020). Thus, we can write the problem as recovering text that has a maxi- mally similar embedding to the ground-truth. We can formalize the search for text f with embedding under encoder é optimization:
G = arg max ppp ((x);
We drop the hereon for simplicity of notation. In practice, this process involves training condi- tional language model to reconstruct unknown text given its embedding learning problem amortizing the combinatorial optimization (Equation (1)) into the weights of neural network. Directly learning to generate sat- isfactory text in this manner is well-known in the literature to be difficult problem.
Φ(x). We can view this
t = arg max COS( (0(z),
(1)

---

3.2
Controlling Generation for Inversion
which requires sequence of embeddings as input with some dimension dene not necessarily equal to the dimension d of ø's embeddings. Similar to Mokady et al. (2021), we use small MLP to project a single embedding vector to a larger size, and reshape to give it sequence length as input to the encoder. For embedding E Re;
To improve upon this model, we propose Vec2Text shown in Figure 1. This approach takes inspira- tion from methods for Controlled Generation, the task of generating text that satisfies known con- dition (Hu et al., 2018; John et al., 2018; Yang and Klein, 2021). This task is similar to inversion in that there is a observable function that deter- mines the level of control. However, it differs in that approaches to controlled generation (Dathathri et al., 2020; Li et al., 2022) generally require differ- entiating through & to improve the score of some intermediate representation. Textual inversion dif- fers in that we can only make queries to o, and cannot compute its gradients.
EmbToSeq = W2 (Wi
where Wi E Rdxd and W2 E IR (dddm))) for some nonlinear activation function a and predetermined encoder "length" 8. We use a separate MLP to project three vectors: the ground-truth embedding e, the hypothesis embedding a(t), and the differ- ence between these vectors é. Given the word embeddings of the hypothesis (+(+) are [wi...w,). the input (length 3s n) to the encoder is as fol- lows:
Model. The method guesses an initial hypoth- esis and iteratively refines this hypothesis by re- embedding and correcting the hypothesis to bring its embedding closer to c. Note that this model re- quires computing new embedding ę(t) ((x(1)) in order generate each new correction (+(+1)) We define our model recursively by marginalizing over intermediate hypotheses:
concat (onca((EmbToSq((e))
(mm EmbToSeq(e e(!)), (wI...
We feed the concatenated input to the encoder and train the full decoder model using standard language modeling loss.
((x(1+1) (e)= = =p(al0 e)p(z((())) (e,x(",e)))
x(1)
((() = ((x!0)
Inference. In practice we cannot tractably sum out intermediate generations z(()) so we approxi- mate this summation via beam search. We perform inference from model greedily the token level but implement beam search at the sequence-level (t)) At each step of correction, we consider some number of possible corrections as the next step. For each possible correction, we decode the top possible continuations, and then take the top b unique continuations out of b potential contin- uations by measuring their distance in embedding space to the ground-truth embedding
with base case of the simple learned inversion:
(0) e) = (0) | 1,0,
Here, (0) represents the initial hypothesis gener- ation, (1)) the correction of 2(("), and so on. We train this model by first generating hypotheses (0) from the model in Section 3.1, computing 2!0), and then training model on this generated data.
This method relates to other recent work gener- ating text through iterative editing (Lee et al., 2018; Ghazvininejad et al., 2019). Especially relevant is Welleck al. (2022), which proposes to train a text-to-text 'self-correction' module to improve language model generations with feedback.
Experimental Setup
Embeddings. Vec2Text is trained to invert two state-of-the- art embedding models: GTR-bas (Ni et al., 2021), T5-based pre-trained transformer for text retrieval, and www.eeeendnnndddaaaaaae-eaaaa-ae available via the OpenAl API. Both model fami- lies are among the highest-performing embedders on the MTEB text embeddings benchmark (Muen- nighoff et al., 2023).
Parameterization. The backbone of our model, p(z(++1)
is parameterized as stan- dard -ecoder transformer (Vaswani et al., 2017; Raffel al., 2020) conditioned on the previ- ous output.
One challenge is the need to input conditioning embeddings e and a(0) into a transformer encoder,

---

method
tokens
pred tokens
bleu
ifl
exact
cOs
Bag-of-words (Song and Raghunathan, 2020)
32
32
0.3
51
0.0
0.70
GPT-2 Decoder (Li et al., 2023)
32
32
1.0
47
0.0
0.76
GTR Natural Questions
Base |0 steps]
32
32
31.9
67
0.0
0.91
(+ beam search)
32
32
34.5
67
1.0
0.92
nucleus)
32
32
25.3
60
0.0
0.88
Vec2Text [1 step]
32
50.7
80
0.0
0.96
[20 steps]
32
83.9
40.2
0.99
[50 steps]
32
32
85.4
97
40.6
0.99
[50 steps + sbeam]
32
32
97.3
99
92.0
0.99
Base steps]
31.8
31.8
26.2
0.0
0.94
OpenAl MSMARCO
Vec2Text step]
31.8
31.9
44.1
5.2
0.96
[20 steps]
31.8
31.9
61.9
87
15.0
0.98
[50 steps]
31.8
31.9
62.3
87
14.8
0.98
[50 steps + sbeam]
31.8
31.8
83.4
96
60.9
0.99
Base [0 steps]
80.9
84.2
17.0
54
0.6
0.95
MSMARCO
OpenAI
Vec2Text [I step]
80.9
81.6
29.9
68
1.4
0.97
[20 steps]
80.9
79.7
43.1
78
3.2
0.99
[50 steps]
80.9
80.5
44.4
78
3.4
0.99
[50 steps sbeam]
80.9
80.6
55.0
84
8.0
0.99
Table 1: Reconstruction score on in-domain datasets. Top section of results come from models trained to reconstruct tokens of text from Wikpedia, embedded using GTR-base. Remaining results come from models trained to reconstruct up 32 128 tokens from MSMARCO, embedded using OpenAl text-embeddings-aaaaa
Datasets. We train our GTR model on 5M pas- sages from Wikipedia articles selected from the Natural Questions corpus (Kwiatkowski et al., 2019) truncated to 32 tokens. We train our two OpenAl models (Bajaj et al., 2018), both on sions of the MSMARCO corpus with maximum 32 128 tokens per example For evaluation, consider the evaluation datasets from Natural Ques- tions and MSMarco, as well as two out-of-domain settings: the MIMIC-III database clinical notes (Johnson et al., 2016) in addition to the variety of datasets available from the BEIR benchmark (Thakur et al., 2021).
Metrics. We use two types of metrics to measure the progress and the accuracy of reconstructed text. First we consider our main goal of text reconstruc- tion. To measure this we use word-match metrics including: BLEU score (Papineni al., 2002), measure of n-gram similarities between the true and reconstructed text; Token F1, the multi-class Fi score between the set of predicted tokens and the set of true tokens; Exact-match, the percent- age of reconstructed outputs that perfectly match the ground-truth. We also report the similarity on the internal inversion metric in terms of recovering the vector embedding in latent space. We use co- sine similarity between the true embedding and the embedding of reconstructed text according d.
Baseline. As a baseline, we train the base model (0) e) to recover text with no correction steps. We also evaluate the bag of words model from Song and Raghunathan (2020). To balance for the increased number of queries allotted to the cor- rection models, also consider taking the top-N predictions made from the unconditional model via beam search and nucleus sampling (p 1p=0.99 0.9) and reranking via cosine similarity.
Models and Inference. We initialize our models from the T5-base checkpoint (Raffel et al., 2020). Including the projection head, each model has ap- proximately 235M parameters. We set the projec- tion sequence length 16 for all experiments, as preliminary experiments show diminishing re- turns by increasing this number further. We per- form inference on all models using greedy token- level decoding. When running multiple steps
'By 2023 pricing of $0. 00.0001 per 1000 tokens, emhedding 5 million documents of 70 tokens each costs $35.

---

dataset
tokens
method
bleu
token FI
2 10-4 with warmup and linear decay. We train models for 100 epochs. We use batch size of 128 and train all models on NVIDIA A6000 GPUs. Under these conditions, training our slowest model takes about two days.
Base
36.2
73,8
quora
15.7
Vec2Text
95.5
98.6
Base
13.2
49.5
signal
23.7
Vec2Text 80.7
92.5
Base
15.5
54.1
msmarco
72.1
Vec2Text 59.6
86.1
5
Results
Base Vec2Text 44.9
12.8
49.3
climate-fever
73.4
5.1 Reconstruction: In-Domain
82.6
Base 12.6
49.2
Table 1 contains in-domain results. Our method outperforms the baselines on all metrics. More rounds is monotonically helpful, although we see diminishing returns we are able to recover 77% of BLEU score in just rounds of correction, al- though running for 50 rounds indeed achieves a higher reconstruction performance. We find that running sequence-level beam search (sbeam) over the iterative reconstruction is particularly helpful for finding exact matches of reconstructions, in- creasing the exact match score by to times across the three settings. a relative sense, the model has more trouble exactly recovering longer texts, but still is able to get many of the words.
fever
73.4
Vec2Text 45.1
82.7
Base 15.4
50.3
dbpedia-entity
91.3
Vec2Text 48.0
77.9
Base 11.0
47.1
ng
94.7
Vec2Texx 32.7
72.7
Base
15.4
50.1
hotpotqa
94.8
Vec2Text 46.6
78.7
Base Vec2Text 21.5
6.6
44.1
fiqa
103.8
63.6
Base Vec2Text
41.5
wehis-touche2022
105.2
19.6
69.7
Base
7.1
41.5
cqadupstack
106.4
Vec2Text 23.3
64.3
Base 6.8
44.1
arguana
113.5
Vec2Text 23.4
66.3
5.2
Reconstruction: Out-of-Domain
Base
5.9
38.5
scidocs
125.3
We evaluate our model on 15 datasets from the BEIR benchmark and display results in Table 2. Quora, the shortest dataset BEIR, is the easiest to reconstruct, and our model is able to exactly re- cover 66% of examples. Our model adapts well to different-length inputs, generally producing re- constructions with average length error of fewer than 3 tokens. In general, reconstruction accuracy inversely correlates with example length (discussed more Section 7). On all datasets, we are able to recover sequences with Token F1 of least and cosine similarity to the true embedding of at least 0.95.
Vee2Text 17.7
57.6
Base 5.6
36.3
trec-covid
125.4
Vee2Text 19.3
58.6
Base 4.9
34.4
rubust04
127.3
Vec2Text 15.5
54.5
Base
5.3
35.7
bioasq
127.4
Vec2Text 22.8
59.5
Base 4,9
35.2
scifact
127.4
Vec2Text 16.6
56.6
Base Vec2Text 25.8
6.2
39.6
nicorpus
127.7
64.8
Base
4.9
34.8
trec-news
128.0
Vec2Text
14.5
51.5
Table 2: Out-of-domain reconstruction performance measured datasets from the BEIR benchmark. We sort datasets in order of average length in order to em- phasize the effect of sequence length on task difficulty.
5.3
Case study: MIMIC
As specific threat domain, we consider MIMIC- III clinical notes (Johnson et al., 2016). Because the original release of MIMIC is completely deiden- tified, we instead use the "pseudo re-identified" ver- sion from Lehman et al. (2021) where fake names have been inserted in the place of the deidentified ones.
sequence-level beam search, we only take new generation it is closer than the previous step in cosine similarity to the ground-truth embedding.
We use unconditional models to seed the initial hypothesis for our iterative models. We examine the effect of using a different initial hypothesis in Section 7.
Each note is truncated 32 tokens and the notes are filtered so that they each contain least one name. We measure the typical statisties of our method as well as three new ones: the percentage of first names, last names, and complete names
We use the Adam optimizer and learning rate of

---

method
first
last
full
bleu
0.30
Retrieval NDCG@10
Base
40.0
27.8
10.8
4.9
33.1
0.
0.78
Reconstruction
Vec2Text
94.2
95.3
89.2
55.6
80.8
26.0
0.98
0220
MIMGC-II Entity reconstruction rate
0.15
Recanstruct
Clinical event
0.10
Nonbldlogical location
20
Therapeutic precedure
0.05
0.00
0
Blolgiiaa
Noise Level (A)
Sign symptom
Disease disorder
Medication
Figure Retrieval performance and reconstruction ac- curacy across varying levels of noise injection.
Diagnstiic procedure
betalled description
0.0
Gaussian noise directly to each embedding with the goal of effectively defending against inver- sion attacks while preserving utility in the nearest- neightbor retrieval setting. We analyze the trade-off between retrieval performance and reconstruction accuracy under varying levels of noise.
Table 3: Performance of our method on reconstructing GTR-embedded clinical notes from MIMIC III (Johnson al., 2016).
that are recovered. Results are shown in Table 3. Vec2Text is able to recover 94% of first names, 95% of last names, and 89% of full names (first, last format) while recovering 26% of the docu- ments exactly.
Formally, we define new embedding model as:
Pruisy (z)) x(x)) A 1.e. N(0
For the recovered clinical notes from Section 5.3, we extract entities from each and recovered note using a clinical entity extractor (Raza al., 2022). We plot the recovery percentage (bot- tom) with the average entity recovery shown as a dashed line. Our model most accurate at re- constructing entities of the type "Clinical Event", which include generic medical words like 'arrived', "progress', and 'transferred". Our model is least accurate in the "Detailed Description" category, which includes specific medical terminology like "posterior' and 'hypoxic', as well as multi-word events like 'invasive ventilation stop 4:00 pm'.
where À is hyperparameter controlling the amount of noise injected.
We simulate this scenario with GTR-base using our self-corrective model with 10 steps of correction, given the noisy embedder @ruisy- To measure retrieval performance, we take the mean NDCG@ 10 (a metric of retrieval performance; higher is better) across 15 different retrieval tasks from the BEIR benchmark, evaluated across vary- ing levels of noise.
We graph the average retrieval performance Figure 2 (see A.2 for complete tables of results), At noise level of X = 10 we see retrieval per- formance is preserved, while BLEU score drops by 10%. At noise level of 0.01, retrieval perfor- mance barely degraded (2%) while reconstruc- tion performance plummets to 13% of the original BLEU, Adding any additional noise severely im- pacts both retrieval performance and reconstruction accuracy. These results indicate that adding small amount of Gaussian noise may be straightforward way to defend against naive inversion attacks, al- though is possible that training with noise could in theory help Vec2Text recover more accurately from @noisy. Note that low reconstruction BLEU score not necessarily indicative that coarser infer- ences, such as clinical area or treatment regimen, cannot be made from embeddings.
Although we are able to recover 26% of 32- token notes exactly, the notes that were not exactly recovered are semantically close to the original. Our model generally matches the syntax of notes, even when some entities slightly garbled; for example, given the following sentence from doc- tor's note "Rhona Arntson npn/- resp: infant re- mains orally intubated on imv r fi" our model predicts "Rhona Arpson nrft:# infant remains intu- bated orally on resp. imv. m/n fi*
6 Defending against inversion attacks
Is it easy for users of text embedding models pro- tect their embeddings from inversion attacks? We consider a basic defense scenario as sanity check. To implement our defense, the user addes a level of

---

80
25000
60
20000
BLED
Count
15000
40
10900
20
Feedback
No feedback
0
0
20
40
0.70
0.95
Iteration (t)
Cosine distance from hypothesis to true embedding
Figure Distribution of cos(e, () over training data. The mean training output from the GTR base
Figure 3: Recovery performance across multiple rounds of self-correction comparing models with access to @ vs text-only (32 tokens per sequence).
model cosine similarity of 0.924 with the true embedding.
7 Analysis
few generated samples with high cosine similarity but low BLEU score. This implies that better fol- lowing embedding geometry will further improves systems. Theoretically some embeddings might be impossible to recover. Prior work (Song et al., 2020; Morris, 2020) has shown that two different sequences 'collide' in text embedding space, having similar embeddings even without any word overlap. However, our experiments found ev- idence that collisions are a problem; they either do not exist or our model Iearns during training to avoid outputting them. Improved systems should be able to recover longer text.
How much does the model rely on feedback from @? Figure shows an ablation study of the im- portance of feedback, i.e. performing corrections with and without embedding the most recent hy- pothesis. The model trained with feedback (i.e. ad- ditional conditioning on (xx(1) is able make accurate first correction and gets better BLEU score with more rounds. The model trained with no feedback can still edit the text but does not receive more information about the gcometry of the embedding space, and quickly plateaus. The most startling comparison is in terms of the number of cxact matches: after 50 rounds of greedy self- correction, our model with feedback gets 52.0% of examples correct (after only 1.5% initially); the model trained without feedback only perfectly matches 4.2% of examples after 50 rounds.
Does having strong base model matter? We ablate the impact of initialization by evaluating 32-token Wikipedia model different initializa- tions of (0)), as shown Section After running for 20 steps correction, our model is able to re- cover from an unhelpful initialization, even when the initialization is a random sequence of tokens. This suggests that the model is able to ignore bad hypotheses and focus on the true embedding when the hypothesis is not helpful.
During training, the model only learns to cor- rect single hypothesis to the ground-trui truth sample. Given new text at test time, our model is able to correct the same text multiple times, "pushing" the text from 0.9 embedding similarity to 1.0. We plot the closeness of the first hypothesis to the ground- truth in the training data for the length-32 model in Figure 4. We see that during training the model learns correct hypotheses across wide range closenesses, implying that corrections should not go "outtoo-disstriuution as they approach the ground-truth.
Related work
Inverting deep embeddings. The task of invert- ing textual embeddings is closely related to re- search on inverting deep visual representations in computer vision (Mahendran and Vedaldi, 2014; Dosovitskiy and Brox, 2016; Teterwak et al., 2021; Bordes al., 2021), which show that high amount of visual detail remains in the logit vector of im- age classifier, and attempt to reconstruct input im- ages from this vector. There is also a line of work reverse-enginerring the content certain text em-
How informative are embeddings for textual re- covery? We graph BLEU score vs. cosine simi- larity from selection of of reconstructed text in- puts in Figure 5. We observe a strong correlation between the two metrics. Notably, there are very

---

Input
Nabo Gass (25 August, 1954 in Ebingen, Germany) a German painter and glass artist.
Round (0.85):
Nabo Gass (11 August 1974 in Erlangen, Germany) is artist.
Round 2 (0.99):
Naho Gass (h, 18 August 1954 in Egeland, Germany) German painter and glass artist.
Round 3 (0.99):
Nabo Gass (25 August 1954 in Ebingen, Germany) is German painter and glass artist.
Round 4 (1.00):
Nabo Gass (25 August, 1954 in Ebingen. Germany) German painter and glass artist.
Table Example of our corrective model working in multiple rounds. Left column shows the correction number, from Round (initial hypothesis) to Round 4 (correct guess). The number in parenthesis is the cosine similarity between the guess's embedding and the embedding of the ground-truth sequence (first row).
Initialization
token fl
cos
exact
et al., 2020) shows that faces can be reconstructed from their deep embeddings. Similar questions have been asked about text data: Lehman et al. (2021) attempt to recover sensitive information such as names from representations obtained from a model pre-trained on clinical notes, but fail to recover exact text. Kim et al. (2022) propose a similarity mechanism for text embeddings and consider shallow bag-of- words inversion model. Abdalla et al. (2020) analyze the privacy leaks in training word embeddings on medical data and are able to recover full names in the training data from learned word embeddings. Dziedzic et al. (2023) note that stealing sentence en- coders by distilling through API queries works well and is difficult for API providers to prevent. Song and Raghunathan (2020) considered the problem of recovering text sequences from embeddings, but only attempted to recover bags of words from the embeddings of shallow encoder model. Li et al. (2023) investigate the privacy leakage of embed- dings by training a decoder with a text embedding as the first embedding fed to the decoder. Com- pared to these works, we consider the significantly more involved problem of developing a method to recover the full ordered text sequence from more realistic state-of- teefthhh-rtt text retrieval models.
Random tokens
0.95
0.99
50.0
"the
0.95
0.99
49.8
"there's no reverse on a motorcycle, us my friend found out quite dramatically the other day"
0.96
0.99
52.0
model p(z[0) cc)
0.96
0.99
51.6
Table 5: Ablation: Reconstruction score on Wikipedia data (32 tokens) given various initializations. Our self- correction model is able to faithfully recover the original text with greater than S0 BLEU score, even with poor initialization. Models run for 20 steps of correction.
100
80
8LED
60
40
20
0.85
0.95 Cosine similarity
0.90
1.00
Figure 5: Cosine similarity vs BLEU score on 1000 reconstructed embeddings from Natural Questions text.
beddings: Ram et al. (2023) analyze the contents of text embeddings by projecting embeddings into the model's vocabulary space to produce distri- bution of relevant tokens. Adolphs et al. (2022) train single-step query decoder to predict the text of queries from their embeddings and use the de- coder produce more data to train a new retrieval model. We focus directly on text reconstruction and its implications for privacy, and propose an iterative method that works for paragraph-length documents, not just sentence-length queries.
Gradient leakage. There parallels between the use of vector databases store embeddings and the practice of federated learning, where users share gradients with one another in order to jointly train a model. Our work on analyzing the pri- vacy leakage of text embeddings is analogous to research on gradient leakage, which has shown that certain input data can be reverse- enrerse-nngineeeed from the model's gradients during training (Melis et al., 2018; Zhu al., 2019; Zhao et al., 2020; Geiping al., 2020). Zhu et al. (2019) even shows that they can recover text inputs of masked language model by backpropagating to the input layer to match the gradient. However, such techniques do not apply
Privacy leakage from embeddings. Research has raised the question of information leakage from dense embeddings. In vision, Vec2Face (Duong

---

textual inversion: the gradient of the model is relatively high-resolution; we consider the more difficult problem of recovering the full input text given only a single dense embedding vector.
more sophisticated search algorithms top of our corrective module.
Scalability to long text. Our method is shown to recover most sequences exactly up to 32 tokens and some information up 128 tokens, but we have not investigated the limits of inversion beyond embed- dings of this length. Popular embedding models support embedding text content on the order of thousands tokens, and embedding longer texts is common practice (Thakur et al., 2021). 1). Future work might explore the potential and difficulties of inverting embeddings of these longer texts.
Text autoencoders. Past research has explored natural language processing learning models that map vectors to sentences (Bowman et al., 2016). These include some retrieval models that trained with a shallow decoder to reconstruct the text or hag-of-words from the encoder-outputted embedding (Xiao et al., 2022: Shen al., 2023; Wang et al., 2023). Unlike these, we invert embed- dings from a frozen, pre-trained encoder.
Access to embedding model. Our threat model assumes that an adversary has black-box access to the model used to generate the embeddings the compromised database. the real world, this is realistic because practitioners so often rely on the same few large models. However, Vec2Text requires making query to the black-box embed- ding model for each step of refinement. Future work might explore training imitation embed- ding model which could be queried at inference time to save queries to the true embedder.
9 Conclusion
We propose Vec2Text, a multi-step method that iteratively corrects and re-embeds text based on fixed point in latent space. Our approach can recover 92% of 32-noken text inputs from their em- beddings exactly, demonstrating that text embed- dings reveal much of the original text. The model also demonstrates the ability to extract critical clini- cal information from clinical notes, highlighting its implications for data privacy sensitive domains like medicine.
11 Acknowledgements
Our findings indicate a sort of equivalence be- tween embeddings and raw data, in that both leak similar amounts of sensitive information. This cquivalence puts a heavy burden on anonymization requirements for dense embeddings: embeddings should be treated as highly sensitive private data and protected, technically and perhaps legally, in the same way as one would protect raw text.
Thanks to Simran Arora and Piotr Teterwak for helpful conversations about the research. JXM is supported by a NSF GRFP. VK is supported by NSF 2145577 and NIH MIRA R35GM151243. VS is partially supported by NSF 1916717. AMR is supported by NSF 2242302, NSF CAREER 2037519, IARPA HIATUS, and Sloan Fellow- ship.
10 Limitations
References
Adaptive attacks and defenses. We consider the setting where adversary applies noise newly generated embeddings, but the reconstruction mod- ules were trained from un-noised embeddings. Fu- ture work might consider reconstruction attacks or defenses that are adaptive to the type of attack or defense being used.
Mohamed Abdalla, Moustafa Abdalla, Graeme Hirst, and Frank Rudzicz. 2020. Exploring the privacy- preserving properties of word embeddings: Al- gorithmic validation study. J Med Internet Res. 22(7):e18055,
Leonard Adolphs, Michelle Chen Huebscher, Christian Buck, Sertan Girgin, Olivier Bachem, Massimiliano Ciaramita, and Thomas Hofmann. 2022. Decoding neural retriever's latent space for query suggestion.
Search thoroughness. Our search is limited; in this work we do not test beyond searching for 50 rounds or with sequence beam width higher than 8. However, Vec2Text gets monotonically better with more searching. Future work could find even more exact matches by searching for more rounds with a higher beam width, or by implementing
Charu C. Aggarwal and ChengXiang Zhai. 2012. A Survey of Text Clustering Algorithms, pages 77-128. Springer US, Boston, MA.
Payal Bajaj, Daniel Campos, Nick Craswell, Li Deng, Jianfeng Gao, Xiaodong Liu, Rangan Majumder, An- drew McNamara, Bhaskar Mitra, Tri Nguyen, Mir

---

Rosenberg, Xia Song, Alina Stoica, Saurabh Tiwary, and Tong Wang. 2018. Ms marco: A human gener- ated machine reading comprehension dataset.
freely accessible critical care database. Scientific Data, 311):160035.
Vladimir Karpukhin, Barlas Oğuz. Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen tau Yih. 2020. Dense passage retrieval for open- domain question answering.
Florian Bordes, Randall Balestriero, and Pascal Vincent. 2021. High fidelity visualization of what your self- supervised representation knows about. Truns. Mach. Learn. Res., 2022.
Donggyu Kim, Garam Lee, and Sungwoo Oh. 2022. Toward privacy-presevving text embedding similarity with homomorphic encryption. In Proceedings of the Fourth Workshop on Financial Technology and Nat- ural Language Processing ( (FinNLP), pages 25-36, Abu Dhabi, United Arab Emirates (Hybrid). Associa- tion for Computational Linguistics.
Sebastian Borgeaud, Arthur Mensch, Jordan Hoffmann, Trevor Cai, Eliza Rutherford, Katie Millican, George van den Driessche, Saan-Bpptstte Lespiau, Bogdan Damoc, Aidan Clark, Diego de Las Casas, Aurelia Guy, Jacob Menick, Roman Ring, Tom Hennigan, Saffron Huang, Loren Maggiore, Chris Jones, Albin Cassirer, Andy Brock, Michela Paganini, Geoffrey Irving, Oriol Vinyals, Simon Osindero, Karen Si- monyan, Jack W. Rac, Erich Elsen, and Laurent Sifre. 2022. Improving language models by retrieving from trillions of tokens.
Ryan Kiros, Yukun Zhu, Ruslan Salakhutdinov, Richard S. Zemel, Antonio Torralba, Raquel Urta- sun, and Sanja Fidler. 2015. Skip-thought vectors.
Tom Kwiatkowski, Jennimaria Palomaki, Olivia Red- field, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Ken- ton Lee, Kristina Toutanova, Llion Jones, Matthew Kelcey, Ming- Wei Chang, Andrew M. Dai, Jakob Uszkoreit, Quoe Le, and Slav Petrov. 2019. Natu- ral questions: A benchmark for question answering research. Transactions of the Association for Compu- tational Linguistics, 7:452-466.
Samuel R. Bowman, Luke Vilnis, Oriol Vinyals, An- drew M. Dai, Rafal Jozefowicz, and Samy Bengio. 2016. Generating sentences from continuous space.
Sumanth Dathathri, Andrea Madotto, Janice Lan, Jane Hung, Eric Frank, Piero Molino, Jason Yosinski, and Rosanne Liu. 2020. Plug and play language models: A simple approach controlled text generation. In International Conference on Learning Representa- tions.
LangChain. 2023. Hwchase I /angghhain building ap- plications with Ilms through composability.
Alexey Dosovitskiy and Thomas Brox. 2016. Inverting visual representations with convolutional networks.
Quoc V. Le and Tomas Mikolov. 2014. Distributed representations of sentences and documents.
Chi Nhan Duong, Thanh Dat Truong, Kha Gia Quach, Hung Bui, Kaushik Roy, and Khoa Luu. 2020. Vec2face: Unveil human faces from their blackbox features in face recognition.
Jason Lee, Elman Mansimov, and Kyunghyun Cho. 2018. Deterministic non-autoregressive neural se- quence modeling by iterative refinement.
Adam Dziedzic, Franziska Boenisch, Mingjian Jiang, Haonan Duan, and Nicolas Papernot. 2023. Sentence embedding encoders easy to steal but hard to defend. In ICLR 2023 Workshop on Pitfalls of limited data and computation for Trustworthy MI..
Eric Lehman, Sarthak Jain, Karl Pichotta, Yoav Gold- berg, and Byron C. Wallace. 2021. Does bert pre- trained on clinical notes reveal sensitive data?
Haoran Li, Mingshi Xu, and Yangqiu Song. 2023. Sen- tence embedding lcaks more information than you expect: Generative embedding inversion attack to recover the whole sentence.
Jonas Geiping, Hartmut Bauermeister, Hannah Dröge, and Michael Moeller. 2020. Inverting gradients- how easy is it to break privacy federated learning?
Marjan Ghazvininejad. Omer Levy, Yinhan Liu, and Luke Zettlemoyer. 2019. preddctt Parallel de- coding of conditional masked language models.
Xiang Lisa Li, John Thickstun, Ishaan Gulrajani, Percy Liang, and Tatsunori B. Hashimoto. 2022. Diffusion- Im improves controllable text generation.
Zhiting Hu, Zichao Yang, Xiaodan Liang, Ruslan Salakhutdinov, and Eric P. Xing. 2018. Toward con- trolled generation of text.
Aravindh Mahendran and Andrea Vedaldi. 2014. Un- derstanding deep image representations by inverting them. 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 5188-5196.
Vineet John, Lili Mou, Hareesh Bahuleyan, and Olga Vechtomova. 2018. Disentangled representation learning for non-pane paraallll text style transfer.
Luca Melis, Congzheng Song, Emiliano De Cristofaro, and Vitaly Shmatikov. 2018. Exploiting unintended feature leakage in collaborative learning.
Alistair E.W. Johnson, Tom J. Pollard, Lu Shen, Li- wei H. Lehman, Mengling Feng, Mohammad Ghas- semi, Benjamin Moody, Peter Szolovits, Leo An- thony Celi, and Roger G. Mark. 2016. Mimic-iii,
Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Efficient estimation of word representa- tions in vector space.

---

Ron Mokady, Amir Hertz, and Amit Bermano, 2021. Clipcap: Clip prefix for image captioning.
Piotr Teterwak, Chiyuan Zhang, Dilip Krishnan, and Michael C. Mozer. 2021. Understanding invariance via feedforward inversion of discriminatively trained classifiers.
John X. Morris, 2020. Second-order nlp adversarial examples.
Nandan Thakur, Nils Reimers, Andreas Rücklé, Ab- hishek Srivastava, and Iryna Gurevych. 2021. Beir: A heterogenous benchmark for zero-shot evaluation of information retrieval models.
Niklas Muennighoff. Nouamane Tazi, Loic Magne, and Nils Reimers. 2023. Mteh: Massive text embedding benchmark.
Arvind Neelakantan, Tao Xu, Raul Puri, Alec Rad- ford, Jesse Michael Han, Jerry Tworek, Qiming Yuan, Nikolas Tezak, Jong Wook Kim, Chris Hallacy. Johannes Heidecke, Pranav Shyam, Boris Power, Tyna Eloundou Nekoul, Girish Sastry, Gretchen Krueger, David Schnurr, Felipe Petroski Such, Kenny Hsu, Madeleine Thompson, Tabarak Khan, Toki Sherbakov, Joanne Jang, Peter Welinder, and Lilian Weng. 2022. Text and code embeddings by con- trastive pre-training.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need.
daaas. 2023. Vdaas/vald: Vald. a highly scalable dis- tributed vector search engine.
Liang Wang. Nan Yang. Xiaolong Huang, Binxing Jiao, Linjun Yang, Daxin Jiang. Rangan Majumder, and Furu Wei. 2023. Simlm: Pre-training with represen- tation bottleneck for dense passage retrieval.
Jianmo Ni, Chen Qu, Jing Lu, Zhuyun Dai, Gus- tavo Hernández Abrego, Ji Ma, Vincent Y. Zhao, Yi Luan, Keith B. Hall, Ming-Wei Wei Chang, and Yinfei Yang. 2021. Large dual encoders generalizable retrievers.
Weaviate. 2023. Weaviate vector database.
Sean Welleck, Ximing Lu, Peter West, Faeze Brah- man, Tianxiao Shen, Daniel Khashabi, and Yejin Choi. 2022. Generating sequences by learning self-correct.
Kishore Papineni, Salim Roukos, Todd Ward, and Wei- Jing Zhu. 2002. Bleu: method for automatic evalu- ation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Compu- tational Linguistics, pages 31 1-3118, Philadelphia, Pennsylvania. USA. Association for Computational Linguisties.
Shitao Xiao, Zheng Liu, Yingxia Shao, and Zhao Cao. 2022. Retromae: Pre-training retrieval-oriented lan- guage models via masked auto-encoder.
Pinecone. 2023. Pinecone.
Kevin Yang and Dan Klein. 2021. FUDGE: Controlled text generation with future discriminators. In Pro- ceedings of the 2021 Conference of the North Amer- ican Chapter of the Association for Computational Linguistics: Human Language Technologies. Associ- ation for Computational Linguistics.
Qdrant. 2023. Qdrant vector database.
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text trans- former.
Shunyu Yao, Jeffrey Zhao, Dian Yu. Nan Du Izhak Shafran, Karthik Narasimhan, and Yuan Cao. 2023. React: Synergizing reasoning and acting in language models.
Ori Ram, Liat Bezalel, Adi Zicher, Yonatan Belinkov, Jonathan Berant, and Amir Gloherson. 2023. What are you token about? dense retrieval as distributions over the vocabulary.
Bo Zhao, Konda Reddy Mopuri, and Hakan Bilen. 2020.
idlg: Improved deep leakage from gradients.
Shaina Raza, Deepak John Reji, Femi Shajan, and Syed Raza Bashir. 2022. Large-scale application of named entity recognition to biomedicine and epi- demiology. PLOS Digital Health, 1(12):00000152.
Ligeng Zhu, Zhijian Liu, and Song Han. 2019. Deep leakage from gradients.
A Appendix
Tao Shen, Xiubo Geng, Chongyang Tao, Can Xu, Xiao- long Huang, Binxing Jiao, Linjun Yang, and Daxin Jiang. 2023. Lexmae: eexicon-bottleneen pretrain- ing for large-scale retrieval.
A.1 Additional analysis
How does word frequency affect model correct- ness? fig. 6 shows the number of correct predic- tions (orange) and incorrect predictions (blue) for ground- truth words, plotted across word frequency in the training data. Our model generally predicts words better that are more frequent in the training data, although is still able to predict correctly a
Congzheng Song and Ananth Raghunathan. 2020. Infor- mation leakage in embedding models. Proceedings of the 2020 ACM SIGSAC Conference on Computer and Communications Security.
Congzheng Song, Alexander M. Rush, and Vitaly Shmatikov. 2020. Adversarial semantic collisions.

---

Correctiness vs word frequency training on ArXiv
Falte negative
True poslive
5000
TM goowad woo
2000
100
107
100
Froquency training data
Figure 6: Correctness evaluation samples from ArXiv data.
number of words that were not seen during train- ing Peaks between 104 and 10 come from the characters -. and ), which appear frequently in the training data, but are still often guessed incor- rectly in the reconstructions.
A.2 Full defense results
Results on each dataset from BEIR under vary- ing levels of Gaussian noise are shown in Ap- pendix A.2. The model is GaRebase. Note that the inputs limited to 32tokens, far shorter than the average length for some corpuses, which is why baseline (A 0) NDCG@ numbers are lower than typically reported. We included the full results (visualized in Figure 2) as Appendix A.2.
"We hypothesize this because all test tokens were present in the training data, and the model is able to reconstruct unseen words from seen tokens.

---

orgrons
Hoong
clinore-fese dhpedts ennry
1qp
menarto
nntorpes
nitetli
scidees
seibet
+grallm
rec--omi
tnc-neos
bebrs-orcchellp
0
0:328
0.115
0.116
0:316
01:08
0.647
0.239
0.305
0.579
0.30
0.99
0.347
0:761
0.57
0.245
0.215
0.00
0.329
0.115
0.135
0.307
0.238
0.647
0.239
0.305
0.879
0.294
0.256
0.346
0.261
0.581
0.246
0.235
001
0.334
0.155
01112
0:301
0.385
0.615
0.234
0.296
0.85
0.19
0.00
0.215
0.389
0.978
0.214
0.235
Table 6: BEIR performance (NDCG@ @ 10) for GTR-base at varying levels of noise (32 tokens).
NDCG@10
BLEU
0.000
0.302
80.372
0.001
0.302
72.347
0.010
0.296
10.334
0.100
0.002
0.148
1,000
0.001
0.080
Table 7: Retrieval performance and reconstruction performance across varying noise levels X.


Reviews:
[{"review_id": "DlgcVo9fUu", "reviewer": "Reviewer_a8TL", "summary": "", "questions": "- Could you run each experiment multiple times, report the mean and standard deviation, and conduct statistical significance tests to compare Vec2Text with the strongest baseline in each column?\n\n- Could you report the performance of Base and other Vec2Text variants on BEIR?\n\n- Instead of adding noises to the embeddings to defend against inversion attacks, I am thinking about a simpler solution: just using some less accurate embedding models (e.g., those ranked 10th or 15th on MTEB). Their retrieval performance should be on par with SOTA embedding models with a certain level of noise, but is it possible that they enjoy an even lower reconstruction accuracy? I understand this is a complex question and needs a thorough empirical study, so I do not expect a perfect answer from the authors, but I suggest they add this to their discussion.", "limitations": "", "rating": 5, "confidence": 3, "soundness": 5, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your comments.\n\n> It is unclear how the reconstruction performance will be affected by randomness. Could you run each experiment multiple times, report the mean and standard deviation...\n\nThe numbers reported in the paper are always taken from the mean of 1000 test datapoints. We plan to add standard errors in the final version. Here are results on Natural Questions with standard errors:\n\n| method | bleu | token f1 | exact | cos\n| ----- | ----- | ----- | ----- | ----- | \n| Base (0 steps) | 29.1 ± 1.2 | 65.1 ± 0.8 | 1.5 ± 0.6 | 90.0 ± 0.3\n| Vec2Text | | | | \n| --> [1 step] | 47.7 ± 2.9 | 78.0 ± 0.6 | 9.7 ± 1.3 | 95.1 ± 0.2\n| --> [20 steps] | 81.3 ± 1.1 | 94.4 ± 0.4 | 51.8 ± 2.2 | 98.8 ± 0.1\n| --> [50 steps] | 82.9 ± 1.0 | 95.2 ± 0.3 | 52.4 ± 2.2 | 99.0 ± 0.1\n| --> [50 steps + sbeam] | **97.0** ± 1.3 | **99.1** ± 0.5 | **94.0** ± 2.3 | **99.9** ± 0.0\n\n> ...and conduct statistical significance tests?\n\nTo confirm the difference in BLEU scores between Vec2Text and the baseline, we perform a paired t-test on with a confidence level of 0.95 and a null hypothesis that the BLEU scores of our method is equivalent to the baseline. We collect 384 BLEU scores from running the baseline and Vec2Text on reconstructing 32-token GTR-embedded inputs from NQ. We observe a t-statistic of -27.9 and a p-value of 9.82e-103, much smaller than the significance level of 0.05. Thus, we reject the null hypothesis and conclude that the performance of our method exceeds the baseline with a confidence level of 95%.\n\n> It is unclear whether Vec2Text[50 steps+sbeam] also outperforms baselines and other Vec2Text variants on cross-domain datasets, such as BEIR.\n\nHere we report the results of Vec2Text (using the new and improved model) compared to the baseline across the datasets of BEIR, measuring performance by BLEU score. We see that Vec2Text provides on average 3.4x performance improvement across these out-of-domain test datasets:\n\n| dataset          |   Baseline |   Vec2Text |\n|-----------------|-----------|-----------|\n| arguana          |        6.8 ± 1.1 |    **20.6 ± 0.9** |\n| bioasq           |      6.7 ± 0.9   |       **22.8 ± 3.0** |\n| climate-fever    |       12.8 ± 1.7 |       **44.9 ± 3.3** |\n| cqadupstack      |        7.1 ± 1.6 |      **24.2 ± 0.2** |\n| dbpedia-entity   |       15.4 ± 2.2 |       **45.3 ± 2.6** |\n| fever            |       12.6 ± 1.1 |       **45.1 ± 0.5** |\n| fiqa             |        6.6 ± 0.6 |      **21.6 ± 1.8** |\n| hotpotqa         |       15.4 ± 2.6 |       **38.9 ± 1.1** |\n| msmarco          | 15.5 ± 0.8 | **59.6 ± 4.0** |\n| nfcorpus         |        6.2 ± 0.33 | **26.1 ± 1.9** |\n| nq               |       11.0 ± 0.9 | **32.7 ± 1.0** |\n| quora            |       36.2 ± 1.8 | **95.6 ± 2.9** |\n| robust04         |     4.6 ± 0.2  |  **15.5 ± 0.6** |\n| scidocs          |      5.5 ± 0.3   | **17.2 ± 1.1** |\n| scifact          |        5.0 ± 0.4 |  **18.1 ± 1.4** |\n| signal1m         |       13.2 ± 1.5 | **72.3 ± 2.2** |\n| trec-covid       |        5.6 ± 0.6 |  **19.2 ± 0.4** |\n| trec-news        |        4.9 ± 0.2 |  **14.5 ± 0.5** |\n| webis-touche2020 |        6.6 ± 0.5 | **19.6 ± 0.5** |\n\n\n> Instead of adding noises to the embeddings to defend against inversion attacks, I am thinking about a simpler solution: just using some less accurate embedding models. \n\nThis is a good idea for a possible defense: swapping the embedder (regardless of accuracy) should severely inhibit the performance of our system. (Note that this is out of our threat model, as we assume that our system trains on embeddings from a particular embedder, we do not expect to generalize to other embedders.) \n\nWe the effect of changing the embedder on performance by replacing embeddings from GTR with those from ClinicalBERT using our model trained on GTR embeddings, and observe the following inversion performance:\n\n\nmethod | bleu | token f1 | exact | cos\n| ---- | ---- | ---- | ---- | ---- |\nBase (0 steps) | 1.0 ± 0.5 | 6.4 ± 0.3 | 0.0 ± 0.0 | 48.6 ± 0.8\nVec2Text | | | |\n--> [1 step] | 1.8 ± 0.0 | 12.3 ± 0.0 | 0.0 ± 0.0 | 52.0 ± 0.4\n--> [20 steps] | 1.7 ± 0.1 | 11.6 ± 0.0 | 0.0 ± 0.0 | 65.5 ± 0.7\n--> [50 steps] | 1.7 ± 0.1 | 11.6 ± 0.5 | 0.0 ± 0.0 | 65.5 ± 0.7\n--> [50 steps + sbeam] | 1.6 ± 0.0 | 10.6 ± 0.7 | 0.0 ± 0.0 | 65.6 ± 0.6 \n[Gaussian noise + GTR + Vec2Text] | 1.9 ± 0.0 | 10.6 ± 0.3 | 0.0 ± 0.0 | 1.8 ± 0.0\n\nFor reference, we include an additional row showing the performance of our GTR model given the proper embedder under multivariate Gaussian noise with a factor of 1.0. Even these essentially random generations achieve a token F1 score of 10.6 by simply outputting common words.\n\nWhen we substitute the embedder with GTR with ClinicalBERT, performance severely drops: exact match performance drops from 94% to 0. Vec2Text barely outperforms random language model generations in token-match metrics, although iteration does slightly help in locating sequences with higher similarity to the ground-truth.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "+ The observations in this study have important implications for several popular issues, such as privacy in large language models and vector databases.\n\n+ The empirical study is comprehensive. Various retrieval-related benchmark datasets are used, covering both in-domain and cross-domain reconstruction settings. Two state-of-the-art text embedding models (based on T5 and GPT, respectively) are considered. A case study is also presented on MIMIC-III in the medical domain. The Analysis section is inspiring as well.\n\n+ Based on the empirical observations, the authors also propose a simple yet effective way to defend against inversion attacks to text embeddings.", "reasons_to_reject": "- It is unclear how the reconstruction performance will be affected by randomness. The authors should run each experiment multiple times with the mean and standard deviation reported. Also, significance tests are missing. It is unclear whether the improvement of Vec2Text[50 steps+sbeam] over baselines and other Vec2Text variants is statistically significant or not. In fact, some gaps of tf1 and cos in Table 1 are quite subtle, therefore p-values should be reported.\n\n- It is unclear whether Vec2Text[50 steps+sbeam] also outperforms baselines and other Vec2Text variants on cross-domain datasets, such as BEIR.", "questions_to_authors": "- Could you run each experiment multiple times, report the mean and standard deviation, and conduct statistical significance tests to compare Vec2Text with the strongest baseline in each column?\n\n- Could you report the performance of Base and other Vec2Text variants on BEIR?\n\n- Instead of adding noises to the embeddings to defend against inversion attacks, I am thinking about a simpler solution: just using some less accurate embedding models (e.g., those ranked 10th or 15th on MTEB). Their retrieval performance should be on par with SOTA embedding models with a certain level of noise, but is it possible that they enjoy an even lower reconstruction accuracy? I understand this is a complex question and needs a thorough empirical study, so I do not expect a perfect answer from the authors, but I suggest they add this to their discussion.", "paper_topic_and_main_contributions": "This paper explores the possibility of reconstructing full-text content from dense text embeddings. The authors cast this problem as a controlled generation task, aiming to generate text that, when re-embedded, closely matches a fixed point in the latent space. While a naive model conditioned on the embedding performs poorly, the authors propose a multi-step method, Vec2Text, which iteratively corrects and re-embeds text, and achieves an astonishing accuracy in recovering text inputs exactly. Experiments on various datasets across different domains (i.e., NQ, MSMARCO, MIMIC-III, and 15 datasets from BEIR) support the major claim of this paper: text embeddings reveal almost as much as text. Moreover, to defend against such inversion attacks, the authors also study the possibility of adding moderate noises to the embeddings, which, according to their experiments, significantly decreases the reconstruction accuracy without hurting the retrieval performance too much.", "excitement": 5, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "mN3Tu2IMLc", "reviewer": "Reviewer_3tQ6", "summary": "", "questions": "Major:\n- Consider embeddings from bert-like models (mainly used in the clinical space for privacy concerns) as input for the MIMIC dataset.\n- Report a better investigation of the types of best-reconstructed sentences and the information they carry for each dataset used.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your comments.\n\n> Privacy concerns in the healthcare space are currently addressed by using encoder-only models  ... Consider embeddings from bert-like models.\n\nWe trained an additional base and corrector model given the ClinicalBERT encoder (Huang et al (2019)). Here we report its performance on recovering information from MIMIC notes:\n\nmethod | first | last | full | bleu | tf1 | exact | cos\n|---|---|---|---|---|---|---|---|\nBase | 40.0 | 28.1 | 12.5 | 3.9 | 31.7 | 0.0 | 77.1\nVec2Text | 91.7 | 84.2 | 75.4 | 43.8 | 75.1 | 0.2 | 96.8\n\nThese experiments show that embeddings from BERT-based models can be successfully inverted. We also plan to add full results on NQ for this ClinicalBERT model to the appendix.\n\n> they use 32-token text that is not a comparable length when dealing with clinical text where each note can be thousand of tokens long\n\nOur work is the first to show that text sequences of *any* length can be exactly recovered given just their embeddings, indicating that embedding inversion is a threat to privacy. We run other experiments on text up to 128 tokens. We leave it to future work to investigate how inversion performs with much longer embedded sequences.\n\nEven small, highly incomplete fractions of clinical notes reveal very sensitive information, such as general medical area (eg, oncology), details of diagnosis (eg, type of tumor), some aspects of the treatment regimen, etc.  \n\n> Privacy concerns in the healthcare space are currently addressed by using encoder-only models and deidentified clinical notes\n\nWe note that GTR, the embedder we use for MIMIC, *is* an encoder-only model as it uses the encoder-part of T5. We also note the above results from ClinicalBERT, another encoder is analyzed in this rebuttal.\n\nWe also note that while deidentification is one defence, there are multiple examples of re-identification attacks against medical datasets that were de-identified to the HIPAA standards, for example:\n- Sweeney, Latanya. “Matching Known Patients to Health Records in Washington State Data”. https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2289850\n- Yoo, Ji Su, Alexandra Thaler, Latanya Sweeney, and Jinyan Zang. \"Risks to Patient Privacy: A Re-identification of Patients in Maine and Vermont Statewide Hospital Data.\" Technology Science (October 2018)\n\nDeidentification provides very brittle protection at best.  If embeddings can be reversed to deidentified notes, these notes become very vulnerable to reidentification.\n\n> Report a better investigation of the types of best-reconstructed sentences and the information they carry for each dataset used.\n\nWe performed manual error analysis on 100 examples from the Natural Questions dataset, of which Vec2Text (with 50 iterations) only fails on 6 of them. It appears that reordering two similar entities was the most common failure mode for our model. Text with few or easily distinguished entities appears easier to reconstruct.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The work is methodologically sound and the experiments are thorough. It includes: embedding inversion experiments, ablation studies, Gaussian noise addition as a way to fix the problem, and the impact of word frequency investigation. Given the recent availability of generative language models that can be directly queried in inference, the topic of privacy concerns related the generation of sensitive text is central and need further investigation. This applies especially in cases where underrepresented populations are at risk of harm and in the clinical space.", "reasons_to_reject": "Despite its relevance, the work mainly highlights the performance of the approach and to a less extent the actual harm that can come from different tasks. The authors leveraged several benchmark datasets but do not report a thorough characterization of the extent of harm that the embedding inversion can have for each dataset. In particular, authors use the MIMIC dataset to prove that names and sensitive information can be retrieved. Nevertheless, they use 32-token text that is not a comparable length when dealing with clinical text where each note can be thousand of tokens long. Privacy concerns in the healthcare space are currently addressed by using encoder-only models and deidentified clinical notes, hence the experiment performed by the authors on MIMIC does not seem particularly relevant.", "questions_to_authors": "Major:\n- Consider embeddings from bert-like models (mainly used in the clinical space for privacy concerns) as input for the MIMIC dataset.\n- Report a better investigation of the types of best-reconstructed sentences and the information they carry for each dataset used.", "paper_topic_and_main_contributions": "The manuscript \"Text Embeddings Reveal (Almost) As Much As Text\" presents a framework that is able to reconstruct a 32-token input text from dense embeddings. \n\nMain contributions:\n- Addressing privacy concerns for vector databases;\n- Thorough investigation of the embedding inversion task leveraging embedding geometry;\n- Controlled Generation approach able to recover up to 92% of input text.", "excitement": 4, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Typos_Grammar_Style_and_Presentation_Improvements": "Some typos in the text and Related Works section should go after Introduction for clarity and readability."}, "extra_text": {}}, {"review_id": "kJbElR9nlJ", "reviewer": "Reviewer_Qu6b", "summary": "", "questions": "- Why do you think the method--adding Gaussian noise are effective?\n- During the iterative process, I think the hyperparameters should be adjusted.", "limitations": "", "rating": 3, "confidence": 3, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your comments.\n\n> Although the paper report interesting findings, I cannot clarify the main contribution.\n\nOur main contribution is Vec2Text, a method for iteratively recovering sequences from their embeddings. We show that with a known embedder, this can cause privacy leakage.\n\n> For small length of tokens, the performance is much better compared to the baseline, but I cannot find the explanation.\n\nWe looked into this question further and we believe it was due to an optimization. We recently shifted to initializing model parameters from scratch instead of initializing from the zero-step inversion model. After this change we are able to train our model for many more tokens without overfitting. \n\nThis resulted in Vec2Text achieving the following performance reconstructing text from OpenAI embeddings of MSMARCO of up to 128 tokens:\n\n|              method               |  token f1  |  bleu  |  exact | cos | \n| ---- | ---- | ---- | ---- | ---- |\n| Base                   |  54 |   17.0  |    0.6  |  0.95 |\n| Vec2Text               |        |       |         |       |\n| --> [1 step]            |  68 | 29.9   |    1.4  |  0.97 |\n| --> [20 steps]          |  78 | 43.1  |    3.2  |  0.99 |\n| --> [50 steps]          |  78 | 44.4   |    3.4  |  0.99 |\n| --> [50 steps + sbeam]   |   84 |  55.0  |    8.0  |  0.99 |\n\nWith longer training, new long-sequence Vec2Text model significantly outperforms the one from the original draft of our paper (55.0 vs. the originally reported BLEU of 33.9), showing a similar relative improvement.\n\nDue to the fixed-size embedding, we do expect lower absolute recovery performance for longer sequences. In the future, we could achieve better results using Vec2Text on the same embeddings by scaling model parameters or using more compute at search time.\n\n> Discussion on Table 4 and Table 5 is missing.\n\nThanks for noticing the lack of references here – we do discuss these results although we do not directly reference the tables from the text. We will fix in a future version.\n\n> there is no clear reason for not comparing with other embedding inversion methods and the methods mentioned in related work section\n\nWe were not able to find many alternative baseline approaches. We do include a bag-of-words baseline in Table 1, which was proposed in Song (2022). \n\nThe only other baseline from related work (Adolphs et al (2022) and Li et al (2023)) trains a decoder model with the projected sentence embedding as the first token; we expect this to perform weaker than our proposed architecture, an encoder-decoder with sequence projection. To compare, we trained this model on NQ using a GPT-2 backbone and will add this result to a future version. Results show this architecture performs far worse than our baseline:\n\n|              method               |  token f1  |  bleu  |  exact | cos\n| ---- | ---- | ---- | ---- | ---- |\n| New baseline (GPT-2)              |        47  |   1.0  |      0 | 0.76\n| Existing baseline (T5 + MLP2Seq)  |        67  |  31.9  |      0 | 0.91\n| Vec2Text                          |        99  |  97.3  |     92 | 0.99\n\n> Why do you think the method--adding Gaussian noise are effective?\n\nOur hypothesis is that adding Gaussian noise seems to move embedding vectors out of training distribution. As the noise factor λ increases, noisy embeddings look less like the training data, causing the inversion methods to fail.\n\n> During the iterative process, I think the hyperparameters should be adjusted.\n\nVec2Text has two hyperparameters: number of iterations and sequence-level beam width. We provide results at 0, 1, 20, and 50 iterations, and with sequence-level beam widths of 1 and 8.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- As the authors mentioned, since a lot of systems use vector embeddings and sometimes we can access the embedding itself, the problem that the author tackled is interesting.\n- Proposed Vec2Text is an iterative simple method.\n- After showing the issue, the author suggested a simple method to prevent it.", "reasons_to_reject": "- Although the paper report interesting findings, I cannot clarify the main contribution.\n- As far as I understand, the contribution seems to be more into their proposed method to restore the text.\n- If so, more analysis and ablations are required for the evaluation\n  - For small length of tokens, the performance is much better compared to the baseline, but I cannot find the explanation.\n  - Discussion on Table 4 and Table 5 is missing\n- Furthermore, there is no clear reason for not comparing with other embedding inversion methods and the methods mentioned in related work section; because of this, I cannot convince how much Vec2Text is effective.\n  - Without reasonable baseline, people could consider the performance as a result of the further training process.\n- On the other hand, the Gaussian noise method weakens the contribution of vec2text because the method is not robust on the noise, which means not generalized well.\n\nIn conclusion, I personally feel the findings interesting but the presentation of current version is weak.", "questions_to_authors": "- Why do you think the method--adding Gaussian noise are effective?\n- During the iterative process, I think the hyperparameters should be adjusted.", "paper_topic_and_main_contributions": "This paper shows that embeddings itself are able to be restored as its origin text.\nAs a frame of embedding inversion, the author presents their method Vec2Text, which uses encoder-decoder architecture as well as a new input format. The input format consists of projected original embedding, hypothesis embedding, the difference of them, word embeddings.\nThe performances on several datasets reach around 90%, as the authors said, the observation could make privacy issue.", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Typos_Grammar_Style_and_Presentation_Improvements": "The size of Table 4 should be adjusted."}, "extra_text": {}}, {"review_id": "xBixiC0hLH", "reviewer": "Reviewer_WRMs", "summary": "", "questions": "* It is mentioned in Line 290 that \"More rounds is monotonically helpful\"; is this guaranteed (theoretically)? If so, can we perfectly recover every sequence eventually given infinite rounds of refinement?\n* I'd be curious to know what kinds of raw texts are more challenging than others to recover (i.e., failure cases or cases that require many rounds of refinement).", "limitations": "", "rating": 5, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your comments. We sincerely appreciate your critique and address some of your concerns below.\n\n> It is mentioned in Line 290 that \"More rounds is monotonically helpful\"; is this guaranteed (theoretically)? \n\nWe can only guarantee that the corrected text won’t get further from the initial text by only “accepting” a change if it improves in cosine similarity to the ground-truth embedding. However, it is possible (and common) to “get stuck” and fail to find a correction that improves the current text. \n\nWe feel this is an interesting question and there is room here to develop random methods (MCMC-style) that avoid this phenomenon of getting stuck and perhaps improve recovery of the ground-truth text after many rounds of refinement.\n\n> I'd be curious to know what kinds of raw texts are more challenging than others to recover (i.e., failure cases or cases that require many rounds of refinement).\n\nTo answer this question, we analyze 100 reconstructions from Natural Questions after 1 and 50 rounds of refinement. At a single round, we see reconstructions are generally on topic but do not match the syntax of the true sequence and miss or hallucinate many entities. Entities gradually “snap into place” after more rounds of refinement, and we see 94/100 sequences perfectly recovered.\n\nOut of all 6 sequences that are not recovered after 50 rounds of refinement, we see that the predicted sequence is very close in every case, and contains all of the correct entities, but has swapped two entities or clauses. For example, given the ground truth:\n\n```decision to have the series focus on Patrick and Robin's romance, Ed Martin of the \"Jack Myers Report\" stated, \"...it’```\n\nVec2Text finds the reconstruction:\n\n```decision to have the series focus on Patrick and Robin's romance, Ed Myers of the \"Jack Martin Report\" stated, \"...it’```\n\nNotice that the entities are correct, but the last names “Myers” and “Martin” are swapped. We hypothesize this common failure mode may be due to the bag-of-words information conveyed in common sentence embeddings.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "* Novelty: The paper appears to be very novel to me, from the task it considers (i.e., extracting information as raw texts from text embeddings), to the proposed method (i.e., multi-step refinement in controlled text generation), to the insights that follow (i.e., text embeddings may leak private information). I believe the paper is among the most novel papers I've seen at NLP venues.\n* Clarity: The paper is clear in motivation, method description, and experiment settings.\n* Interesting and effective approach: The proposed method Vec2Text is reasonable, interesting, and effective in extracting the original texts from the embeddings. It significantly outperforms naive and standard decoding approaches (e.g., beam search and nucleus sampling). The authors also propose a simple method for defending against inversion attacks by injecting some random noise into the embeddings.\n* Comprehensive evaluations: The evaluation is very comprehensive, covering many different tasks and domains. The case studies are good analyses for understanding the model's property.\n* Important implications: The insights brought by the paper could have important implications for how language models should be configured so as to preserve the private/sensitive information in the raw text data.\n", "reasons_to_reject": "* The method efficiency (as inversion attack) could be a bit concerning, considering that it adopts a multi-step refinement approach. However, based on my understanding, the major contribution of the paper is not proposing an attack approach, but rather to show that there exist such approaches that may cause privacy concerns regarding leaking out text embeddings. Therefore, I won't consider this as a weakness of the paper.\n\nOverall, I don't have any major concerns about the paper that prevent it from being accepted. ", "questions_to_authors": "* It is mentioned in Line 290 that \"More rounds is monotonically helpful\"; is this guaranteed (theoretically)? If so, can we perfectly recover every sequence eventually given infinite rounds of refinement?\n* I'd be curious to know what kinds of raw texts are more challenging than others to recover (i.e., failure cases or cases that require many rounds of refinement).", "paper_topic_and_main_contributions": "This paper aims to investigate the information stored in text embeddings generated by text encoders and specifically focuses on recovering the original full texts from the text embeddings. The authors formulate the task as controlled text generation and adopt a multi-step approach that gradually refines the hypothesis text closer to the given target embedding.\n\nThe proposed method demonstrates that most short texts (e.g., 32-token) can be reconstructed almost exactly. Such findings show that the private information in the training data could be leaked from the output text embeddings of language models.", "excitement": 5, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This article aims to study the information stored in text embeddings generated by text encoders, with a particular focus on restoring the original full text from text embeddings. The author describes this task as controlled text generation and adopts a multi-step method to gradually refine the hypothetical text to make it closer to the given target embedding. The proposed method indicates that most short texts (such as 32 tokens) can be almost accurately reconstructed. These findings indicate that private information in the training data may have been leaked from the output text embedding of the language model.In summary, these paper solve privacy issues in vector databases， conduct in-depth research on embedding inversion tasks using embedded geometry and he controlled generation method can recover up to 92% of input text. Meanwhile, this paper's experiments are very comprehensive such as various retrieval-related benchmark datasets are used and covering domain including in-domain and closed-domain reconstruction settings.
Decision: Accept-Main