Toward Privacy-preserving Text Embedding Similarity with Homomorphic Encryption (2022)

Abstract: Text embedding is an essential component to build efficient natural language applications based on text similarities such as search engines and chatbots. Certain industries like finance and healthcare demand strict privacy-preserving conditions that user’s data should not be exposed to any potential malicious users even including service providers. From a privacy standpoint, text embeddings seem impossible to be interpreted but there is still a privacy risk that they can be recovered to original texts through inversion attacks. To satisfy such privacy requirements, in this paper, we study a Homomorphic Encryption (HE) based text similarity inference. To validate our method, we perform extensive experiments on two vital text similarity tasks. Through text embedding inversion tests, we prove that the benchmark datasets are vulnerable to inversion attacks and another privacy preserving approach, dχ-privacy, a relaxed version of Local Differential Privacy method fails to prevent them. We show that our approach preserves the performance of models compared to that the baseline has degradation up to 10% of scores for the minimum security.

## introduction recently, various industries provide enhanced user experiences through natural language processing (nlp) applications. ai assistants such as amazon's alexa and google assistant are representative examples that help users to achieve their purposes with a wide range of intentions. to build such complex applications, it is common to utilize machine-learned text representations, i.e., text embeddings to infer similarities between texts @xcite ## method in this section, we propose our homomorphic encryption (he) based method to protect text data privacy. to achieve this, our goal is to approximate text similarity functions for given text embeddings in an encrypted state using the ckks scheme. formally, similar to the definition in (1), we implement encrypted similarity function f unct * that computes the encrypted similarity result sim * from encrypted text embeddings in order to achieve sim * ≈ sim (1) as much as possible. ## conclusions in this paper, we proposed homomorphic encryption based text similarity inference with text embeddings. with our method, users can utilize text embedding based services without revealing the original text, which can be recovered through inversion attacks as we demonstrated in the experiment 4.3. extensive experiments 4.4 on two text similarity tasks proved that our approach does not harm the performance of models. in contrast, the d χprivacy baselines fail to achieve protection from inversion attacks without performance degradation. we hope that this work lays the groundwork for the secure usage of text embeddings in privacy-sensitive industries like finance and more future works on the practical usage of our he approach by resolving the current limitations.