What is in a name? Mitigating Name Bias in Text Embedding Similarity via Anonymization (2025)

Abstract: Text-embedding models often exhibit biases arising from the data on which they are trained. In this paper, we examine a hitherto unexplored bias in text-embeddings: bias arising from the presence of \textit{names} such as persons, locations, organizations etc. in the text. Our study shows how the presence of \textit{name-bias} in text-embedding models can potentially lead to erroneous conclusions in the assessment of thematic similarity. Text-embeddings can mistakenly indicate similarity between texts based on names in the text, even when their actual semantic content has no similarity or indicate dissimilarity simply because of the names in the text even when the texts match semantically . We first demonstrate the presence of name bias in different text-embedding models and then propose \textit{text-anonymization} during inference which involves removing references to names, while preserving the core theme of the text. The efficacy of the anonymization approach is demonstrated on three downstream NLP tasks involving embedding similarities, achieving significant performance gains. Our simple and training-optimization-free approach offers a practical and easily implementable solution to mitigate name bias.

## introduction text-embedding models, which give a concise numerical representation to sentences/paragraphs have become fundamental tools for downstream nlp tasks in fields such as healthcare, education, law and scientific research @xcite @xcite @xcite . a cosine similarity between embeddings is generally used @xcite in a variety of tasks. with a similarity measure, the goal is to find which two texts are similar to or different from each other. text-embedding models are often trained on large amounts of text on the internet. this data can inadvertently contain biases of various kinds, reflecting social prejudices and stereotypes. as a result, these models can generate biased embeddings, reinforcing harmful stereotypes or discriminating against certain cultural groups, genders, etc. @xcite @xcite . furthermore, the presence of bias in models could lead to embeddings that disproportionately emphasize particular parts of the text, consequently failing to capture the true semantics and themes within the text @xcite . while important, existing studies on biases, predominantly examine biases in text-embedding models mostly related to gender, geography, race, religion etc. @xcite @xcite @xcite . in this paper, we demonstrate that text-embedding models exhibit significant bias towards names within the text. to illustrate this, we begin with a motivating example in our observation reveals a critical issue that can significantly impact applications that rely on semantic similarity, including semantic search, information retrieval, and plagiarism detection @xcite : consider the challenge of accurately assessing the similarity between two stories/plots with identical underlying meanings but distinct character names. current methods may erroneously classify these stories as dissimilar, leading to inconsistent and unreliable results. further, based upon our investigation, we would like to mention upfront that the issue is not confined to certain cultures, cross-culture, but is universal in the sense that the name bias issue occurs in a very broad sense. ## related work biases in text-embedding models: textembedding models while powerful, can inadvertently reflect and amplify existing biases and prejudices; there is vast research understanding and mitigating bias in such models. for example, there is work focusing on models that investigate under-representation or misrepresentation of specific groups, such as lgbtq+ individuals, leading to skewed or inaccurate outcomes @xcite @xcite . another type of study focuses on gender bias in word embeddings models @xcite . the study highlights a concerning issue i.e., many embedding models associate specific occupations with particular genders. @xcite studied biases at a sentence-level in sentence transformers influenced by different parts of speech such as common nouns, adverbs etc. while we discuss text-embedding models, it is important to highlight works that investigate bias within large language models (llms) for text-generation which are a part of this ecosystem @xcite . @xcite observed "geographical erasure" where certain regions are underrepresented in llm outputs. @xcite showed that llms often favor developed regions and exhibit negative biases towards locations with lower socioeconomic conditions, particularly on subjective topics such as attractiveness and intelligence. further, some works have also investigated cross-cultural biases in llms for text generation @xcite @xcite . compared to the above work, we investigate name-bias in text-embeddings, an area not previously explored in existing research to the best of our knowledge. debiasing methods: various approaches have been proposed to tackle different kinds of biases in text-embedding models highlighted above. one common technique to remove such biases is to update the training dataset and make it unbiased and re-train the model @xcite . another paradigm involves applying approaches such as disentanglement or alignment where models are fine-tuned to remove biases associated with concepts such as gender, religion etc. @xcite @xcite ). an alternative approach involves post-processing of the embeddings. specifically, it involves adding a debiasing module after encoders to filter out certain biases in the representations cheng et al. @xcite . for more details on this topic, we refer the reader to survey by @xcite for more details. we emphasize some key considerations based upon the discussions above. firstly, all the aforementioned techniques require an optimization phase, involving either retraining the initial model, fine-tuning with a modified loss or postprocessing of the generated embeddings. secondly, these methods are often designed to address specific bias types, such as social, gender, or religious biases. notably, the identification and mitigation of name bias has not been previously explored to our knowledge. ## understanding name bias in this section, we investigate the presence of bias within text-embedding models related to names. our primary objective is to investigate the influence of names containing identity-specific information on the resulting text embeddings, while ensuring the semantic structure of the text remains unchanged. ## methodology: overcoming bias through anonymization previously, we showed that how just changing person names/country names can impact the embeddings significantly. in this section, we introduce a simple inference-time anonymization technique to mitigate the bias caused by names. the core idea is to mitigate the influence of names on embeddings, and making the resulting debiased anonymized embeddings to be more generalizable and less prone to biases related to particular individuals or entities. the anonymization of a text t during inference is achieved through the following process. we first identify in t , occurrences of desired entities such as person names, locations and organizations relevant to the use case. we anonymize the text by removing those occurrences from t . the anonymized text referred to as t anon retains the overall structure and meaning of the original text t while removing any specific references to person names etc. this anonymization can be achieved via tools such as large language models(llms) @xcite or named entity recognition tools @xcite . in our work, we used gemini and anthropic.claude-3-5sonnet text-generative models for anonymization using prompts. depending upon the use-case, different names in text such as person names, cities, countries, organizations can be removed. we would like to clarify that the same process of anonymization can also be done through named-entity recog-nition(ner) tools @xcite , however in our initial experiments we found llms to be more accurate. sample prompts for anonymization are presented in purpose prompt remove person names and location names given below text, please completely delete all person/character names which are proper nouns and city/ country/ village/ town/ continent/ river/ organization names which are proper nouns etc. wherever they occur replace with empty string. completely remove them and not anything else. do not delete monument/landmark names like eiffel tower etc. do not remove he/she/him/her etc.. output contains the modified text only.... the text is provided below :::: remove person names only given below text, please completely delete all person/character names which are proper nouns. wherever they occur replace with empty string. completely remove them and not anything else. do not remove he/she/him/her etc.. output contains the modified text only.... the text is provided below :::: in this section, we investigate the performance of the anonymized text embeddings on three downstream tasks. the tasks are based on obtaining a similarity score between pieces of texts. these tasks are primarily based upon semantic similarity which find applications in areas such as information retrieval, clustering, plagiarism detection, question answering etc. @xcite . the tasks that we evaluate on differ in various aspects such as the nature of the task, evaluation methodology, the judgment score available, etc. on these tasks, our exper-iments show that embeddings based on anonymized text can significantly help in downstream tasks. ## conclusion in this work, we highlight the bias in text embeddings stemming from the presence of names in the text. we showed concrete examples, over multiple text-embedding models, that similarities between embeddings can be dominated by names in the text rather than the semantic meanings of the text. we then proposed a method to mitigate bias by performing anonymization at inference time. this involved the removal of names from the text and using the anonymized text to create the embeddings. our findings demonstrate that anonymized text embeddings significantly outperform deanonymized text embeddings on tasks involving semantic similarity. while we proposed one way to mitigate the issue through anonymization, a deeper question that remains is: how to train text-embedding models such that the embeddings capture the semantics more than the names in the text? below we discuss the limitations of the proposed work.