Unsupervised Mitigating Gender Bias by Character Components: A Case Study of C hinese Word Embedding (2022)

Abstract: Word embeddings learned from massive text collections have demonstrated significant levels of discriminative biases. However, debias on the Chinese language, one of the most spoken languages, has been less explored. Meanwhile, existing literature relies on manually created supplementary data, which is time- and energy-consuming. In this work, we propose the first Chinese Gender-neutral word Embedding model (CGE) based on Word2vec, which learns gender-neutral word embeddings without any labeled data. Concretely, CGE utilizes and emphasizes the rich feminine and masculine information contained in radicals, i.e., a kind of component in Chinese characters, during the training procedure. This consequently alleviates discriminative gender biases. Experimental results on public benchmark datasets show that our unsupervised method outperforms the state-of-the-art supervised debiased word embedding models without sacrificing the functionality of the embedding model.

## introduction investigations into the representation learning revealed that word embeddings are often prone to exhibit discriminative gender stereotype biases @xcite . consequently, these biased word embeddings have effects on downstream applications @xcite . mitigating gender stereotypes in word embedding are becoming a research hotspot due to its penitential application, and a number of the existing debias works are dedicated to the english language @xcite . however, debiasing on chinese, one of the most spoken languages, has drawn less attention these days. in the chinese language, "radical" is a graphical component of chinese characters, which serves * equal contribution † corresponding authors as an indexing component in the chinese dictionary. radical can suggest part of the meaning of the character due to the phono-semantic attribute of the chinese language. for example, "氵(water)" is the radical of "河 (river), 湖 (lake)". consequently, a series of works have shown that radicals can enhance the word embedding quality @xcite @xcite . as part of the radical system, the gender-related radicals, i.e., "女(female)" and "亻(man)", contains gender information of the corresponding character. specifically, the radical "女(female)" can denote female and "亻(man)" can denote people, which includes male gender information. for example, characters "姐(sister),妇(wife),妈(mother),姥(grandma)" all have the radical of "女(female)", demonstrating that these are feminine words. hence, we assume that radical is a natural information source to capture feminine and masculine information, and such information can help the model learn gender definition. once the model learns what is the definition of gender, it can identify the gender bias that is not actually relevant to gender. to this end, we propose our chinese genderneutral word embedding model (cge) that is based on the classic word2vec model, where the basic idea is to predict the target word given its context words. cge has two variations, i.e., radical-added cge and radical-enhanced cge. radical-added cge emphasizes the gender definition information by directly adding the radical embedding to the word embedding. we next propose a radicalenhanced cge, where radical embeddings are employed to predict the target word instead of adding to the word embedding. this is a more flexible approach, where the gradients of the embeddings of words and radicals can be different in the training process. note that the radical can be extracted from the character itself, hence, our model can also learn gender-neutral word embedding in an unsupervised fashion. experimental results show that our methods outperform the supervised models. compared with previous works, our work is focused on the chinese language, and utilizes radicals, a special component of chinese character. ## methodology we will take cbow for example and demonstrate our frameworks based on cbow. ## conclusion in this paper, we proposed two methods for unsupervised training in chinese gender-neutral word embedding by emphasizing gender information stored in chinese radicals in explicit and implicit ways. our first model directly incorporates radical embedding in its word embedding, and the second one implicitly utilizes radical information. experimental results show that our unsupervised method outperforms the supervised debiased word embedding models without sacrificing the functionality of the embedding model.