GBEM - UA : Gender Bias Evaluation and Mitigation for U krainian Large Language Models (2025)

Abstract: Large Language Models (LLMs) have demonstrated remarkable performance across various domains, but they often inherit biases present in the data they are trained on, leading to unfair or unreliable outcomes—particularly in sensitive areas such as hiring, medical decision-making, and education. This paper evaluates gender bias in LLMs within the Ukrainian language context, where the gendered nature of the language and the use of feminitives introduce additional complexity to bias analysis. We propose a benchmark for measuring bias in Ukrainian and assess several debiasing methods, including prompt debiasing, embedding debiasing, and fine-tuning, to evaluate their effectiveness. Our results suggest that embedding debiasing alone is insufficient for a morphologically rich language like Ukrainian, whereas fine-tuning proves more effective in mitigating bias for domain-specific tasks.

## introduction in recent years, llms have become essential across various domains, including healthcare (nazi and peng, 2023), education @xcite , and recruitment @xcite . however, these models are trained on vast amounts of data, which may contain biases that become embedded in their outputs. such bias prevents models from accurately representing true population characteristics, leading to unfair or unreliable outcomes. this can lead to unfair treatment of certain groups, particularly in sensitive applications such as hiring, medical decision-making, and education. in the context of this work, we define bias as the production of opposite outputs when only the target words (e.g., "male" and "female") are changed. one of the most concerning biases arises in hiring scenarios. for example, in @xcite , hiring bias was demonstrated using prompts related to candidate selection. their results showed that 10 different llms exhibited gender bias in hiring decisions, producing unequal outputs for male and female candidates with identical experience and resumes. while various forms of bias exist, including gender, age, cultural, and regional biases @xcite , our work focuses specifically on gender bias in hiring decisions. it is important to emphasize that the use of ai in hiring is widely recognized as high-risk due to potential ethical and fairness concerns. in recent years, many works have focused on bias mitigation. most of these approaches aim to reduce bias while maintaining the model's overall accuracy. while various debiasing techniques have been developed to mitigate bias in englishlanguage models, their effectiveness in other languages remains largely untested. this gap is especially relevant for ukrainian, a language with complex grammatical gender structures that influence how professions and roles are described. for example, in the ukrainian language, feminized forms (feminitives) arise in the context of professions. specifically, each profession has a corresponding feminitive form -a word used to describe a female professional. for instance, "чиновник" foot_0 and "чиновниця" foot_1 , "лiкар" foot_2 and "лiкарка" foot_3 , and more. as llms typically have not been trained on femininitive words, they may possess bias in this regard. this study aims to assess the applicability of existing english-language debiasing methods to ukrainian. to facilitate this, we introduce a ukrainian-language dataset specifically designed to measure and analyze gender bias in job-related contexts. by evaluating different debiasing strategies, we contribute to the broader effort of making ai systems more fair and inclusive across diverse linguistic and cultural settings. ## dataset currently, there are no publicly available datasets for measuring and mitigating gender bias in the "hiring problem" in ukrainian language. while some real-world datasets with candidate profiles exist such as the one presented in @xcite , they are limited to it jobs and are too complex for the smaller models we aimed to use. additionally, we did not translate existing english datasets @xcite , as one of our main goals was to evaluate bias specifically related to feminine forms, that do not exist in english. to address this challenge, we propose a synthetic dataset foot_4 specifically designed to measure gender bias in the context of the "hiring problem". to the best of our knowledge, this is the first dataset created for this task. the dataset was created using a list of professions foot_5 and by prompting gpt-4 foot_6 , asking it to generate both relevant and non-relevant experience examples for each profession. our dataset comprises all possible combinations of male and female pronouns and their corresponding professions in ukrainian. specifically, we include a sample of 351 professions. note that we included only "simple professions" consisting of single-word names. each profession has 8 sentence variations with each of the male / female, feminitive / nonfeminitive, and relevant / irrelevant experiences. note that male is not used in feminitive form, so we propose it in the dataset just for completeness. despite the dataset being synthetically generated, we manually reviewed and verified the data to ensure quality and correctness. the dataset contains the following columns: sentence, profession, experience, is_male, is_correct, is_feminitive. for an example, refer to appendix a. the presented dataset can be used to measure and mitigate bias in the "hiring problem". it is distributed under the mit license. we aim to capture explicit bias using a questionanswering qa based approach. the qaaccmetric used to evaluate the accuracy against our predefined labels in the dataset is the f1 score, which is calculated by comparing the identified prediction of the model based on cosine similarity with the ground truth. while this metric provides insight into the model's overall accuracy in comparison to the predefined labels, it does not fully capture the nuances of model behavior, particularly in terms of potential biases. to capture variations in model behavior across genders, we introduce a metric that measures the differences in predictions. ideally, we expect the qadiffmetric metric to be 0, which means that the predictions are consistent across genders. more details about definition of qa metrics can be found in the supplementary materials b.1. ## experiments results we tested the presented bias mitigation tenchniques in section 4 on 6 models that are capable of answering and understanding ukrainian language. from the results presented in tables 1 and 2 , we observed that the average difference in performance metrics between feminitive contexts (i.e., gendered feminine forms) and non-feminitive contexts was consistently larger. this suggests a potential bias introduced by the use of feminitives, indicating that word form can influence model predictions. the application of hard and soft debiasing techniques resulted in a slight reduction in the observed metric differences, yielding an average relative improvement of 18.9% with hard debias. however, this improvement was accompanied by a reduction in overall model accuracy (see appendix d tables 5 and 6), most notably impacting the probabilitybased approach. these findings suggest that while hard and soft debiasing methods have some effect on mitigating bias, their performance is limited, which aligns with expectations given the complex nature of contextual embeddings in transformer-based architectures. fine-tuning led to a notable improvement in overall accuracy across evaluated tasks, achieving approximately 0.9 on qa accuracy metrics. concurrently, the disparity between metrics in feminitive and non-feminitive contexts decreased substantially. this suggests that fine-tuning not only enhances performance but also helps mitigate some of the context-based biases. notably, for example, with qwen2.5-3b-instruct, we were able to achieve zero difference after applying hard debiasing. however, this came at the cost of lower qa accuracy. following finetuning, qa accuracy improved significantly, but the difference re-emerged, indicating a trade-off between fairness and performance. prompt-based debiasing yielded inconsistent results, indicating that this approach cannot be reliably used to mitigate bias. all experiments are available on the github repository 16 . ## intended use the presented dataset can be leveraged for the purposes outlined below: ## discussion in this work, we propose a benchmark for measuring gender bias in ukrainian and evaluate three mitigation strategies: fine-tuning, prompt-based debiasing, and embedding-level debiasing. while techniques adapted from english are somewhat effective, their performance is influenced by ukrainian's morphological richness, especially when dealing with femininitive forms. fine-tuning on domainspecific, gender-balanced data yielded the most consistent improvements, whereas prompt-based mitigation was easier to apply but less stable. notably, femininitive forms often led to unpredictable outputs, likely because of their underrepresentation in the training data-highlighting the need for linguistically diverse corpora. overall, our findings stress the importance of language-specific approaches and more inclusive benchmarks to ensure fairness in multilingual llms. ## limitations the dataset we propose is synthetic, generated through controlled combinations of gendered pronouns, feminitive forms, and experience labels. while this allows for systematic analysis, some generated sentences may not reflect the most natural or commonly used language forms. also, our dataset contains only single-word professions. the generalizability of our results remains an open question. we evaluated a small number of openly available llms, and the extent to which our findings apply to other models-especially closed-source or larger-scale multilingual mod-els-requires further exploration. additionally, our prompt debiasing evaluation relies on a single prompt, which may not be representative enough to fully assess the effectiveness of the method. ## ethical consideration we used chatgpt and grammarly to assist with paraphrasing and improving the clarity of writing throughout this paper. these tools were used strictly for language refinement and did not contribute to the research findings or analysis. additionally, our dataset was synthetically generated using gpt-4 to create controlled examples for measuring gender bias in the ukrainian language. our acting dean, oles dobosevych, for their support and for providing an inspiring academic environment that made this research possible. we are also grateful to rostyslav hryniv for mentoring the project in its early stages as part of the linear algebra course.