Debunking Biases in Attention (2023)

Abstract: Despite the remarkable performances in various applications, machine learning (ML) models could potentially discriminate. They may result in biasness in decision-making, leading to an impact negatively on individuals and society. Recently, various methods have been developed to mitigate biasness and achieve significant performance. Attention mechanisms are a fundamental component of many state-of-the-art ML models and may potentially impact the fairness of ML models. However, how they explicitly influence fairness has yet to be thoroughly explored. In this paper, we investigate how different attention mechanisms affect the fairness of ML models, focusing on models used in Natural Language Processing (NLP) models. We evaluate the performance of fairness of several models with and without different attention mechanisms on widely used benchmark datasets. Our results indicate that the majority of attention mechanisms that have been assessed can improve the fairness performance of Bidirectional Gated Recurrent Unit (BiGRU) and Bidirectional Long Short-Term Memory (BiLSTM) in all three datasets regarding religious and gender-sensitive groups, however, with varying degrees of trade-offs in accuracy measures. Our findings highlight the possibility of fairness being affected by adopting specific attention mechanisms in machine learning models for certain datasets

## introduction recently, with the prosperity and popularity of large language models (llm) all over different industries, they have achieved outstanding results with considerably high accuracy in various downstream tasks according to naseem et al. @xcite . however, with incredible advancements come new challenges, particularly in the realm of fairness and biasness. the study @xcite demonstrated that google translate api, a popular and widely used machine translation system, exhibited a strong tendency towards male defaults, particularly in the field associated with stereotypes. as the llms are trained on large datasets, they have the potential to perpetuate or even amplify the bias inherent in the dataset @xcite . this problem has sparked a growing interest in exploring the fairness nature of nlp models and how to mitigate the biases. one of the most captivating research directions is using attention mechanisms. as the fundamental building block of the modern nlp paradigm, the attention mechanism was first introduced in 2014 in the machine translation domain @xcite . they have been proven to promote performance in different downstream nlp tasks significantly. despite that attention mechanisms can serve as post-processing debiasing techniques @xcite , few pieces of research have been done investigating the potential for attention mechanisms to affect the fairness of models. according to our knowledge, how they explicitly influence fairness has not been thoroughly explored yet. in this paper, we explore the impact of the attention mechanism on fairness. the key contributions of this work are: we investigate how different attention mechanisms affect the fairness of two recurrent neural networks (rnn) based models i.e., bigru and bilstm with different attention mechanisms in terms of offensive language classification tasks. our work studied the effects that attention mechanism can bring to bigru and bilstm on three different datasets, jigsaw @xcite , hate speech offensive language (hsol) @xcite and hatexplain @xcite , in terms of fairness and biasness. more specifically, we investigate influencing gender and religious biases in comparison experiments involving bigru and bilstm with or without different attention mechanisms and using equalized odd metrics. ## background and related works this section presents an overview of related work in attention mechanisms, including their developments and applications. following that, we will discuss the researches and techniques that have been utilized in the field of fairness. finally, we will examine the works and results from the intersection of attention mechanisms and fairness of the models. ## fairness in attention we investigated how the attention mechanism can affect group fairness across two different but homogeneous types of neural networks: bilstm @xcite and bigru @xcite . the reason for the choices of these two architectures is that as we want to investigate how attention mechanisms affect fairness performance, any self-attention-based architectures such as transformers @xcite become inappropriate choices. we chose to focus on text toxicity classification as our downstream tasks due to the relevance between the fairness performance of nlp models and the nature of text toxicity tasks. the definition of toxicity we incorporate here is from @xcite stated as 'anything that is rude, disrespectful, or unreasonable that would make someone want to leave a conversation.' ## results in this section, the results of the fairness comparison, the attention analysis, and the prediction analysis are reported. further experimental results and diagrams are analyzed and discussed in the appendix. ## discussions and limitations our study covered three types of widely used single attention with different mechanisms of assigning attention weights. however, we did not cover some compound attention mechanisms such as dual attention mechanism @xcite and co-attention @xcite , which might contain different patterns affecting the fairness of the models. also, transformer @xcite , the cornerstone of plms, should be considered in this study as it is composed of multiple self-attention modules, and the intersection impact of multiple attention mechanisms can be studied by incorporating this model. apart from the classifier itself, the different word representation models, which are well discussed in naseem et al. @xcite , can also be brought into scope since word embedding can also affect fairness. from the dataset aspect, the quality of text can be further improved with pre-processing techniques mentioned in naseem et al. @xcite to ensure better performance and reduce the effect of the irrelevant factors. also, since the toxicity classification tasks are not easy even for a human, there are noisy data inside the chosen datasets since we found that we disagree with some of the humanannotated labels by manual checking. furthermore, the hsol and jigsaw datasets are imbalanced in terms of distributions of different classes. therefore, modifications can be made to the loss function in the same way as focal loss @xcite or dice loss @xcite to mitigate the influence of data imbalance. ## conclusion in this work, we have investigated bigru and bilstm with three types of widely used attention mechanisms in three datasets regarding religious and gender-sensitive groups in terms of fairness performance as well as accuracy performance. the results demonstrate that all three types of attention mechanisms can mitigate the bias with a trade-off in accuracy in most scenarios of our experiments. these findings highlight that attention mechanisms, effective methods derived from human intuition of focusing, have the potential to be developed and incorporated as a debiasing methodology for bias mitigation in toxicity classification tasks.