An Overview of Fairness in Data – Illuminating the Bias in Data Pipeline (2021)

Abstract: Data in general encodes human biases by default; being aware of this is a good start, and the research around how to handle it is ongoing. The term ‘bias’ is extensively used in various contexts in NLP systems. In our research the focus is specific to biases such as gender, racism, religion, demographic and other intersectional views on biases that prevail in text processing systems responsible for systematically discriminating specific population, which is not ethical in NLP. These biases exacerbate the lack of equality, diversity and inclusion of specific population while utilizing the NLP applications. The tools and technology at the intermediate level utilize biased data, and transfer or amplify this bias to the downstream applications. However, it is not enough to be colourblind, gender-neutral alone when designing a unbiased technology – instead, we should take a conscious effort by designing a unified framework to measure and benchmark the bias. In this paper, we recommend six measures and one augment measure based on the observations of the bias in data, annotations, text representations and debiasing techniques.

## introduction irrespective of the data sources, the majority of the bias prevails in data itself @xcite @xcite . the inherent bias in data affects the core nlp tasks such as part-of-speech (pos) tagging, pos chunking @xcite , and dependency parsing @xcite . other than data bias, the techniques used to represent the data also pose a threat to nlp systems @xcite @xcite . eventually the bias magnifies itself due to these biased data representation in downstream applications such as named entity recognition (ner) @xcite , coreference resolution @xcite @xcite , sentiment analysis @xcite , machine translation @xcite , social data analysis @xcite @xcite , and bio-medical language processing @xcite . however,a very little attention is given to data collection and processing. wikipedia seems like a diverse data source but fewer than 18% of the site's biographical entries represents women @xcite . @xcite observed that the dataset extracted from social media data has its own bias in various aspects such as age, gender, racism, location, job, and religion. existing research classified biases at various levels, including the bias in data -source itself, the bias in the data analysis pipeline, and the biased data in building the systems. for example, using a biased abusive language detection system may result in discrimination against a group of minority peoples such as african-american @xcite . the survey by blodgett and o'connor (2017) on bias in nlp systems found that works related to bias failed to explain why the system behaviours are 'biased', harmful, what kind of behaviours lead to bias, in what ways, to whom and why. the paper also discusses the need to conceptualise bias better by linking the languages and social hierarchies in the society. another survey by @xcite analyses the bias framework and issues with the current debiasing methods. their study reveals that the gender bias in nlp is still at a nascent level and lacks unified metrics and benchmark evaluation methods. this paper audits or surveys the present situation in analysing the bias at various levels of data. this paper has been divided into four section. in section 2, we analyse the bias in language resources or corpora, in word representations, in pre-trained language models. in section 3, based on the observation of bias in corpora level, social data, text representations, metric to measure and mitigate bias, we infer six recommendations and one measure augmenting the existing one. in section 4, we conclude with the need of practicing standard procedures across the data pipeline in nlp systems to support ethical and fairness usage of data. ## data bias language resources are present in two forms: 1) online digital data 2) annotated corpus. in fact language technologies are data-driven methods based on statistical learning techniques. there needs to be a framework to monitor the flow of data from the source to the nlp system and to measure the bias emanating at each level. the dataset can be representative of a specific genre or domain and a balanced one to avoid selectional bias as noted by @xcite . the bias also emanates from labelling the corpus by annotators. @xcite observed bias in widely-used penn treebank annotated corpora which contain many errors. apart from the linguistic experts, using non-experts in the annotation process through crowd-sourcing platforms also leads to considerable bias in data @xcite . figure.1 shows the effects of bias in data which leads to bias in gender, bias in syntactic and shallow parsing across domains, bias due to the disparity in language resources represented by tree hierarchy. ## observations and inferences the biases that are much extensively experimented can be categorized as: fine-grained such as gender, racism, religion, and location; coarse-grained such as demographic and gender+occupation. most experiments or analysis are performed on observing gender bias and its stereotypes from data and through word representations. the other types of bias, such as racism, sexism, and religion, are extensively studied in social data. the other biases from intersectional bias attributes such as demographic (age, gender, location), gender+racism, and gender+location+job are not extensively studied. much attention is needed to study the other types and bias based on the intersectional view. ## conclusion the above analysis at various levels reveals that most nlp systems consider bias at a nascent level. computer scientists should strive to develop algorithms that are more robust to biases in the data. various approaches are being pursued. such debiasing approaches are promising, but they need to be refined and evaluated in the real world. all need to think about appropriate notions of fairness in data. should the data be representative of the world as it is, or of a world that many would aspire to? to address these questions and evaluate the broader impact of training data and algorithms, machine-learning researchers must engage with social scientists, and experts in other domains. based on the observations of methods used at various levels, we recommend six measures and augment one measure to support ethical practice and fairness in the usage of data. practising and adopting various normative procedures across the data pipeline in nlp systems would enhance the equality, diversity and inclusion of different subgroups of peoples and their languages.