## 5 Conclusions
### Challenges and Future Research Directions  
Our review also identified various challenges with respect to existing research on fair algorithmic decisions. First, our review identified multiple definitions of what is a fair decision-making algorithm and diverse approaches to ensuring fairness in algorithmic decisions. This becomes particularly true as fairness being a social construct gets measured in various notions that often correspond to differing lens in social sciences, justice, economics, and moral philosophy. Such diverse (and often uncorrelated) definitions and methods on the one hand provides a variety of tools to address different manifestations of bias and discrimination embedded in data. On the other hand existence of different definitions has led the research community into diverging path of research endeavors leading to a defragmented domain of science. For instance, consider the two salient measures of fairness, (a) algorithmic fairness that requires the score that an algorithm produces to be equally accurate for all members vs. (b) algorithmic fairness that requires that the algorithm produces the same percentage of errors in terms of prediction for each group under consideration. Even though there exists normative commonality across these measures, there is so far no algorithmic solution to achieve parity in both these dimensions. Moreover, there is no consensus in literature on what is the best definition of fairness under a given circumstance. Theoretical and empirical evidence showing that different definitions of fairness cannot be satisfied at once makes it even difficult endeavour for policy-makers. To this end, evaluating each definition and method to decide on which definition and method to consider for a given task is a daunting task. Therefore, it is important for the algorithmic fairness community to move towards a converging path. For instance, a unified framework by [PERSON] et al. [109] is an important and encouraging first step in this direction. To this end, we hope that this article provides a broad overview for such an effort to successfully be accomplished. Moreover, our view is that such a unified framework should cross domains of algorithms and not just remain limited to machine learning. Second, our review also discovered that a large majority of above reviewed work is centered on the development of statistical definitions of fairness and methods to expose and remove the corresponding biases. Research efforts need to be directed to bridge the gap between mathematical and algorithmic research in academia and their application in practice. See, for example [PERSON], [PERSON], and [PERSON] for such a work [110]. In order to make fair algorithms accessible to practitioners, easy-to-use and off-the-shelf tools need to be developed. We have already seen encouraging first steps in this direction from the Human-Computer Interaction (HCI) community (see, e.g., the works by the authors of [111; 112]). Future work should therefore aim at linking the definition of fairness studied in research to the definition of fairness based on user's perception. For example, in their recent work [PERSON], [PERSON], and [PERSON] [46] found that most simplistic mathematical definitions of fairness (i.e., demographic parity) most closely matches the people's idea of fairness in practice. This association remains true even when the participants were explicitly informed about the existence of other more complicated notions of fairness [46]. In the same vein, [PERSON] et al. [113] conducted a systematic investigation of commercial product teams' challenges and needs for support in developing fairer ML systems. The study identified various areas of alignment and disconnect between the challenges faced by teams in practice and the solutions proposed in the fair ML research literature. Similar research associating the work in academia and practice should be beneficial in making the fairness in algorithms literature more realistic and more easily accessible to practitioners.