## 5 Evaluation Measures
### Integrity Measures
#### 5.3.3 Fairness  
Another topic in AI and Explainable AI (XAI) research is fairness. This topic has emerged from the growing application of AI systems in critical societal domains such as healthcare, finance, and criminal justice. As these systems increasingly influence decisions that affect individuals' lives, ensuring fairness in AI is paramount to preventing discrimination and bias. Because of this, it becomes important that AI systems in critical domains make unbiased decisions that are perceived as fair by the various stakeholders. Studying perceived fairness has a long history in psychology and organizational science, where it has been conceptualized as a multi-dimensional construct, including issues like distributive fairness, procedural fairness, and interactional fairness. In our sample, the studies of [PERSON] et al. [86], [PERSON] et al. [73], and [PERSON] et al. [70] have mainly evaluated perceived fairness in the context of explainability. The studies of [PERSON] et al. [86] and [PERSON] et al. [73] focus on _distributive (or group) fairness_ and _procedural fairness_. [PERSON] et al. [86] define distributive fairness as the fairness of AI decision-making outcomes (or ends). Similarly, [PERSON] et al. [73] interpret group fairness as the model's comparative treatment of different groups, such as gender, race, or age. Both define procedural fairness in a complementary way as the fairness of the process (or means) by which individuals are judged and how AI systems generate outcomes. The study of [PERSON] et al. [70] addresses _informational fairness_ concerns in automated decision-making. They define informational fairness as individuals feeling provided with adequate information and explanations about the decision-making process and its outcomes. Because of the different research interests, the studies operationalize fairness differently. [PERSON] et al. [73] conducted an experimental study comparing two ML models for predicting bail denial. These models implement various fairness-related properties. They used a mixed-methods approach where participants should rate the fairness, bias, and utility of each model on a five-point Likert scale. In addition, the survey included free-respond questions asking the participants to give reasons for their ratings. These responses were thematically coded. [PERSON] et al. [86] also studied ML models for predicting bail denial, focusing on how human perceptions of process fairness might conflict with the prediction accuracy of these systems. In the first step, they evaluated participants' judgments of algorithmic process fairness by asking three yes/no questions: _\"Do you believe it is fair or unfair to use this information?\"_[86], _\"Do you believe it is fair or unfair to use this information if it increases the accuracy of the prediction?\"_[86], and _\"Do you believe it is fair or unfair to use this information if it makes one group of people (e.g., African American people) more likely to be falsely predicted?\"_[86]. In the second step, they assessed how the prediction accuracy of ML models would change if the participants' fairness judgments were considered. [PERSON] et al. evaluated how explanations affect people's perception of informational fairness by using a scenario of an automated decision-making process for approving loan eligibility. The experimental design covers the following conditions: (i) baseline with no explanation, (ii) what factors are considered in making the decision, (iii) the relative importance of these factors, and (iv) additional counterfactual scenarios. For each condition, participants rated informational fairness and trustworthiness using Likert-scaled items. The study also collected qualitative feedback through open-ended questions to gain deeper insights into what information they believe is necessary to judge the fairness of the decision-making and their views on the appropriateness of the explanations provided.