SlowGuess/NeurIPS_2023_OCR huggingface dataset:
FAIR PROJECTION [2] is a post-processing technique that can accommodate fairness constraints in a setting with multiple labels and multiple groups. The fair model is obtained from 'projecting' a pre-trained (and potentially unfair) classifier onto the set of models that satisfy target group-fairness requirements.
E Additional experiments and details on the experimental setup
Our proposed methodology can be summarized in the pipeline in Figure 6.
Figure 6: Flow chart of experimental procedure.
E.1 Data
The HSLS dataset [27, 28] is an education dataset collected from  $23,000+$  students across high schools in the USA. Features of the dataset contain extensive information on students' demographic information, their parents' income and education level, schools' information, and students' academic performances across years. We apply the pre-processing techniques adopted by Alghamdi et al. [2], with the number of samples reduced to 14,509. For the binary classification task with fairness constraints, the group attribute chosen is RACE  $\in$  {WHITE, NON-WHITE} and the prediction label is students 9th-Grade GRADEBIN  $\in$  {0, 1}, binarized according to whether a student's grade is higher or lower than the median.
The ENEM dataset [16] is a Brazilian high school national exam dataset introduced by Alghamdi et al. [2]. It has 138 features containing students' demographic information, socio-economic questionnaire answers (e.g., parents' education level and if they own a computer), and students' exam scores. Adopting the preprocessing technique in Alghamdi et al. [2], we sample 50K samples without replacement from the processed ENEM Year 2020 data. Identical to HSLS, the group attribute chosen is  $\text{RACE} \in \{\text{WHITE}, \text{NON-WHITE}\}$  and the prediction label is students Grade binarized into GRADEBIN  $\in \{0,1\}$  according to whether a student's grade is higher or lower than the median.
For the widely known Adult dataset [33], also known as "Census Income" dataset, we choose the group attribute as  $\mathrm{SEX} \in \{\mathrm{MALE}, \mathrm{FEMALE}\}$  and predicted label to be INCOME  $\in \{0,1\}$ , where income bin denotes whether a person's income is higher or lower than  $50\mathrm{K/yr}$ .
E.2 Competing Baseline Models
We use the Scikit-learn implementation of logistic regression, gradient boosting, and random forest as baseline models. For logistic regression and gradient boosting, the default hyperparameter is used; for random forest, we set the number of trees and minimum number of samples per leaf to 10 to prevent over-fitting. To get 10 competing models for each hypothesis class, we use 10 random seeds (specifically 33–42).
In practice, the competing models, i.e.,  $h \in \widehat{\mathcal{R}}_m$  can be obtained using different methodologies, such as sampling and adversarial weight perturbation [26, 40]. We suggest one method for sampling. First, split the data into training, validation, and test dataset. We train a set of models by changing the randomized procedures in the training process, e.g., using different initializations, different cuts for cross-validation, data shuffling, etc. In this paper, we change the random seed feed into the baseline models to obtain competing models. We use the validation set to measure  $\epsilon$  corresponding to this empirical Rashomon Set.
E.3 Competing Fair Models
For EQODDS, REJECTION, and REDUCTION, we use the functions EQODDSPOSTPROCESSING, REJECTOPTIONCLASSIFICATION, and EXPONENTIATEDGRADIENTREDUCTION from AIF360 toolkits [4]. For LEVERAGING and FAIR PROJECTION, we use the codes provided in the corresponding Github repositories of Chzhen et al. [12] and Alghamdi et al. [2].
Remark 3. We observe in practice that fairness classifiers are more confident and have scores that are thresholded-like (Figure 7 Left). From the similarity in the shape of the thresholded baseline curve and the fair models' curves (Figure 7 Right), thresholding-like behavior of some interventions may explain some—but certainly not all (see Figure 1 Right)—increase in score std dev and the ensuing arbitrariness. Recall from the violin plot in Figure 1 that the largest group (blue area) are
Figure 7: Left: We plot the score distribution of the REDUCTION approach as an example. The scores concentrate around 0 and 1, while scores of the baseline classifier are normally distributed. Right: Given this thresholded-like scores of fair classifiers, we include thresholded baseline in the quantile plot of score std.. The thresholded Baseline curve largely overlaps with Rejection curve with MEO 0.26 since they have the same MEO level.
Figure 8: Left: Accuracy-fairness curves of baseline random forest models v.s. fair models on the ENEM dataset. Right: Quantiles of per-sample score std. across high/low fairness models and baseline.
those individuals for which std. increases from 0 to a large positive value (median around  $>0.15$ ). Hence, the blue area shows that significant arbitrariness is introduced by the fairness intervention, in addition to and separate from the effects of thresholding the baseline.
E.4 Complete Experimental Plots
In order to evaluate the predictive multiplicity of models with similar accuracy and fairness levels, we divide the accuracy-fairness frontier plots into 8x8 grids and put models in the corresponding bins. To compare the arbitrariness of models satisfying high/low fairness constraints, we select bins in two different MEO ranges and bins with baseline models. Then, we compute the standard deviation of scores of models corrected by REDUCTION in the three bins (high fairness/low fairness/baseline) and plot the quantile curves. We use PANDAS package's quantile function with its default linear interpolation method.
Across three baseline model classes (random forest, gradient boosting, and logistic regression), fair models exhibit higher score arbitrariness. Especially at top quantiles, all fair models have standard deviations in scores going up to 0.5. This means that for the individuals at the top percentile, the model prediction can flip if another random seed is used in model training.
Furthermore, we evaluate the predictive multiplicity of models corrected by different fairness intervention methods. Across datasets, all fairness intervention methods exhibit maximal standard deviation of scores of 0.5 at top quantiles for random forest baseline methods. LEVERAGING [12] exhibit score arbitrariness comparable to that of baseline for GBM and Logistic Regression methods. REJECTION and LEVERAGING output thresholded scores directly, while REDUCTION outputs probabilities (with most scores close to 0 or 1).
Figure 9: Left: Accuracy-fairness curves of baseline gradient boosting models (GBM) v.s. fair models on the ENEM dataset. Right: Quantiles of per-sample score std. across high/low fairness models and baseline.
Figure 10: Left: Accuracy-fairness curves of baseline logistic regression models v.s. fair models on the ENEM dataset. Right: Quantiles of per-sample score std. across high/low fairness models and baseline.
Figure 11: Left: Accuracy-fairness curves of baseline random forest models v.s. fair models on the HSLS dataset. Right: Quantiles of per-sample score std. across high/low fairness models and baseline.
Figure 12: Left: Accuracy-fairness curves of baseline gradient boosting models (GBM) v.s. fair models on the HSLS dataset. Right: Quantiles of per-sample score std. across high/low fairness models and baseline.
Figure 13: Left: Accuracy-fairness curves of baseline logistic regression models v.s. fair models on the HSLS dataset. Right: Quantiles of per-sample score std. across high/low fairness models and baseline.
Figure 14: Left: Accuracy-fairness curves of baseline random forest models v.s. fair models on the Adult dataset. Right: Quantiles of per-sample score std. across high/low fairness models and baseline.
Figure 15: Left: Accuracy-fairness curves of baseline gradient boosting models (GBM) v.s. fair models on the Adult dataset. Right: Quantiles of per-sample score std. across high/low fairness models and baseline.
Figure 16: Quantile plot on models in high-fairness bin for various fairness interventions v.s. baseline models on ENEM. Fair models produce larger score std. at top percentiles compared to the baseline model.
Figure 17: Quantile plot on models in high-fairness bin for various fairness interventions v.s. baseline models on HSLS. Fair models produce larger score std. at top percentiles compared to the baseline model.