SlowGuess/NeurIPS_2023_OCR huggingface dataset:
To address the fairness concerns of having clusters with small sample size or sample imbalance, we require each cluster to have balanced data from each group. A feature  $j$  is considered for splitting only if  $\mathcal{I}_q^{j0}$  and  $\mathcal{I}_q^{j1}$  satisfy bounded representation for chosen parameters  $\alpha, \beta$ :
$$
\beta \leq \frac {| \{\boldsymbol {x} _ {i} \in \mathcal {I} _ {q} ^ {j k} | s _ {i} = s \} |}{| \mathcal {I} _ {q} ^ {j k} |} \leq \alpha , \quad s \in \mathcal {S}, k \in \{0, 1 \}.
$$
We also impose a minimum permissible cluster size  $k$  for  $\mathcal{I}_q^{j0}$  and  $\mathcal{I}_q^{j1}$ . We will denote the set of features  $j$  that satisfy these constraints by  $G(\mathcal{M}_q)$ . At each splitting iteration for each cluster  $\mathcal{M}_q$ , we split on  $j^* \in \arg \min_{G(\mathcal{M}_q)} L(\mathcal{M}_q, j)$ . After obtaining clusters  $\{\mathcal{M}_q\}_{q=1}^Q$ , we apply an off-the-shelf fairness intervention to train a fair linear classifier on each cluster. The fairness constraints we enforce during clustering ensure that the classifiers for each cluster can generalize to fresh data. We summarize our recursive partitioning method in Algorithm 1 (Appendix C).
5 Adapting Non-Linear Fair Classifiers to Missing Values
The algorithms above take advantage of the interpretability of linear classifiers to "deactivate" missing features via zero imputation. We now provide a general algorithm for nonlinear classifiers where the effect of zero imputation may be less clear. We introduce FairMissBag, a fair bagging algorithm that can be applied to any fair classifier to handle missing values. The algorithm proceeds in three steps (see Figure 4b, Appendix C for an illustration).
- The training data  $\mathcal{D}$  is separated into groups by sensitive attribute and label  $D_{s,y} = \{(x_i, y_i, s_i) | s_i = s, y_i = y\}$ . From each  $D_{s,y}$ , we sample  $|\mathcal{D}_{s,y}|$  data points uniformly with replacement. We combine these samples to create a new dataset of the same size as the original dataset. Missing indicators are added to each data point and an imputation method is used on each subsample separately. We repeat this process  $B$  times, creating  $B$  distinct datasets.
- Apply an off-the-shelf fairness intervention to train a classifier on each resampled dataset, yielding  $B$  classifiers in total.
- To make a prediction on a new data point, one of the  $B$  fair classifiers is chosen uniformly at random. We return the prediction from this classifier. Alternatively, if the classifiers output a probability or score for each label, the scores from all classifiers are averaged to generate an output.
In the first step, the uniform resampling procedure ensures sufficient sample size and diversity by preserving the balance of data across sensitive attribute and outcome, relative to the original data. Additionally, the inclusion of missing indicators as input features (Section 4.1) prevents the issue of reduced fairness-accuracy curve described in Theorem 1. Our procedure is similar to the fair resampling procedure introduced in Kamiran and Calders [28] and Wang and Singh [58] except that we relax the independence assumption of  $S$  and  $Y$ . This is advantageous in, for example, healthcare scenarios where a demographic group may have a higher prevalence of a disease (e.g., determined by age). It is likely desirable for a model to select patients from this group for treatment at a higher rate rather than enforce independence between  $S$  and  $Y$  by equalizing the selection rate between groups.
We show that our ensemble strategy in step 3 also preserves fairness by extending the analysis in [19]. We use equalized odds here as an example; the same fairness guarantee also holds for other fairness constraints such as statistical parity and equality of opportunity. If the base classifiers output scores (probabilities) for each class, an analogous argument justifies averaging the scores.
Lemma 1. Suppose we have an ensemble of classifiers  $\mathcal{C} = \{\mathcal{C}_j\}_{j=1}^M$  such that each classifier satisfies an equalized odds constraint, i.e., there exists  $\varepsilon$  such that for all  $j$ ,
$$
\left| \operatorname * {P r} (\mathcal {C} _ {j} (X) = 1 | Y = y, S = s) - \operatorname * {P r} (\mathcal {C} _ {j} (X) = 1 | Y = y, S = s ^ {\prime}) \right| \leq \varepsilon \quad \forall y \in \{0, 1 \}, s, s ^ {\prime} \in \mathcal {S}.
$$
Let  $p(j)$  denote a probability distribution over the classifiers and  $\mathcal{C}(\pmb{x})$  be the prediction obtained by randomly selecting a base classifier  $\mathcal{C}_k$  from  $p(j)$  and computing  $\mathcal{C}_k(\pmb{x})$ . Then  $\mathcal{C}$  also satisfies the equalized odds constraint.
6 Numerical Experiments
We evaluate our adaptive algorithms with a comprehensive set of numerical experiments by comparing our algorithms coupled with state-of-the-art fairness intervention algorithms on several datasets. We describe the setup and implementation and discuss the results of the experiments. Refer to Appendix D for additional experiment details.
6.1 Experimental setup
Datasets. We test our adaptive algorithms on COMPAS [34], Adult [12], the IPUMS Adult reconstruction [9, 15], and the High School Longitudinal Study (HSLS) dataset [22]. We refer the reader to Appendix D.2 for details on pre-processing each dataset and generating missing values for COMPAS and the two versions of Adult. The sensitive attribute is race in COMPAS and sex in Adult.
HSLS is a dataset of over 23,000 high school students followed from 9th through 12th grade that includes surveys from students, teachers, and school staff, demographic information, and academic assessment results [22]. We use an 11-feature subset of the dataset. The sensitive attribute is race (White/Asian versus Black, Hispanic, Native American, and Pacific Islander) and the predictive label is a student's 9th grade math test score. We consider only data points where both race and 9th grade math test score are present because our experiments use fairness interventions that use knowledge of the group attribute and label to calculate fairness metrics. HSLS exhibits high rates of missingness as well as significant disparities in missingness with respect to race. For instance,  $35.5\%$  of White and Asian students did not report their secondary caregiver's highest level of education; this proportion increases to  $51.0\%$  for underrepresented minority students [23].
Fairness intervention algorithms. We use DispMistreatment [61] and FairProjection [2] as two benchmark algorithms and defer additional experimental results using other benchmarks (Reduction [1], Eq0dds [20], ROC [29], Leveraging [8]) to Appendix E. All of these algorithms are designed for complete data and require missing values to be imputed before training.
For each combination of adaptive algorithm and fairness intervention, we plot the fairness-accuracy tradeoff curve obtained from varying hyperparameters (see Appendix D.3). We obtain a convex curve by omitting points that are Pareto dominated, i.e. there exists another point with higher accuracy and lower discrimination. The fairness metrics used are FNR difference and mean equalized odds, defined as  $\frac{1}{2}$  (FNR difference + FPR difference).
6.2 Linear fair classifiers
We test different combinations of the above fairness interventions with the adaptive algorithms in Section 4 using logistic regression as the base classifier. For the missing pattern clustering algorithm, we reserve part of the training set as validation to calculate the clustering objective function. We present results for FairProjection on IPUMS Adult/COMPAS and FairProjection and DispMistreatment on HSLS. We defer results for other benchmarks to Appendix E.
IPUMS Adult/COMPAS. Figure 1 displays fairness-accuracy curves for COMPAS and the IPUMS Adult reconstruction. We observe that for the datasets with MCAR and MAR missing values, the three methods tested have comparable performance with impute-then-classify. However, for the MNAR datasets, adding missing indicators and affinely adaptive classification significantly outperform the baseline especially with regard to accuracy, which is consistent with Theorem 1. We omit results for missing pattern clustering as the majority of trials did not find multiple clusters.
HSLS. Figure 2 displays the fairness-accuracy curves for DispMistreatment and FairProjection on HSLS. For DispMistreatment, adding missing indicators achieves marginally improved performance over the baseline, while affinely adaptive classification achieves the highest accuracy but at the cost of fairness. For FairProjection, all three adaptive methods outperform the baseline.
Discussion. The experiments on linear fair classifiers show that using the adaptive fairness-intervention algorithms proposed in Section 4 consistently outperform impute-then-classify in terms of accuracy and fairness across different datasets. The performance improvement of the adaptive
(a) IPUMS Adult, FairProjection
(b) COMPAS, FairProjection
Figure 1: Comparison of adaptive fairness-intervention algorithms on IPUMS Adult and COMPAS using FairProjection. For each original dataset, we generate three datasets with missing values corresponding to MCAR, MAR, and MNAR mechanisms (Appendix D.2). Error bars depict the standard error of 10 runs with different train-test splits.
fairness-intervention algorithms also increases when the missing patterns of the data encode significant information about the label, as is the case in the MNAR COMPAS/Adult datasets. Interestingly, although affinely adaptive classification and missing pattern clustering are both more expressive variants of adding missing indicators, adding missing indicators was able to achieve comparable performance to both methods in most cases. We include an additional experiment in Appendix E on a synthetic dataset (see Appendix D.1) where missing pattern clustering outperforms all other adaptive algorithms.
In short, we believe that the best adaptive algorithm is not universal. Our suggestion is to consider the choice of algorithm as a hyperparameter and select by using a validation set and considering the trade-offs between the methods proposed. Missing value indicators, though performing worse on average, has the benefit of being simple and interpretable. Affinely adaptive classification provides a potential accuracy gain at the cost of additional complexity, and missing pattern clustering should be preferred when there is sufficient data to cluster missing patterns in an advantageous way. Regardless of this choice, however, our results show that preserving information about missing values is important to achieve the best possible group fairness and accuracy.
6.3 Non-linear fair classifiers
We test the FairMissBag ensemble method proposed in Section 5 with FairProjection, Reduction, and EqOdds. The base classifier for all fairness-intervention algorithms is a random forest consisting of 20 decision trees with maximum depth 3. We run experiments with three imputation methods: mean, K-nearest neighbor (KNN), and iterative imputation [44]. We present results for FairMissBag on HSLS and defer additional experimental results to the appendix.
Figure 2: Comparison of adaptive fairness-intervention algorithms on HSLS dataset, using DispMistreatment (left) and FairProjection (right). Error bars depict the standard error of 10 runs with different train-test splits.
Figure 3: Results for FairMissBag on HSLS, using Reduction, EqOdds, and FairProjection. Fairness-accuracy curves under mean, KNN, and iterative imputation are shown, respectively. Error bars depict the standard error of 5 runs with different train-test splits.
Figure 3 displays the results of the FairMissBag experiment on HSLS. We observe that for Reduction, FairMissBag yields a meaningful fairness-accuracy improvement for KNN and iterative imputation. A similar improvement is observed for Eq0dds for mean and KNN imputation. For FairProjection, FairMissBag generally extends the fairness-accuracy curve, providing higher accuracy with a tradeoff of increased fairness risk.
Discussion. We observe that using FairMissBag yields fairness and accuracy improvements over impute-then-classify. Additionally, for both accuracy and fairness, the relative performance of the three imputation methods we tested depends on the dataset used and the fairness-intervention algorithm applied. We conclude that the choice of imputation method can significantly influence accuracy and fairness. This corroborates previous findings in e.g. [23, 26, 36] that the best imputation method for a given prediction task and setup is not always a clear choice.
In this work, we investigated the impact of missing values on algorithmic fairness. We introduced adaptive algorithms that can be used with any preexisting fairness-intervention algorithm to achieve higher accuracy and fairness than impute-then-classify. The flexibility and consistency of our algorithms highlight the importance of preserving the information in the missing patterns in fair learning pipelines. We note, however, that using missingness information in a fair classifier is not immune to potential negative impacts. For example, an individual may be incentivized to purposefully hide data if their true values are less favorable than even NA. In missing pattern clustering, an individual may be classified less favorably or accurately than if a classifier from a different cluster were used. These scenarios highlight considerations with respect to individual and preference-based fairness notions [55] and the importance of making careful, informed choices for both the adaptive algorithm and the fairness intervention. Another potential challenge is if sensitive groups are not known prior to deployment and must be learned online. Nevertheless, we hope that our insights can inform future work in fair classification with missing values.
Data imputation remains commonly used in data science pipelines. However, the predictions output by ML models are dependent on the imputation method used. When considering single data points, this presents an important challenge from an individual fairness perspective, because the choice of imputation may arbitrarily affect an individual's outcome [13]. The effect of imputation choice on individual outcomes parallels the "crisis in justifiability" that arises from model multiplicity, where a model may be unjustifiable for an individual if another equally accurate model provides the individual a better outcome [5, 41]. A future work direction is examining how to choose an imputation given these challenges and how the user may be involved in making this choice. Additionally, increased classification error or outcome volatility may harm individuals with many missing values. This raises the critical question of whether to withhold classification for individuals with too many missing values and how to provide resources to unclassified individuals. Ultimately, the ubiquity of missing data in the real world means that new developments in fairness with missing values have the potential to be broadly impactful and create better, fairer societal outcomes.
The authors would like to thank Prof. Haewon Jeong for her valuable input in the early stage of this project. This material is based upon work supported by the National Science Foundation under grants CAREER 1845852, CIF 2312667, FAI 2040880, CIF 1900750.
[60] Yang, F., Cisse, M., and Koyejo, O. O. (2020). Fairness with overlapping groups; a probabilistic perspective. Advances in Neural Information Processing Systems, 33.
[61] Zafar, M. B., Valera, I., Gomez-Rodriguez, M., and Gummadi, K. P. (2019). Fairness constraints: A flexible approach for fair classification. The Journal of Machine Learning Research, 20(1):2737-2778.
[62] Zhang, Y. and Long, Q. (2021a). Assessing fairness in the presence of missing data. Advances in neural information processing systems, 34:16007-16019.
[63] Zhang, Y. and Long, Q. (2021b). Fairness in missing data imputation. arXiv preprint arXiv:2110.12002.
A Omitted Proofs
A.1 Proof of Theorem 1
Proof. We prove this theorem via construction. We consider  $\mathcal{X} = \{0,1\}$  and construct the probability distribution  $P_{S,X,Y}$  in the following way:
$$
\operatorname * {P r} (Y = 0, X = 0 | S = s) = \operatorname * {P r} (Y = 0, X = 1 | S = s) = \frac {1 - \alpha_ {s}}{2}, \quad \operatorname * {P r} (Y = 0, X = \mathrm {N A} | S = s) = 0,
$$
$$
\Pr (Y = 1, X = 0 | S = s) = \Pr (Y = 1, X = 1 | S = s) = 0, \quad \Pr (Y = 1, X = \mathrm {N A} | S = s) = \alpha_ {s},
$$
and  $\operatorname*{Pr}(S = s) = q_s$ . By the construction of  $P_{S,X,Y}$ , we have
$$
\Pr (Y = 1, S = 0) = \alpha_ {0} q _ {0}, \quad \Pr (Y = 1, S = 1) = \alpha_ {1} q _ {1},
$$
$$
\operatorname * {P r} (Y = 0, S = 0) = (1 - \alpha_ {0}) q _ {0}, \qquad \qquad \operatorname * {P r} (Y = 0, S = 1) = (1 - \alpha_ {1}) q _ {1}.
$$
For this probability distribution, we can compute
$$
I (M; Y) = (1 - \alpha) \log {\frac {1}{1 - \alpha}} + \alpha \log {\frac {1}{\alpha}} = g (\alpha).
$$
Consider a classifier  $h$  such that  $h(0) = h(1) = 0$  and  $h(\mathsf{NA}) = 1$ . This classifier satisfies  $\mathbb{E}\left[\mathbb{I}(h(X) = Y)\right] = 1$  and  $\mathrm{Disc}(h) = 0$  where  $\mathrm{Disc}(h)$  is the equalized odds constraint. As a result,  $F_{\epsilon}(P_{S,X,Y}) = 1$  for any  $\epsilon$ . By symmetry, we assume  $f_{\mathrm{imp}}(\mathsf{NA}) = 1$  without loss of generality. Therefore, we have
$$
\operatorname * {P r} (Y = 0, \hat {X} = 0 | S = s) = \frac {1 - \alpha_ {s}}{2} \qquad \operatorname * {P r} (Y = 0, \hat {X} = 1 | S = s) = \frac {1 - \alpha_ {s}}{2},
$$
$$
\Pr (Y = 1, \hat {X} = 0 | S = s) = 0, \quad \Pr (Y = 1, \hat {X} = 1 | S = s) = \alpha_ {s}.
$$
We represent a probabilistic classifier  $h$  as
$$
\Pr (\hat {Y} = 0 | \hat {X} = 0) = p _ {0}, \quad \Pr (\hat {Y} = 1 | \hat {X} = 0) = 1 - p _ {0},
$$
$$
\Pr (\hat {Y} = 0 | \hat {X} = 1) = p _ {1}, \qquad \qquad \Pr (\hat {Y} = 1 | \hat {X} = 1) = 1 - p _ {1}.
$$
Consequently,
$$
\operatorname * {P r} (\hat {Y} = 0 | Y = 0, S = s) = \frac {p _ {0} + p _ {1}}{2}, \qquad \operatorname * {P r} (\hat {Y} = 1 | Y = 0, S = s) = 1 - \frac {p _ {0} + p _ {1}}{2},
$$
$$
\Pr (\hat {Y} = 0 | Y = 1, S = s) = p _ {1}, \quad \Pr (\hat {Y} = 1 | Y = 1, S = s) = 1 - p _ {1},
$$
which ensures that  $\operatorname{Disc}(h) = 0$  where  $\operatorname{Disc}(h)$  is the equalized odds constraint. Furthermore,
$$
\begin{array}{l} \mathbb {E} \left[ \mathbb {I} (\hat {Y} = Y) \right] = \operatorname * {P r} (\hat {Y} = Y) \\ = \operatorname * {P r} (\hat {Y} = 1 | Y = 1, S = 0) \operatorname * {P r} (Y = 1, S = 0) + \operatorname * {P r} (\hat {Y} = 1 | Y = 1, S = 1) \operatorname * {P r} (Y = 1, S = 1) \\ + \Pr (\hat {Y} = 0 | Y = 0, S = 0) \Pr (Y = 0, S = 0) + \Pr (\hat {Y} = 0 | Y = 0, S = 1) \Pr (Y = 0, S = 1) \\ = (1 - p _ {1}) \left(\alpha_ {0} q _ {0} + \alpha_ {1} q _ {1}\right) + \frac {p _ {0} + p _ {1}}{2} \left(1 - \alpha_ {0} q _ {0} - \alpha_ {1} q _ {1}\right). \\ \end{array}
$$
Note that  $\alpha_0q_0 + \alpha_1q_1 = \alpha$ . Hence,
$$
\begin{array}{l} \max  _ {h} \mathbb {E} \left[ \mathbb {I} (\hat {Y} = Y) \right] = \max  _ {p _ {0}, p _ {1} \in [ 0, 1 ]} (1 - p _ {1}) \alpha + \frac {p _ {0} + p _ {1}}{2} (1 - \alpha) \\ = \max _ {p _ {0}, p _ {1} \in [ 0, 1 ]} \alpha + \frac {1 - \alpha}{2} p _ {0} + \frac {1 - 3 \alpha}{2} p _ {1}. \\ \end{array}
$$
Since  $\alpha < 1/3$ , we have  $1 - \alpha > 0$  and  $1 - 3\alpha > 0$ . As a result, the above objective function reaches its maximum  $1 - \alpha$  when  $p_0$  and  $p_1$  are both equal to 1. In other words,  $F_{\epsilon}(P_{S,\hat{X},Y}) = 1 - \alpha$ .
A.2 Proof of Lemma 1
Proof. Let  $y \in \{0,1\}$  and  $s, s' \in S$ . Then
$$
\begin{array}{l} | \Pr (\mathcal {C} (X) = 1 | Y = y, S = s) - \Pr (\mathcal {C} (X) = 1 | Y = y, S = s ^ {\prime}) | \\ = \left| \sum_ {j} p _ {j} \Pr (\mathcal {C} _ {j} (X) = 1 | Y = y, S = s) - \sum_ {j} p _ {j} \Pr (\mathcal {C} _ {j} (X) = 1 | Y = y, S = s ^ {\prime}) \right| \\ \leq \sum_ {j} p _ {j} | \Pr (\mathcal {C} _ {j} (X) = 1 | Y = y, S = s) - \Pr (\mathcal {C} _ {j} (X) = 1 | Y = y, S = s ^ {\prime}) | \\ \leq \sum_ {j} p _ {j} \varepsilon = \varepsilon . \\ \end{array}
$$
B Information-Theoretic Interpretation of Adding Missing Indicators
We provide an information-theoretic perspective for how adding misisng indicators preserves information in the missing patterns. Let  $Y$  be the unobserved predictive label we wish to recover,  $X$  a variable representing the input features, and  $\hat{X}$  the input features under an imputation function  $f_{\mathrm{imp}}$ . By the data processing inequality,  $I(Y;X) \geq I(Y;\hat{X})$ , showing that imputation can lose information about  $Y$ . In contrast, because we can recover  $X$  perfectly from  $(\hat{X},M)$  without using any information from  $Y$ ,  $I(Y;X) = I(Y;(\hat{X},M))$ , i.e. all of the original information about  $Y$  is preserved.
Furthermore, we can use Fano's inequality [Theorem 6.3 in 46] to relate the information loss from imputation to classification error probability. Let  $h$  be a classifier and denote the classification error as  $p_{e}(X) = P(Y \neq h(X))$ . Fano's inequality states that
$$
H _ {2} \left(p _ {e} (X)\right) + p _ {e} (X) \log (| \mathcal {Y} | - 1) \geq H (Y | X).
$$
where  $\mathcal{V}$  denotes the support of  $Y$ ,  $H_{2}(p) = -p\log_{2}p - (1 - p)\log_{2}(1 - p)$  is the binary entropy function, and  $H(Y|X) = -\sum_{i,j}p(y_i,x_j)\log p(y_i|x_j)$  is the conditional entropy. Since  $Y\in \{0,1\}$ , the inequality reduces to
$$
H _ {2} \left(p _ {e} (X)\right) \geq H (Y | X) \Rightarrow p _ {e} (X) \geq H (Y | X) - (\log_ {2} 3 - 1)
$$
since  $H_{2}(p) \leq p + \log_{2} 3 - 1$  with equality achieved at  $p = \frac{1}{3}$ . This is a lower bound on  $p_{e}(X)$  in terms of  $H(Y|X)$ . Now, we can compare  $H(Y|\hat{X})$  and  $H(Y|(\hat{X}, M))$ . By above, we have
$$
I (Y; (\hat {X}, M)) \geq I (Y; \hat {X})
$$
$$
H (Y) - H (Y | (\hat {X}, M)) \geq H (Y) - H (Y | \hat {X})
$$
$$
H (Y | (\hat {X}, M)) \leq H (Y | \hat {X}).
$$
Therefore, Fano's inequality gives a better lower bound for  $p_e(\hat{X}, M)$  than for  $p_e(\hat{X})$ . We conclude that a classifier trained from adding missing indicators is more likely to achieve higher accuracy under the same group fairness constraint than a classifier trained using impute-then-classify.
C More Details on Algorithms
We include a figure illustrating the three adaptive algorithms presented in Section 4 for fair linear classifiers and the FairMissBag algorithm presented in Section 5. We also provide pseudocode for the recursive partitioning procedure in the missing pattern clustering algorithm and the fair uniform resampling procedure in FairMissBag.
C.1 Illustration of adaptive algorithms
(a) Fair linear classifiers
(b)FairMissBag
Figure 4: Illustration of the three adaptive algorithms for fair linear classifiers (left) and FairMissBag (right). Adding missing indicators and affinely adaptive classification encode missing value patterns via additional input features, while missing pattern clustering partitions the dataset based on missing patterns. FairMissBag is a three-step algorithm consisting of fair resampling, using a fairness intervention to train a fair classifier on each resampled dataset, and obtaining a prediction from the ensemble of fair classifiers.
C.2 Recursive partitioning algorithm for missing pattern clustering
We begin with a single cluster containing the entire dataset. At each step, we split a cluster on a feature  $j^{*} \in G(\mathcal{M}_{q})$  that minimizes the objective function  $L(\mathcal{M}_{q},j)$ , where  $G(\mathcal{M}_q)$  denotes the set of features that satisfy the bounded representation and minimum cluster size constraints.