SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Raymond Feng $^{1}$ , Flavio P. Calmon $^{1}$ , Hao Wang $^{2}$
$^{1}$ John A. Paulson School of Engineering and Applied Sciences, Harvard University  $^{2}$ MIT-IBM Watson AI Lab
raymond_feng@college.harvard.edu, flavio@seas.harvard.edu, hao@ibm.com
Missing values in real-world data pose a significant and unique challenge to algorithmic fairness. Different demographic groups may be unequally affected by missing data, and the standard procedure for handling missing values where first data is imputed, then the imputed data is used for classification—a procedure referred to as “impute-then-classify”—can exacerbate discrimination. In this paper, we analyze how missing values affect algorithmic fairness. We first prove that training a classifier from imputed data can worsen the achievable values of group fairness and average accuracy. This is because imputing data results in the loss of the missing pattern of the data, which often conveys information about the predictive label. We present scalable and adaptive algorithms for fair classification with missing values. These algorithms can be combined with any preexisting fairness-intervention algorithm to handle missing data while preserving information encoded within the missing patterns. Numerical experiments with state-of-the-art fairness interventions demonstrate that our adaptive algorithms consistently achieve higher fairness and accuracy than impute-then-classify across different datasets.
Missing values exist in many datasets across a wide range of real-world contexts. Sources of missing values include underreporting in medical information [18], structural missingness from data collection setups [43], and human error. Missing data poses a challenge to algorithmic fairness because the mechanisms behind missing values in a dataset can depend on demographic attributes such as race, sex, and socioeconomic class. For example, in housing data, non-white housing applicants or sellers may be less likely to report their race due to concerns of discrimination [51]. Real-world data collection and aggregation processes can also lead to disparate missingness between population groups. If missing data is not handled carefully, there is a high risk of discrimination between demographic groups with different rates or patterns of missing data. For example, low-income patients can have more missing values in medical data involving the results of costly medical tests [37], making them less likely to qualify for early interventions or get treatment for insufficiently severe conditions [17].
The issue of missing values has not been sufficiently addressed in current fairness literature. Most fairness-intervention algorithms that have been proposed are designed for complete data and cannot be directly applied to data with missing values [e.g., 1, 2, 7, 10, 25, 32, 60, 61]. A common approach to handle missing values is to first perform imputation, filling in missing entries with appropriate values, and then apply an off-the-shelf fairness-intervention algorithm to the imputed data (impute-then-classify). However, this standard procedure has several limitations. In many contexts and data formats, imputation is not an appropriate method for handling missing values. In healthcare data, some missing data is categorized as "intended missing data," which, for example, the National Cancer Institute does not recommend imputing [43]. For instance, if a respondent answers NO to the question "Were you seen for an illness or injury?", they should skip the question "How often
did you get care for an illness or injury as soon as you wanted?". Imputing the second question does not make sense knowing the format of the data. The missingness pattern of the data can also convey important information about the predictive label. For instance, in surveys with questions about alcohol consumption history, missing responses are often correlated with alcoholism due to pervasive social stigma surrounding alcoholism [39]. A model trained on imputed data will suffer from reduced performance, especially for groups with higher missing data rates.
In this work, we investigate how missing values in data affect algorithmic fairness. We focus on the setting where input features of predictive models may have missing values. Although a growing body of research [27, 62] is dedicated to addressing the issue of missing group (sensitive) attributes, the problem of missing input features is equally crucial and widespread yet less studied. Our objective in this paper is to bridge this gap and offer a comprehensive study encompassing theory and algorithms for training fair classifiers in the presence of missing input features, particularly when missingness patterns vary per population group (as observed in practice).
First, we show that the performance of a fair classifier trained with impute-then-classify can be significantly lower than one trained directly from data with missing values. Impute-then-classify may fail because imputation loses information from the missing pattern of the data that can be useful for prediction. Our result is information-theoretic: a data scientist cannot improve this fundamental limit by altering the hypothesis class, using a different imputation method, or increasing the amount of training data.
Next, we introduce methods for adapting fairness-intervention algorithms to missing data. In Section 4, we present three adaptive algorithms for training fair linear classifiers on data with missing values, and in Section 5, we introduce an algorithm that adapts general fairness interventions to handle missing values. We treat the linear case separately because the interpretability of linear classifiers facilitates their widespread use in practice [11, 47, 54, 56]. Our algorithms work by modifying the dataset to preserve the information encoded in the missing patterns, then applying an off-the-shelf fairness-intervention algorithm to the new dataset. This avoids the information-theoretic limitation of impute-then-classify described in our theoretical result above. Most importantly, our algorithms are flexible because they can be combined with any preexisting fairness intervention to handle missing values fairly, accurately, and efficiently. Other advantages include scalability to high-dimensional data, generalizability to new data, and the ability to handle new missing patterns that occur only at testing time.
We benchmark our adaptive algorithms through comprehensive numerical experiments. We couple the adaptive algorithms with state-of-the-art fair classification algorithms and compare the performance of these methods against impute-then-classify. Our results show that the adaptive algorithms achieve favorable performance compared to impute-then-classify on several datasets. The gap in performance is more pronounced when missing data patterns encode more information, such as when data missingness is correlated with other features, i.e., not missing uniformly at random. Although the relative performance of the adaptive methods depends on the data distribution, even a simple method such as adding missing indicators as input features consistently achieves better fairness-accuracy curves than impute-then-classify. When applying fairness interventions on missing data, we find that utilizing information from missing patterns is crucial to mitigate the fairness and accuracy reduction incurred by imputation. Our results suggest that practitioners using fairness interventions should preserve information about missing values to achieve the best possible group fairness and accuracy.
1.1 Related work
Missing values in supervised learning. A line of recent works has investigated the limitations of performing imputation prior to classification [4, 26, 36]. In a remarkable paper, Le Morvan et al. [36] found that even if an imputation method is Bayes optimal, downstream learning tasks generally require learning a discontinuous function, which weakens error guarantees. This motivated a neural network algorithm that jointly optimizes imputation and regression using an architecture for learning optimal predictors with missing values introduced in [35, 42]. Among related work in supervised learning with missing values, the closest to ours is [4] and [31]. Bertsimas et al. [4] devised several algorithms for a general linear regression task with missing values where the missingness pattern of the data is utilized to make predictions. In particular, the static regression with affine intercept algorithm involves adding missing indicators as feature variables and has been found to improve performance for standard prediction tasks [45, 50, 52, 57]. Our work generalizes their algorithms
to fair classification tasks. We also extend the family of ensemble approaches in Khan et al. [31] to fair classification using a general, non-linear base classifier. Our approach is similar to [49] in using bootstrapping to satisfy strengthened fairness conditions; a key difference is that we explicitly use (known) sensitive attributes and labels when drawing subsamples to ensure sample diversity.
Missing values in fair machine learning. Discrimination risks related to missing values were studied in [6, 14, 23, 48, 53, 58]. Caton et al. [6], Fernando et al. [14], Jeong et al. [23], Mansoor et al. [40], Schelter et al. [48], Zhang and Long [63] provided empirical assessments on how different imputation algorithms impact the fairness risks in downstream prediction tasks. These studies have highlighted that the choice of imputation method impacts the fairness of the resulting model and that the relative performance of different imputation methods can vary depending on the base model. To mitigate discrimination stemming from missing values, researchers have proposed different methods for fair learning with missing values for tabular data [23, 58] and for graph data [53]. Among these works, the closest one to ours is Jeong et al. [23] which proposed a fair decision tree model that uses the MIA approach to obtain fair predictions without the need for imputation. However, their algorithm hinges on solving a mixed integer program, which can be computationally expensive.
Another related line of work studies how one can quantify the fairness of a ML model when the required demographic information (i.e., sensitive group attributes) is missing. Methods to estimate fairness of models in the absence of sensitive group attributes have been developed using only observed features in the data [27] or complete case data [62]. Multiple works have also studied fairness when predictive labels themselves are missing. Wu and He [59] propose a model-agnostic approach for fair positive and unlabeled learning. Ji et al. [24] propose a Bayesian inference approach to evaluate fairness when the majority of data are unlabeled. Lakkaraju et al. [33] propose an evaluation metric in settings where human decision-makers selectively label data and make decisions using unobserved features. In contrast, we focus on algorithm design for the case of missing non-group attribute input features.
Throughout, random variables are denoted by capital letters (e.g.,  $X$ ), and sets are denoted by calligraphic letters (e.g.,  $\mathcal{X}$ ). We consider classification tasks where the goal is to train a classifier  $h$  that uses an input feature vector  $\pmb{x} = (x_{1},\dots ,x_{d})$  to predict a label  $y$ . We restrict our domain to binary classification tasks where  $y\in \{0,1\}$ , but our analysis can be directly extended to multi-class tasks. In practice,  $\pmb{x}$  may contain missing values, so we let  $\pmb{x}\in \prod_{i = 1}^{d}(\mathcal{X}_i\cup \{\mathrm{NA}\})$ .
We adopt the taxonomy in [38] and consider three main mechanisms of missing data: missing completely at random (MCAR), where the missingness is independent of both the observed and unobserved values, missing at random (MAR), where the missingness depends on the observed values only, and missing not at random (MNAR), where the missingness depends on the unobserved values. We introduce a missingness indicator vector  $\mathbf{m} \in \mathcal{M} = \{0,1\}^d$  that represents the missingness pattern of  $\mathbf{x}$ : the missingness indicator  $m_j = 1$  if and only if the  $j$ th feature of  $\mathbf{x}$  is missing.
We evaluate differences in classifier performance with respect to a group (sensitive) attribute  $S \in S$ . Fairness-intervention algorithms often aim to achieve an optimal fairness-accuracy trade-off by solving a constrained optimization problem
$$
\max  _ {h} \mathbb {E} [ \mathbb {I} (h (X) = Y) ] \quad \text {s . t . D i s c} (h) \leq \epsilon \tag {1}
$$
for some tolerance threshold  $\epsilon \geq 0$ . Here  $\mathbb{I}$  is the indicator function, which equals 1 when the classifier  $h$  produces the correct prediction  $h(X) = Y$ , and otherwise, it equals 0.  $\mathrm{Disc}(h)$  represents the violation of a (group) fairness constraint. For example, we can define  $\mathrm{Disc}(h)$  for equalized odds as
$$
\operatorname {D i s c} (h) = \max  _ {y, \hat {y}, s, s ^ {\prime}} | \Pr (h (X) = \hat {y} | Y = y, S = s) - \Pr (h (X) = \hat {y} | Y = y, S = s ^ {\prime}) |.
$$
Equalized odds addresses discrimination by constraining the disparities in false positive and false negative rates between different sensitive groups. Analogous expressions exist for other group fairness constraints such as equality of opportunity and statistical parity [13, 20]. In the case where multiple, overlapping sensitive attributes exist in the data, our framework can be extended to the multiaccuracy and multicalibration fairness measures introduced in Kim et al. [32] and Hébert-Johnson et al. [21].
3 Characterization of Fairness Risk on Imputed Data
Methods for learning a fair classifier can fail when data contains missing values. For example, many fairness interventions use base classifiers such as linear classifiers and random forest that do not support missing values [e.g., 2, 61]. Others use gradient-based optimization methods that require a continuous input space [e.g., 1]. Impute-then-classify is the most common way of circumventing this issue. However, a key limitation of impute-then-classify is that imputing missing values loses information that is encoded in the missingness of the data. If the data contains missing values that are not MCAR, the fact that a feature is missing can be informative. A model trained on imputed data loses the ability to harness missingness to inform its predictions of the label. We introduce a theoretical result that rigorously captures this intuition.
We represent an imputation mechanism formally as a mapping  $f_{\mathrm{imp}}: \prod_{i=1}^{d} (\mathcal{X}_i \cup \{\mathrm{NA}\}) \to \prod_{i=1}^{d} \mathcal{X}_i$  and denote the imputed features by  $\hat{x} = f_{\mathrm{imp}}(x)$ . We define the underlying data distribution by  $P_{S,X,Y}$  and the distribution of imputed data by  $P_{S,\hat{X},Y}$ . We use the mutual information to measure the dependence between the missing patterns  $M$  and the label  $Y$ :
$$
I (M; Y) \triangleq \sum_ {\boldsymbol {m} \in \mathcal {M}} \sum_ {y \in \{0, 1 \}} P _ {M, Y} (\boldsymbol {m}, y) \log \frac {P _ {M , Y} (\boldsymbol {m} , y)}{P _ {M} (\boldsymbol {m}) P _ {Y} (y)}.
$$
Recall the constrained optimization problem in (1) for training a fair classifier. When the optimization is over all binary mappings  $h$ , the optimal solution only depends on the data distribution  $P_{S,X,Y}$  and the fairness threshold  $\epsilon$ . In this case, we denote the optimal solution of (1) by  $F_{\epsilon}(P_{S,X,Y})$ . The next theorem states that even in the simple case where a single feature has missing values, imputethen-classify can significantly reduce a classifier's achievable performance under a group fairness constraint when the missingness pattern of the data is informative with respect to the predictive label.
Theorem 1. Suppose that  $X$  is composed by a single discrete feature (i.e.,  $d = 1$ ) and  $\operatorname{Disc}(h)$  is the equalized odds constraint. Let  $g:[0,1] \to \mathbb{R}$  be the binary entropy function:  $g(a) = -a\log(a) - (1 - a)\log(1 - a)$ . For any  $\epsilon$  and  $a < 1/3$ , there exists a data distribution  $P_{S,X,Y}$  such that  $I(M;Y) = g(a)$  and
$$
\sup  _ {f _ {i m p}} F _ {\epsilon} (P _ {S, \hat {X}, Y}) \leq F _ {\epsilon} (P _ {S, X, Y}) - a.
$$
This information-theoretic result shows that applying a fairness-intervention algorithm to imputed data can suffer from a significant accuracy reduction which grows with the mutual information between the missing patterns and the label. Note that the supremum is taken over all imputation mechanisms; the inequality holds regardless of the type of classifier, imputation method used, or amount of training data available. In other words, the negative impact to performance due to information loss from imputation is unavoidable if using impute-then-classify.
4 Adapting Linear Fair Classifiers to Missing Values
In the last section, we showed how a classifier trained from impute-then-classify may exhibit suboptimal performance under a (group) fairness constraint. We now address this issue by presenting adaptive algorithms that can be combined with fairness interventions to make fair and accurate predictions on data with missing values. We extend prior work in [4] to a fair classification setting.
We focus first on adapting fairness interventions to missing values in linear classifiers, and present results for non-linear methods in the next section. We treat the linear case separately since, despite their simplicity, linear classifiers are widely used in practice and in high-stakes applications (e.g., medicine, finance) due to their interpretability [11, 47, 54, 56]. The adaptive algorithms we propose allow a model to handle any missing pattern that appears at training and/or test (deployment) time.
We propose three methods tailored to linear classification: (i) adding missing values as additional features, (ii) affinely adaptive classification, and (iii) missing pattern clustering. Our methods encode missing value patterns via adding new features in the case of (i) and (ii) and partitioning the dataset in the case of (iii), allowing a fairness intervention to then be applied on the encoded data. For example, if the fairness intervention is a post-processing scheme, a linear classifier is fit on the encoded data and then modified by the intervention. For in-processing schemes [1, 7, 30], the fair linear classifier is simply trained on the encoded data. The three methods are illustrated in Figure 4a (Appendix C).
4.1 Adding missing indicator variables
Our first adaptive algorithm is to pass  $\mathbf{m}$  as part of the input feature vector and impute missing values by zero. Letting  $\hat{\mathbf{x}}$  denote the imputed original features, our new feature vectors are now of the form  $(\hat{\mathbf{x}}, m) \in (\prod_{i=1}^{d} \mathcal{X}_i) \times \{0,1\}^d$ , and we can then train an off-the-shelf fairness-intervention algorithm on the new data. This approach is highly scalable as the number of features in the new data is  $2d$ . We can interpret learning a linear classifier  $\mathbf{w} = (w_1, \dots, w_d, b_1, \dots, b_d)$  on the transformed dataset (where  $b_j$  is the coefficient of  $m_j$ ) as equivalent to jointly learning a linear classifier  $\mathbf{w}' = (w_1, \dots, w_d)$  on the original data and an imputation method that replaces missing values in feature  $j$  with  $\frac{b_j}{w_j}$  [4]. Alternatively, noting that  $\mathbf{w}^\top(\hat{\mathbf{x}}, m) = \sum_{j=1}^{d} w_j x_j (1 - m_j) + \sum_{j=1}^{d} b_j m_j$ , adding missing indicators effectively allows the bias term to be adjusted by  $b_j$  when feature  $j$  is missing.
Intuitively, using missing indicators captures the information encoded in the missingness patterns of the data. We provide an information-theoretic interpretation of this intuition in Appendix B.
4.2 Affinely adaptive classification
As mentioned above, adding missing indicator variables allows the bias term of a linear classifier to be adjusted by a constant when a feature is missing. We can treat the coefficients of the classifier in a similar way. Affinely adaptive regression includes additional input features of the form  $m_{k}(1 - m_{j})x_{j}$  for  $j,k\in [d]$  with  $j\neq k$ , then applying zero imputation and an off-the-shelf fairness intervention. This can be interpreted as training a linear classifier with coefficients  $\mathbf{w}_0 + \mathbf{W}\mathbf{m}$ , where  $\mathbf{w}_0$  is an initial set of coefficients and  $W_{jk}$  denotes the adjustment to  $w_{j}$  when  $x_{k}$  is missing:
$$
\left(\boldsymbol {w} _ {0} + \mathbf {W} \boldsymbol {m}\right) ^ {\top} \hat {\boldsymbol {x}} = \sum_ {j = 1} ^ {d} w _ {0 j} (1 - m _ {j}) x _ {j} + \sum_ {j = 1} ^ {d} \sum_ {k = 1} ^ {d} W _ {j k} m _ {k} (1 - m _ {j}) x _ {j}.
$$
Affinely adaptive classification generalizes simply adding missing indicators by including a bias term as an additional input feature. However, a significant drawback is that affinely adaptive classification requires  $O(n_{\mathrm{miss}}d)$  features where  $n_{\mathrm{miss}}$  is the number of original features that contain missing values, though this number may be small in practice.
4.3 Missing pattern clustering
The previous two algorithms transform the original dataset by adding additional input features encoding data missingness. We now introduce a two-step missing pattern clustering algorithm. First, the missing patterns in the data are split into clusters. Then, an off-the-shelf fairness intervention algorithm is applied on each cluster separately. Rather than encoding data missingness with additional input features, this algorithm avoids the limitation of impute-then-classify by using missing patterns to partition the dataset. We also impose group fairness constraints in the clustering process to ensure sufficient sample size and diversity in each cluster. This allows the fair classifiers on each cluster to generalize to fresh data.
During clustering, we split all possible missing patterns  $\mathcal{M}$  into  $Q$  clusters  $\{\mathcal{M}_q\}_{q=1}^Q$  using a recursive partitioning method. Let  $\mathcal{P}$  denote the current (intermediate) partition, so initially  $\mathcal{P} = \{\mathcal{M}\}$ . At each step, we split each cluster  $\mathcal{M}_q \in \mathcal{P}$  based on the missingness of some feature  $j$  into two new sets  $\mathcal{M}_q^{j0} = \{\pmb{m} \in \mathcal{M}_q : m_j = 0\}$  and  $\mathcal{M}_q^{j1} = \{\pmb{m} \in \mathcal{M}_q : m_j = 1\}$ . We split the training dataset correspondingly into  $\mathcal{I}_q^{j0} = \{(\pmb{x}_i, y_i) : \pmb{m}_i \in \mathcal{M}_q^{j0}\}$  and  $\mathcal{I}_q^{j1} = \{(\pmb{x}_i, y_i) : \pmb{m}_i \in \mathcal{M}_q^{j1}\}$ . To determine the feature on which to split, we introduce a loss function:
$$
L \left(\mathcal {M} _ {q}, j\right) \triangleq \min  _ {\boldsymbol {w}} \sum_ {i \in \mathcal {I} _ {q} ^ {j 0}} \ell \left(y _ {i}, \boldsymbol {w} ^ {\top} \boldsymbol {x} _ {i} ^ {\prime}\right) + \min  _ {\boldsymbol {w}} \sum_ {i \in \mathcal {I} _ {q} ^ {j 1}} \ell \left(y _ {i}, \boldsymbol {w} ^ {\top} \boldsymbol {x} _ {i} ^ {\prime}\right)
$$
where  $\pmb{x}_i'$  is  $\pmb{x}_i$  with missing values imputed by zero. In other words,  $L(\mathcal{M}_q, j)$  is the total loss from training linear classifiers on  $\mathcal{I}_q^{j0}$  and  $\mathcal{I}_q^{j1}$ .
$L(\mathcal{M}_q,j)$  satisfies two desirable properties. First, it can be calculated efficiently as it does not include fairness risk, which is crucial as there are  $O(d)$  candidate features for each split. Second,  $L(\mathcal{M}_q,j)$  guarantees that the total loss with respect to the standard classifiers
$$
\sum_ {\mathcal {M} _ {q} \in \mathcal {P}} \min  _ {\boldsymbol {w}} \sum_ {i \in \mathcal {I} _ {q}} \ell \left(y _ {i}, \boldsymbol {w} ^ {\top} \boldsymbol {x} _ {i} ^ {\prime}\right)
$$
never increases after a split, provided that we split a cluster  $\mathcal{M}_q$  only if  $L(\mathcal{M}_q,j) < \min_{\boldsymbol{w}}\sum_{i\in \mathcal{I}_q}\ell (y_i,\boldsymbol{w}^\top \boldsymbol{x}_i')$ , i.e. the two new classifiers collectively perform at least as well as a classifier trained on the whole cluster.