SlowGuess/NeurIPS_2023_OCR huggingface dataset:
The main challenge in deriving the upper bound of Theorem 1 lies in analyzing the following problem: given  $X$  following a non-isotropic Gaussian distribution with mean  $\mu$ , find upper bounds on  $\mathbf{E}[(X / \|X\| - \mu / \| \mu \|)^2]$  and  $\mathbf{E}[(\|X\| - \|\mu\|)^2]$ . Solving this problem provides estimation errors for the norm and direction estimators, as the OLS estimator is an unbiased estimator with noise following the non-isotropic Gaussian distribution. Our key technical contribution is the derivation of these bounds (Theorems 4 and 5).
Technical difficulty in deriving the lower bound in Theorem 1. The minimax optimal error characterizes the intrinsic complexity of the regression problem, as no algorithm can surpass this error. In our analysis of the lower bound presented in Theorem 1, we demonstrate that the fair regression problem's complexity, under the model Eq (2), is characterized by the complexity in estimating the direction  $\beta_s^* / \| \beta_s^*\|$ . The primary challenge lies in establishing this characterization.
To overcome this challenge, we investigate the geometric structure of the error term  $\mathcal{E}_n(f;\beta_s^*,\mu .)$  concerning the parameters  $\beta_{s}^{*}$  and  $\mu$  . We then reveal that the geometric structure of  $\mathcal{E}_n(f;\beta_s^*,\mu .)$  is characterized by the geometric structure of the direction  $\beta_s^* /\| \beta_s^*\|$  (Theorem 7).
5 Estimator
In this section, we present a detailed construction of the estimators that attain the minimax error as delineated in Theorem 1. Existing theoretical results, such as those found in Agarwal et al. [2], Chzhen et al. [7, 8], are incapable of addressing unbounded non-sensitive features  $X$  or unbounded outcomes  $Y$ , rendering them inapplicable to our problem. Consequently, we have developed a novel estimator accompanied by rigorous analytical techniques.
Estimator construction. In constructing the optimal regressor for model Eq (2), we leverage the results from Lemma 1 and employ a plugin estimator. The method involves estimating the components of terms in Eq (5) and substituting the obtained estimates into the same equation. Concretely, we derive estimators  $\widehat{\|\beta\|}, \tilde{\beta}_s, \hat{\mu}_s, \hat{p}_s, \hat{\beta}_s'$ , and  $\hat{\mu}_s'$ , with the following correspondence:
$$
\underbrace {\| \beta_ {s} ^ {*} \|} _ {\| \widehat {\beta_ {\cdot}} \|} \Bigg \langle \underbrace {\beta_ {s} ^ {*}} _ {\tilde {\beta} _ {s}}, x - \underbrace {\mu_ {s}} _ {\tilde {\mu} _ {s}} \Bigg \rangle + \sum_ {s ^ {\prime} \in [ M ]} \underbrace {p _ {s ^ {\prime}}} _ {\hat {p} _ {s ^ {\prime}}} \Bigg \langle \underbrace {\beta_ {s ^ {\prime}} ^ {*}} _ {\hat {\beta} _ {s ^ {\prime}} ^ {\prime}}, \underbrace {\mu_ {s ^ {\prime}}} _ {\tilde {\mu} _ {s ^ {\prime}}} \Bigg \rangle .
$$
Table 2: Estimator construction. In this table,  $\hat{\beta}_{b,s}$  and  $\hat{\beta}_{b,s}^{\prime}$  denote OLS estimands obtained from subsets  $D_{b,s}$  and  $D_{b,s}^{\prime}$ , respectively. "Sample" refers to the subset utilized for estimand calculation, while "Definition" provides the corresponding estimator's definition. "Sample" in  $\|\widehat{\beta}\|$  is left empty, as it is derived from  $\hat{p}_s$  and  $\|\widehat{\beta_s}\|$ .
Estimator	Sample	Definition
p̂s	n.	p̂s = ns/n
||βs||	D1,s	||βs|| = ||β1,s|| if ns > 18d, and ||βs|| = 0 otherwise
||β..||	-	||β..| = ∑s∈[M] p̂s||βs||
βs	D2,s	βs = β2,s/||β2,s|| if ns > 18d, and βs = 0 otherwise
μs	D3,s	μs = 1/n3,s ∑i=1n3,s X3,s,i
β's'	D1,s'	β's' = β1,s' if ns > 12d, and β's' = 0 otherwise
μ's'	D2,s'	μ's' = 1/n2,s ∑i=1n2,s X2,s,i
For technical reasons, we partition the sample to calculate each estimand. Each estimator is assigned a corresponding subset, as shown in Table 2. Under specific conditions,  $n_s > 18d$  or  $n_s > 12d$ , estimators may exhibit altered behavior, primarily as technical considerations for subsequent analyses. We detail the partitioning process as follows. First, we create a histogram of the sensitive attribute  $S_i$ , denoted as  $n_i = (n_1, \dots, n_M)$ , with  $n_s = |\{i \in [n] : S_i = s\}|$ . Simultaneously, we form group-wise samples  $D_s = \{(X_i, Y_i) : i \in [n], S_i = s\}$ . For each  $s \in [M]$ , we partition  $D_s$  into  $D_{1,s}$ ,  $D_{2,s}$ , and  $D_{3,s}$ , ensuring  $|D_{b,s}| := n_{b,s} \geq \lfloor n_s / 3 \rfloor$  for  $b \in [3]$ . Using  $n_i, D_{1,s}, D_{2,s}$ , and  $D_{3,s}$ , we estimate  $\hat{p}_s, \widehat{\|\beta_s\|}, \tilde{\beta}_s$ , and  $\hat{\mu}_s$ , respectively. The combination of  $\hat{p}_s$  and  $\widehat{\|\beta_s\|}$  yields  $\widehat{\|\beta\|}$ . Furthermore, we generate a duplicate of  $D_s$ , denoted as  $D_s'$ , and partition it into  $D_{1,s}'$  and  $D_{2,s}'$ , satisfying  $|D_{b,s}'| := n_{b,s}' \geq \lfloor n_s / 2 \rfloor$  for  $b \in [2]$ . We then use  $D_{1,s}'$  and  $D_{2,s}'$  to estimate  $\hat{\beta}_s'$  and  $\hat{\mu}_s'$ . Precise definitions of the estimator construction and subset partitioning can be found in the appendices.
Incorporating the derived estimators, we construct the final regressor as:
$$
\hat {f} _ {n} (x, s) = \widehat {\| \beta . \|} \left\langle \tilde {\beta} _ {s}, x - \hat {\mu} _ {s} \right\rangle + \sum_ {s ^ {\prime} \in [ M ]} \hat {p} _ {s ^ {\prime}} \left\langle \hat {\beta} _ {s} ^ {\prime}, \hat {\mu} _ {s} ^ {\prime} \right\rangle . \tag {6}
$$
6 Upper Bound Analyses
In this section, we demonstrate the achievability of the upper bound presented in Theorem 1 utilizing the estimator delineated in Section 5. Initially, we conduct an analysis of the estimator's fairness guarantee, subsequently progressing to an examination of the estimator's mean squared deviation.
6.1 Analysis of Fairness
For our fairness guarantee on  $\hat{f}_n$ , we demonstrate the following theorem.
Theorem 2. If  $n \geq 48\ln (M / \delta) / \min_s p_s$ , we have for  $\delta \in (0,1)$
$$
\mathbb {P} \left\{\max  _ {s, s ^ {\prime} \in [ M ]} W _ {2} \left(\nu_ {\hat {f} _ {n} | s}, \nu_ {\hat {f} _ {n} | s ^ {\prime}}\right) > 4 B \sigma_ {X} \sigma_ {X} \sqrt {\frac {4 8 \ln (M / \delta)}{\min  _ {s ^ {\prime \prime} \in [ M ]} n p _ {s ^ {\prime \prime}}}} \right\} \leq \delta .
$$
By proving Theorem 2, we can immediately confirm that the estimator adheres to  $(\alpha, \delta)$ -fairness consistency with  $\alpha \in (0, 1/2]$ .
6.2 Analysis of Estimation Error
In this subsection, we derive an upper bound for the estimation error presented in Theorem 1, focusing on the estimator introduced in Section 5. To derive the upper bound in Theorem 1, we begin by decomposing the mean squared deviation of the estimator in Eq (6) as follows:
Theorem 3. For the estimator defined in Eq (6), the mean square deviation from  $f_{\mathrm{DP}}^{*}$  is bounded above by
$$
\begin{array}{l} \sum_ {s \in [ M ]} p _ {s} \mathbf {E} \left[ \right.\left( \right.\mathbf {E} \left[ \right. \widehat {\| \beta . \|} ^ {2} \left. \right\rvert n. \left. \right] ^ {1 / 2} \mathbf {E} \left[ \right.\left\langle \tilde {\beta} _ {s}, \mu_ {s} - \hat {\mu} _ {s} \right\rangle^ {2} \left. \right\rvert n. \left. \right] ^ {1 / 2} + \sigma_ {X} \mathbf {E} \left[ \right.\left(\widehat {\| \beta . \|} - \overline {{\| \beta_ {:} ^ {*} \|}}\right) ^ {2} \left. \right\rvert n. \left. \right] ^ {1 / 2} + \\ \sigma_ {X} \overline {{\| \beta_ {\cdot} ^ {*} \|}} \mathbf {E} \left[ \left\| \tilde {\beta} _ {s} - \beta_ {s} ^ {*} / \| \beta_ {s} ^ {*} \| \right\| ^ {2} | n. \right] ^ {1 / 2} + \mathbf {E} \left[ \left(\sum_ {s ^ {\prime} \in [ M ]} \hat {p} _ {s ^ {\prime}} \left\langle \hat {\beta} _ {s ^ {\prime}} ^ {\prime} - \beta_ {s ^ {\prime}} ^ {*}, \hat {\mu} _ {s ^ {\prime}} ^ {\prime} \right\rangle\right) ^ {2} | n. \right] ^ {1 / 2} + \\ \mathbf {E} \left[ \left(\sum_ {s ^ {\prime} \in [ M ]} \hat {p} _ {s ^ {\prime}} \langle \beta_ {s ^ {\prime}} ^ {*}, \hat {\mu} _ {s ^ {\prime}} ^ {\prime} - \mu_ {s ^ {\prime}} \rangle\right) ^ {2} \mid n. \right] ^ {1 / 2} + \left| \sum_ {s ^ {\prime} \in [ M ]} (\hat {p} _ {s ^ {\prime}} - p _ {s ^ {\prime}}) \langle \beta_ {s ^ {\prime}} ^ {*}, \mu_ {s ^ {\prime}} \rangle \right| ^ {2} \bigg ]. \tag {7} \\ \end{array}
$$
In Eq (7), the terms correspond to the estimation errors of  $\hat{\mu}_s$ ,  $\|\widehat{\beta}\|, \tilde{\beta}_s, \hat{\beta}_s', \hat{\mu}_s'$ , and  $\hat{p}_s$ , respectively. Standard techniques for the OLS estimator and empirical average yield upper bounds for the first, fourth, fifth, and sixth terms. Nevertheless, the second and third terms in Eq (7) involve non-linear transformations of the OLS estimator (i.e., taking the norm or dividing by the norm), complicating their error analysis. This section's primary technical contributions involve establishing tight upper bounds for the second and third terms in Eq (7).
Estimation error of norm and direction of  $\beta_{s}^{*}$ . Consider  $X_{1},\ldots ,X_{n}\stackrel {\mathrm{id}}{\sim}N(\mu ,\sigma_{X}^{2}I),\beta^{*}\in \mathbb{R}^{d}$  with  $\| \beta^{*}\| \leq B$  for some  $B > 0$ , and  $\xi_1,\dots,\xi_n\stackrel {\mathrm{id}}{\sim}N(0,\sigma_\xi^2)$ . Define  $Y_{i} = \langle \beta^{*},X_{i}\rangle +\xi_{i}$ . The OLS estimator of  $\beta^{*}$  is given by  $\hat{\beta} = (\frac{1}{n} X^{\top}X)^{-1}(\frac{1}{n} X^{\top}Y)$ , where  $X = (X_{1}\dots X_{n})^{\top}$  and  $Y = (Y_{1}\dots Y_{n})^{\top}$ . The direction estimator is  $\hat{\beta} /\| \hat{\beta}\|$ , while the norm estimator is  $\| \hat{\beta}\|$ .
We present the estimation errors for direction and norm in Theorems 4 and 5:
Theorem 4. For  $n > 6d$ , we have
$$
\mathbf {E} \left[ \left\| \frac {\hat {\beta}}{\| \hat {\beta} \|} - \frac {\beta^ {*}}{\| \beta^ {*} \|} \right\| ^ {2} \right] \leq \frac {8 4 e ^ {1 0} \sigma_ {\xi} ^ {2} d}{\sigma_ {X} ^ {2} \| \beta^ {*} \| ^ {2} n} \left(1 + \frac {6}{n - 6}\right).
$$
Theorem 5. For  $n > 6d$ , we have
$$
\mathbf {E} \left[ \left(\| \hat {\beta} \| - \| \beta^ {*} \|\right) ^ {2} \right] \leq \frac {2 1 e ^ {1 0} \sigma_ {\xi} ^ {2} d}{\sigma_ {X} ^ {2} n} \left(1 + \frac {6}{n - 6}\right).
$$
The direction's estimation error (Theorem 4) is  $O\left(\sigma_{\xi}^{2} d / \sigma_{X}^{2} \| \beta^{*} \|^{2} n\right)$ , while the norm's estimation error (Theorem 5) is  $O\left(B^{2} \sigma_{\xi}^{2} d / \sigma_{X}^{2} n\right)$ . Integrating Theorems 3 to 5 yields the  $\sigma_{\xi}^{2} B^{2} dM / n$  term in the upper bound in Theorem 1. The remaining part,  $U \sigma_{\xi}^{2} / n$ , arises from the estimation error of  $\hat{\beta}_{s}'$  (the third term in Eq (7)), dominating other terms in Eq (7).
7 Lower Bound Analyses
In this section, we provide a proof sketch for the lower bound, outlined in Theorem 1. To facilitate a clear and concise presentation of the proof sketch, we introduce several notations. Let  $\theta$  denote the tuple of distribution parameters  $(\beta ,\mu ,)$ , and let  $\Theta$  represent the set of all such parameters, defined as  $\Theta = \mathcal{B}\times \mathcal{M}$ . We use  $\mathbb{P}_{\theta}$  and  $\mathbf{E}_{\theta}$  to denote the probability and expectation operators, respectively, given  $X\sim N(\mu_S,\sigma_X^2 I)$  and  $Y = (\beta_{S},X) + \xi$ , where  $\xi \sim N(0,\sigma_{\xi}^{2})$ . We adopt the shorthand  $\mathcal{E}(f;\theta) = \mathcal{E}(f;\beta ,\mu ,)$  for  $\theta = (\beta ,\mu ,)$ . Moreover, we define  $f_{\theta} = \arg \min_{f\in \mathcal{F}_{\mathrm{DP}}}R(f;\beta ,\mu ,)$  for  $\theta = (\beta ,\mu ,)$ . For two probability distributions  $\pi$  and  $\pi^\prime$ , the Kullback-Leibler (KL) divergence is denoted as  $D_{\mathrm{KL}}(\pi ,\pi ') = \int \ln (\frac{d\pi}{d\pi'} (z))\pi (dz)$ . Finally, we denote the set of all  $L^2$  integrable functions  $f:\mathbb{R}^d\times [M]\to \mathbb{R}$  as  $\mathcal{L}^2$ .
By utilizing Fano's inequality, we establish a lower bound for the minimax error as presented in Theorem 1. Due to the invariance of the distribution of  $S_{1}, \ldots, S_{n}$  under parameter alterations  $\theta$ ,
Fano's inequality can be applied after conditioning on  $S_{1}, \ldots, S_{n}$ , or equivalently,  $n$ . Consequently, we derive the following theorem:
Theorem 6. Let  $\hat{\Theta} \subseteq \Theta$  be a finite set of the parameters such that there exists  $\epsilon > 0$  such that for any  $\theta, \theta' \in \hat{\Theta}$ ,  $\inf_{f} \mathcal{E}(f; \theta) \vee \mathcal{E}(f; \theta') \geq \epsilon$ , where  $\hat{\Theta}$  and  $\epsilon$  is possibly dependent on  $n$ . Let  $|\hat{\Theta}| = K$ . Then, for arbitrary  $\alpha > 0$  and  $\delta \in (0,1)$ , we have
$$
\mathcal {E} _ {n} (\alpha , \delta) \geq \mathbf {E} \left[ \epsilon \left(1 - \frac {\inf  _ {\pi} \frac {1}{K} \sum_ {\theta \in \hat {\Theta}} D _ {\mathrm {K L}} \left(\pi_ {\theta | n .} , \pi\right) + \ln (2)}{\ln (K)}\right) \right],
$$
where  $\pi_{\theta |n}$  denotes the distribution of  $D_{n}$  conditioned on  $n$ . with parameter  $\theta$ , and the expectation is taken over  $n$ .
As demonstrated in Theorem 6, the lower bound for the minimax error can be obtained by constructing  $\hat{\Theta}$  such that: 1)  $\inf_f\mathcal{E}(f;\theta)\vee \mathcal{E}(f;\theta ')\geq \epsilon$  for any  $\theta ,\theta^{\prime}\in \hat{\Theta}$ , and 2)  $\inf_{\pi}\frac{1}{K}\sum_{\theta \in \hat{\Theta}}D_{\mathrm{KL}}\bigl (\pi_{\theta |n},\pi \bigr)\leq \ln (K / 4) / 2$ . With the construction of such a  $\hat{\Theta}$ , a lower bound of  $\mathbf{E}[\frac{\epsilon}{2} ]$  is attained.
We present a theorem that establishes a tight lower bound on  $\inf_{f}\mathcal{E}(f;\theta)\lor \mathcal{E}(f;\theta^{\prime})$
Theorem 7. Let  $\theta$  and  $\theta'$  be the parameters of the distributions such that  $\frac{1}{2\sigma_X^2} \| \mu_s - \mu_s' \|^2 \coloneqq d_s < 1$  for all  $s \in [M]$ . Then, we have
$$
\inf _ {f \in \mathcal {L} ^ {2}} \mathcal {E} (f; \theta) \vee \mathcal {E} (f; \theta^ {\prime}) \geq \sum_ {s \in [ M ]} p _ {s} \frac {\sigma_ {X} ^ {2} e ^ {- d _ {s}}}{4} \left\| \frac {\overline {{\| \beta . \|}} \beta_ {s}}{\| \beta_ {s} \|} - \frac {\overline {{\| \beta_ {s} ^ {\prime} \|}} \beta_ {s} ^ {\prime}}{\| \beta_ {s} ^ {\prime} \|} \right\| ^ {2} \left(1 + \frac {d _ {s}}{2}\right) ^ {1 + \frac {d}{2}}.
$$
The term  $\| \overline{\|\beta_{s}\|}\beta_{s} / \|\beta_{s}\| - \overline{\|\beta_{s}'\|}\beta_{s}' / \|\beta_{s}'\|$  characterizes the lower bound, which is different from the characteristic term in standard linear regression,  $\|\beta_{s} - \beta_{s}'\|^2$ .
We next present the construction of  $\hat{\Theta}$ . We construct  $\hat{\Theta}$  such that each of its elements corresponds to an index from the set  $\mathcal{V} = \{-1,1\}^{M\times (d - 1)}$ , denoted by  $\theta_v = \{\beta_{v,},\mu_{v,}\} \in \hat{\Theta}$ , where  $\beta_{v,s}$  is controlled such that its norm is equivalent to a specified value  $B_{s}$ , i.e.,  $\| \beta_{v,s}\| = B_s$ . This construction ensures that  $\hat{\Theta}\subset \mathcal{B}\times \mathcal{M}$ . Given positive values  $\epsilon_1,\dots,\epsilon_M$  and  $B_{1},\ldots,B_{M}$ , we construct  $\hat{\Theta}$  as follows:
$$
\mu_ {v, s} = 0, \| \beta_ {v, s} \| = B _ {s}, \frac {\beta_ {v , s , 1}}{\| \beta_ {v , s} \|} = \sqrt {1 - \epsilon_ {s} ^ {2}}, \text {a n d} \frac {\beta_ {v , s , i}}{\| \beta_ {v , s} \|} = v _ {s, i - 1} \frac {\epsilon_ {s}}{\sqrt {d - 1}} \text {f o r} i = 2, \dots , d. \tag {8}
$$
We demonstrate the following properties for  $\hat{\Theta}$  defined in Eq (8).
Theorem 8. Given  $\epsilon_1, \dots, \epsilon_M > 0$  and  $B_1, \dots, B_M > 0$ , let  $\hat{\Theta} \subset \Theta$  represent the set of parameters defined in Eq (8). Let  $\pi_{\theta|n}$  be the distribution of the sample  $D_n$  conditioned on  $n$ . With the distribution parameter  $\theta$ . Then, we have 1) for any  $v, v' \in \mathcal{V}$ ,
$$
\inf  _ {f \in \mathcal {L} ^ {2}} \mathcal {E} (f; \theta_ {v}) \vee \mathcal {E} (f; \theta_ {v ^ {\prime}}) \geq \sum_ {s \in [ M ]} p _ {s} \left(\sum_ {s ^ {\prime} \in [ M ]} p _ {s ^ {\prime}} B _ {s ^ {\prime}}\right) ^ {2} \frac {\sigma_ {X} ^ {2} \epsilon_ {s} ^ {2}}{d - 1} d _ {H} (v _ {s}, v ^ {\prime} s),
$$
and 2) for  $v, v' \in \mathcal{V}$ ,
$$
D _ {\mathrm {K L}} \big (\pi_ {\theta_ {v} | n.}, \pi_ {\theta_ {v ^ {\prime}} | n.} \big) = \sum_ {s \in [ M ]} \frac {2 \sigma_ {X} ^ {2} B _ {s} ^ {2} n _ {s} \epsilon_ {s} ^ {2}}{\sigma_ {\xi} ^ {2} (d - 1)} d _ {H} (v _ {s}, v _ {s} ^ {\prime}).
$$
By integrating Theorems 6 to 8 and employing the renowned Varshamov-Gilbert bound, we derive the lower bound in Theorem 1.
This paper investigates a regression problem with  $(\alpha, \delta)$ -fairness consistency as a fairness constraint. Specifically, we demonstrate that, under the constraint of  $(\alpha, \delta)$ -fairness, the minimax optimal error
scales as  $\sigma_{\xi}^{2}B^{2}dM / n$  up to a constant factor, when  $\alpha \in (0,1 / 2]$ . Additionally, we provide the fair regressor that achieves this optimal error.
Potential negative societal impacts. Our study aims to mitigate the negative impact of regression models on social groups, rather than to cause harm. However, our results are only valid for linear models, as defined in Eq (2). Misapplication of our findings to other models may result in discriminatory treatment, which should be avoided.
This work was partly supported by JSPS KAKENHI Grant Numbers JP23K13011 and JP23H00483.
[1] Alekh Agarwal, Alina Beygelzimer, Miroslav Dudík, John Langford, and Hanna Wallach. A Reductions Approach to Fair Classification. In the 35th International Conference on Machine Learning, pages 60–69, 2018.
[2] Alekh Agarwal, Miroslav Dudík, and Zhiwei Steven Wu. Fair Regression: Quantitative Definitions and Reduction-based Algorithms. In 36th International Conference on Machine Learning, ICML 2019, volume 2019-June, pages 120-129. International Machine Learning Society (IMLS), 2019.
[3] Julia Angwin, Jeff Larson, Surya Mattu, and Lauren Kirchner. Machine Bias, 2016. URL https://www propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing.
[4] Toon Calders and Sicco Verwer. Three naive Bayes approaches for discrimination-free classification. Data Mining and Knowledge Discovery, 21(2):277-292, 2010.
[5] M. T. Chao and W. E. Strawderman. Negative Moments of Positive Random Variables. Journal of the American Statistical Association, 67(338):429, 6 1972. ISSN 01621459. doi: 10.2307/2284399.
[6] Evgenii Chzhen and Nicolas Schreuder. A minimax framework for quantifying risk-fairness trade-off in regression. The Annals of Statistics, 50(4):2416-2442, 8 2022. ISSN 0090-5364. doi: 10.1214/22-AOS2198.
[7] Evgenii Chzhen, Christophe Denis, Mohamed Hebiri, and Massimiliano Pontil. Fair Regression with Wasserstein Barycenters. In Advances in Neural Information Processing Systems, volume 33, pages 7321-7331. Curran Associates, Inc., 2020.
[8] Evgenii Chzhen, Christophe Denis, Mohamed Hebiri, and Massimiliano Pontil. Fair Regression via Plug-In Estimator and Recalibration. In Advances in Neural Information Processing Systems, volume 33, pages 19137-19148. Curran Associates, Inc., 2020.
[9] Kade Crockford. How is Face Recognition Surveillance Technology Racist?, 2020. URL https://www.aclu.org/news/privacy-technology/how-is-face-recognition-surveillance-technology-racist/.
[10] Jeffrey Dastin. Amazon scraps secret AI recruiting tool that showed bias against women, 2018. URL https://www.reuters.com/article/us-amazon-com-jobs-automation-insight/amazon-scaps-secret-ai-recruiting-tool-that-showed-bias-against-women-idUSKCN1MK08G.
[11] Moritz Hardt, Eric Price, and Nathan Srebro. Equality of Opportunity in Supervised Learning. In D D Lee, M Sugiyama, U V Luxburg, I Guyon, and R Garnett, editors, Advances in Neural Information Processing Systems 29, pages 3315-3323, Barcelona, Spain, 12 2016. Curran Associates, Inc.
Figure 1: Sample splitting for constructing estimators.
A Comparison of Existing and Our Unfairness Scores
This section compares our unfairness score with existing ones. Recall that our unfairness score is defined as the maximum Wasserstein distance between any two distributions  $\nu_{f|s}$  and  $\nu_{f|s'}$  over all pairs of groups  $s$  and  $s'$ , as follows:
$$
U (f) = \max  _ {s, s ^ {\prime} \in [ M ]} W _ {2} \left(\nu_ {f _ {n} | s}, \nu_ {f | s ^ {\prime}}\right).
$$
In contrast, Agarwal et al. [2], Chzhen et al. [7, 8] use the Kolmogorov distance  $D_{\mathrm{Kol}}$  to measure unfairness, which is defined as:
$$
U _ {\mathrm {K o l}} (f) = \max  _ {s, s ^ {\prime} \in [ M ]} D _ {\mathrm {K o l}} (\nu_ {f | s}, \nu_ {f | s ^ {\prime}}).
$$
The difference between our score and  $U_{\mathrm{Kol}}(f)$  is solely the choice of distance metric. Our score utilizes the Wasserstein distance, while  $U_{\mathrm{Kol}}(f)$  uses the Kolmogorov distance. This difference arises mainly from technical reasons.
In addition, Chzhen and Schreuder [6] proposed another unfairness score, denoted by  $U_{\mathrm{Avg}W_2}(f)$ , which is defined as the average of the Wasserstein distance, as follows:
$$
U _ {\mathrm {A v g} W _ {2}} (f) = \inf  _ {\nu} \sum_ {s \in [ M ]} p _ {s} W _ {2} \left(\nu_ {f | s}, \nu\right).
$$
Here, the score places more emphasis on the major groups, as reflected by the weight of  $p_s$ . This may not be desirable if the unfairness is more prevalent in the minority groups, which may be common in real-world scenarios.
B Estimator Details
This section describes the construction of our optimal estimator in detail. Recall that our estimator is a plugin estimator in which we first estimate the parts of the terms in Eq (5) and then substitute them into Eq (5). Specifically, we construct estimators for  $\| \widehat{\beta}.\|$ ,  $\tilde{\beta}_s$ ,  $\hat{p}_{s'}$ ,  $\hat{\beta}_{s'}'$ , and  $\hat{\mu}_{s'}'$ , where they correspond to the terms in Eq (5) as follows:
$$
\underbrace {\overline {{\| \beta_ {s} ^ {*} \|}} \Bigg \langle \underbrace {\beta_ {s} ^ {*}} _ {\tilde {\beta} _ {s}}, x - \underbrace {\mu_ {s}} _ {\tilde {\mu} _ {s}} \Bigg \rangle} _ {\widehat {\| \beta_ {\cdot} \|}} + \sum_ {s ^ {\prime} \in [ M ]} \underbrace {p _ {s ^ {\prime}}} _ {\hat {p} _ {s ^ {\prime}}} \Bigg \langle \underbrace {\beta_ {s ^ {\prime}} ^ {*}} _ {\hat {\beta} _ {s ^ {\prime}} ^ {\prime}}, \underbrace {\mu_ {s ^ {\prime}}} _ {\tilde {\mu} _ {s ^ {\prime}} ^ {\prime}} \Bigg \rangle .
$$
Algorithm 1: Algorithm of the proposed optimal estimator.
Input :The sample  $D_{n} = \{(X_{i},S_{i},Y_{i})\}_{i = 1}^{n}$    
Output: The regressor  $\hat{f}_n$    
for  $s\gets 1$  to  $M$  do Calculate  $n_s$  and construct the group-wise sample  $D_{s}$  and its duplicate  $D_s^\prime$  . Partition  $D_{s}$  into equal-sized subsets:  $D_{1,s}$ $D_{2,s}$  ,and  $D_{3,s}$  . Compute the estimands  $\hat{p}_s$  from  $n.$ $\| \widehat{\beta_s}\|$  from  $D_{1,s}$ $\tilde{\beta}_s$  from  $D_{2,s}$  ,and  $\hat{\mu}_s$  from  $D_{3,s}$  . Partition  $D_s^\prime$  equally into  $D_{1,s}^{\prime}$  and  $D_{2,s}^{\prime}$  . Compute the estimands  $\hat{\beta}_s^\prime$  from  $D_{1,s}^{\prime}$  and  $\hat{\mu}_s^\prime$  from  $D_{2,s}^{\prime}$  .   
end Calculate  $\hat{f}_n$  using Eq (6);   
return  $\hat{f}_n$