SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\mathbb {P} \left[ \widehat {\boldsymbol {g}} = \mathbf {0} \right] \leq \mathbb {P} \left[ \| \boldsymbol {g} - \widehat {\boldsymbol {g}} \| _ {2} \geq \| \boldsymbol {g} \| _ {2} \right]
$$
$$
\leq \mathbb {P} \left[ \| \boldsymbol {\Pi} _ {\boldsymbol {P} _ {m}} - \boldsymbol {\Pi} _ {\boldsymbol {W} _ {T}} \| _ {2} \geq \frac {g _ {\operatorname* {m i n}}}{2 f _ {\operatorname* {m a x}}} \right] + \mathbb {P} \left[ \| \boldsymbol {f} - \hat {\boldsymbol {f}} \| _ {2} \geq \frac {g _ {\operatorname* {m i n}}}{2} \right].
$$
To have  $\| \Pi_{P_m} - \Pi_{W_T}\| _2 < \frac{g_{\min}}{2f_{\max}}$  with probability at least  $1 - \frac{\delta}{8}$ , it suffices to have a sample complexity in Eqn. (22) and (23). Also, to have  $\left\| \pmb {f} - \widehat{\pmb{f}}\right\| _2 < \frac{g_{\min}}{2}$  with probability at least  $1 - \frac{\delta}{8}$ , it suffices to have a sample complexity in Eqn. (24).
Combining the bounds in the Case 1 with a union bound, we obtain a full sample complexity for Algorithm 3 as follows: a total of  $N_{1}^{(1)} = Tb$  samples are sufficient for ensuring  $(\varepsilon_{\mathrm{f}}, \varepsilon_{\mathrm{o}})$ -PAFO-learnability with probability at least  $1 - \delta$ , where
$$
\begin{array}{l} N _ {1} ^ {(1)} = \Omega \left(\frac {K _ {m , \nu}}{p _ {\mathrm {m i n}}} \left\{\frac {\mathcal {V}}{\Delta_ {m , \nu} ^ {3}} \left(\frac {d m}{\delta^ {2}} \log \frac {m}{\delta} + \frac {f _ {\mathrm {m a x}} ^ {2}}{\eta_ {k} ^ {2} g _ {\mathrm {m i n}} ^ {2}} \log \frac {d}{\delta}\right) + \frac {\mathcal {M} ^ {2}}{\mathcal {V} \Delta_ {m , \nu}} \log \frac {d}{\delta} \right\} \log \left(\frac {d}{\eta_ {k} \delta} \frac {f _ {\mathrm {m a x}}}{g _ {\mathrm {m i n}}}\right)\right). \\ \left. + \frac {\sigma^ {2}}{p _ {\operatorname* {m i n}} g _ {\operatorname* {m i n}} ^ {2} \eta_ {k} ^ {2}} \log \frac {d}{\delta}\right) \\ = \tilde {\Omega} \left(\frac {K _ {m , \nu}}{p _ {\operatorname* {m i n}}} \left\{\frac {\mathcal {V}}{\Delta_ {m , \nu} ^ {3}} \left(\frac {d m}{\delta^ {2}} + \frac {f _ {\operatorname* {m a x}} ^ {2}}{\eta_ {k} ^ {2} g _ {\operatorname* {m i n}} ^ {2}}\right) + \frac {\mathcal {M} ^ {2}}{\mathcal {V} \Delta_ {m , \nu}} \right\} + \frac {\sigma^ {2}}{p _ {\operatorname* {m i n}} g _ {\operatorname* {m i n}} ^ {2} \eta_ {k} ^ {2}}\right). \tag {25} \\ \end{array}
$$
Case 2. Assume  $\pmb{g} = \mathbf{0}$ . In this case,  $\pmb{U} = \pmb{P}_{m}$ . We want to upper-bound the follow probability to be  $< \delta / 2$ :
$$
\begin{array}{l} \mathbb {P} \left[ \left\| \boldsymbol {\Pi} _ {U} \boldsymbol {\Pi} _ {\widehat {U}} ^ {\perp} \right\| _ {2} > \eta_ {k} \right] \\ = \mathbb {P} \left[ \left(\widehat {\boldsymbol {g}} = \mathbf {0} \text {a n d} \left\| \Pi_ {U} - \Pi_ {\widehat {U}} \right\| _ {2} > \eta_ {k}\right) \text {o r} \left(\widehat {\boldsymbol {g}} \neq \mathbf {0} \text {a n d} \left\| \Pi_ {U} \Pi_ {\widehat {U}} ^ {\perp} \right\| _ {2} > \eta_ {k}\right) \right] \\ \leq \mathbb {P} \left[\left\| \boldsymbol {\Pi} _ {\boldsymbol {P} _ {m}} - \boldsymbol {\Pi} _ {\boldsymbol {W} _ {T}} \right\| _ {2} > \eta_ {k} | \widehat {\boldsymbol {g}} = \mathbf {0} \right] + \mathbb {P} \left[ \right.\left\| \boldsymbol {\Pi} _ {\boldsymbol {P} _ {m}} \boldsymbol {\Pi} _ {[ \boldsymbol {W} _ {T} | \widehat {\boldsymbol {h}} ]} ^ {\perp} \right\| _ {2} > \eta_ {k} \left. \right\rvert   \widehat {\boldsymbol {g}} \neq \mathbf {0} \left. \right]. \\ \end{array}
$$
The first probability term is bounded by  $\frac{\delta}{4}$  given that a sufficient number of samples just like Eqn. (19) and (20). To bound the second one, note that  $\widehat{h}$  is orthogonal to every column of  $W_{T}$  (i.e.,  $\Pi_{W_T}\widehat{h} = 0$ ). Thus,
$$
\begin{array}{l} \left\| \Pi_ {P _ {m}} \Pi_ {[ \boldsymbol {W} _ {T} | \hat {\boldsymbol {h}}} ^ {\perp} \right\| _ {2} = \left\| \Pi_ {P _ {m}} - \Pi_ {P _ {m}} \left(\Pi_ {\boldsymbol {W} _ {T}} + \hat {\boldsymbol {h}} \hat {\boldsymbol {h}} ^ {\intercal}\right) \right\| _ {2} \\ = \left\| \boldsymbol {\Pi} _ {\boldsymbol {P} _ {m}} \boldsymbol {\Pi} _ {\boldsymbol {W} _ {T}} ^ {\perp} - \left(\boldsymbol {\Pi} _ {\boldsymbol {P} _ {m}} - \boldsymbol {\Pi} _ {\boldsymbol {W} _ {T}}\right) \widehat {\boldsymbol {h}} \widehat {\boldsymbol {h}} ^ {\intercal} \right\| _ {2} \\ \leq 2 \left\| \Pi_ {P _ {m}} - \Pi_ {W _ {T}} \right\| _ {2}, \\ \end{array}
$$
which can be bounded by  $< \eta_{k}$  with probability at least  $1 - \frac{\delta}{4}$  given that a sufficient number of samples just like Eqn. (19) and (20) again. Therefore, a total of  $N_{1}^{(2)} = Tb$  samples are sufficient for ensuring  $(\varepsilon_{\mathrm{f}}, \varepsilon_{\mathrm{o}})$ -PAFO-learnability with probability at least  $1 - \delta$ , where
$$
\begin{array}{l} N _ {1} ^ {(1)} = \Omega \left(\frac {K _ {m , \nu}}{p _ {\operatorname* {m i n}}} \left\{\frac {\mathcal {V}}{\Delta_ {m , \nu} ^ {3}} \left(\frac {d m}{\delta^ {2}} \log \frac {m}{\delta} + \frac {1}{\eta_ {k} ^ {2}} \log \frac {d}{\delta}\right) + \frac {\mathcal {M} ^ {2}}{\mathcal {V} \Delta_ {m , \nu}} \log \frac {d}{\delta} \right\} \log \left(\frac {d}{\eta_ {k} \delta}\right)\right) \\ = \tilde {\Omega} \left(\frac {K _ {m , \nu}}{p _ {\operatorname* {m i n}}} \left\{\frac {\mathcal {V}}{\Delta_ {m , \nu} ^ {3}} \left(\frac {d m}{\delta^ {2}} + \frac {1}{\eta_ {k} ^ {2}}\right) + \frac {\mathcal {M} ^ {2}}{\mathcal {V} \Delta_ {m , \nu}} \right\}\right). \tag {26} \\ \end{array}
$$
Combining Case 1 (Eqn. (25)) and Case 2 (Eqn. (26)) as  $N_{1} = \mathbb{1}[g \neq \mathbf{0}]N_{1}^{(1)} + \mathbb{1}[g = \mathbf{0}]N_{1}^{(2)}$ , we have our desired statement.
Lastly, we provide the missing proof for our lemmas:
Proof of Lemma 7. Let
$$
\boldsymbol {A} = \left[ \begin{array}{c} \boldsymbol {A} _ {1} \\ k \end{array} \right] \left| \begin{array}{c} \boldsymbol {A} _ {2} \\ d - k \end{array} \right] \quad \text {a n d} \quad \boldsymbol {B} = \left[ \begin{array}{c} \boldsymbol {B} _ {1} \\ \ell \end{array} \right] \left| \begin{array}{c} \boldsymbol {B} _ {2} \\ d - \ell \end{array} \right]
$$
be  $d \times d$  orthogonal matrices. We first show that  $\sin (\operatorname{col}(\mathbf{A}_1), \operatorname{col}(\mathbf{B}_1)) = \| \mathbf{A}_2^\top \mathbf{B}_1 \|_2$ . It can be proven as
$$
\left\| \Pi_ {\boldsymbol {A} _ {1}} ^ {\perp} \boldsymbol {B} _ {1} \right\| _ {2} = \| \boldsymbol {A} _ {2} \boldsymbol {A} _ {2} ^ {\intercal} \boldsymbol {B} _ {1} \| _ {2} \leq \| \boldsymbol {A} _ {2} ^ {\intercal} \boldsymbol {B} _ {1} \| _ {2} = \| \boldsymbol {A} _ {2} ^ {\intercal} \boldsymbol {A} _ {2} \boldsymbol {A} _ {2} ^ {\intercal} \boldsymbol {B} _ {1} \| _ {2} \leq \| \boldsymbol {A} _ {2} \boldsymbol {A} _ {2} ^ {\intercal} \boldsymbol {B} _ {1} \| _ {2}.
$$
Thus, from now on, we will work with  $\| A_2^\top B_1\| _2$  instead of sin  $(\operatorname {col}(A_1),\operatorname {col}(B_1))$
The rest of the proof borrows the idea from the proof of Golub and Loan (2013, Theorem 2.5.1). Observe that
$$
\begin{array}{l} \operatorname {d i s t} \left(\operatorname {c o l} \left(\boldsymbol {A} _ {1}\right), \operatorname {c o l} \left(\boldsymbol {B} _ {1}\right)\right) = \| \boldsymbol {A} ^ {\intercal} \left(\boldsymbol {A} _ {1} \boldsymbol {A} _ {1} ^ {\intercal} - \boldsymbol {B} _ {1} \boldsymbol {B} _ {1} ^ {\intercal}\right) \boldsymbol {B} \| _ {2} \\ = \left\| \left[ \begin{array}{c c} \mathbf {0} & \mathbf {A} _ {1} ^ {\intercal} \mathbf {B} _ {2} \\ - \mathbf {A} _ {2} ^ {\intercal} \mathbf {B} _ {1} & \mathbf {0} \end{array} \right] \right\| _ {2} \\ = \max  \left\{\| B _ {2} ^ {\top} A _ {1} \| _ {2}, \| A _ {2} ^ {\top} B _ {1} \| _ {2} \right\}. \\ \end{array}
$$
Note that  $A_2^\top B_1$  and  $A_2^\top B_1$  are submatrices of a  $d \times d$  orthogonal matrix  $Q$  defined as
$$
Q \triangleq A ^ {\intercal} B = \left[ \begin{array}{c c} A _ {1} ^ {\intercal} B _ {1} & A _ {1} ^ {\intercal} B _ {2} \\ A _ {2} ^ {\intercal} B _ {1} & A _ {2} ^ {\intercal} B _ {2} \end{array} \right].
$$
Readers might notice that  $A_1^\top B_1 \in \mathbb{R}^{k \times \ell}$ . For any unit vector (in 2-norm)  $\pmb{x} \in \mathbb{R}^k$ ,
$$
1 = \left\| \boldsymbol {Q} \left[ \begin{array}{c} \boldsymbol {x} \\ \boldsymbol {0} \end{array} \right] \right\| _ {2} ^ {2} = \left\| \left[ \begin{array}{c} \boldsymbol {A} _ {1} ^ {\intercal} \boldsymbol {B} _ {1} \boldsymbol {x} \\ \boldsymbol {A} _ {2} ^ {\intercal} \boldsymbol {B} _ {1} \boldsymbol {x} \end{array} \right] \right\| _ {2} ^ {2} = \| \boldsymbol {A} _ {1} ^ {\intercal} \boldsymbol {B} _ {1} \boldsymbol {x} \| _ {2} ^ {2} + \| \boldsymbol {A} _ {2} ^ {\intercal} \boldsymbol {B} _ {1} \boldsymbol {x} \| _ {2} ^ {2}.
$$
$$
\begin{array}{l} \left\| \boldsymbol {A} _ {2} ^ {\intercal} \boldsymbol {B} _ {1} \right\| _ {2} ^ {2} = \max  _ {\boldsymbol {x} \in \mathbb {R} ^ {k}, \| \boldsymbol {x} \| _ {2} = 1} \left\| \boldsymbol {A} _ {2} ^ {\intercal} \boldsymbol {B} _ {1} \boldsymbol {x} \right\| _ {2} ^ {2} \\ = 1 - \min _ {\boldsymbol {x} \in \mathbb {R} ^ {k}, \| \boldsymbol {x} \| _ {2} = 1} \| \boldsymbol {A} _ {1} ^ {\intercal} \boldsymbol {B} _ {1} \boldsymbol {x} \| _ {2} ^ {2} \\ = \left\{ \begin{array}{l l} 1 - \sigma_ {\min } (A _ {1} ^ {\intercal} B _ {1}) ^ {2}, & \text {i f} k \geq \ell , \\ 1, & \text {i f} k <   \ell . \end{array} \right. \\ \end{array}
$$
Likewise, working with  $Q^{\intercal}$ , one can deduce that
$$
\begin{array}{l} \| \boldsymbol {B} _ {2} ^ {\intercal} \boldsymbol {A} _ {1} \| _ {2} ^ {2} = \max  _ {\boldsymbol {y} \in \mathbb {R} ^ {\ell}, \| \boldsymbol {y} \| _ {2} = 1} \| \boldsymbol {B} _ {2} ^ {\intercal} \boldsymbol {A} _ {1} \boldsymbol {y} \| _ {2} ^ {2} \\ = 1 - \min_{\boldsymbol {y}\in \mathbb{R}^{\ell},\| \boldsymbol {y}\|_{2} = 1}\| \boldsymbol{B}_{1}^{\intercal}\boldsymbol{A}_{1}\boldsymbol {y}\|_{2}^{2} \\ = \left\{ \begin{array}{l l} 1 - \sigma_ {\min } (\boldsymbol {B} _ {1} ^ {\intercal} \boldsymbol {A} _ {1}) ^ {2}, & \text {i f} k \leq \ell , \\ 1, & \text {i f} k > \ell . \end{array} \right. \\ \end{array}
$$
Therefore, our desired statement naturally follows.
Proof of Lemma 8. To prove the first claim, let  $\mathbf{u}$  and  $\mathbf{v}$  be unit  $(\ell_2 \text{ norm})$  vectors in direction of  $\mathbf{a}$  and  $\mathbf{b}$ , respectively. Then, by Lemma 7,
$$
\left\| \boldsymbol {u} \boldsymbol {u} ^ {\intercal} - \boldsymbol {v} \boldsymbol {v} ^ {\intercal} \right\| _ {2} ^ {2} = 1 - \sigma_ {\min } (\boldsymbol {u} ^ {\intercal} \boldsymbol {v}) ^ {2} = 1 - | \boldsymbol {u} ^ {\intercal} \boldsymbol {v} | ^ {2} = 1 - \cos^ {2} \theta (\boldsymbol {u}, \boldsymbol {v}) = \sin^ {2} \theta (\boldsymbol {a}, \boldsymbol {b}).
$$
Now, observe that  $\| \pmb {b}\| _2\geq \| \pmb {a}\| _2 - \epsilon \geq \| \pmb {a}\| _2 / 2$  and
$$
2 \left(\| \boldsymbol {a} \| _ {2} \| \boldsymbol {b} \| _ {2} - \langle \boldsymbol {a}, \boldsymbol {b} \rangle\right) \leq \| \boldsymbol {a} \| _ {2} ^ {2} + \| \boldsymbol {b} \| _ {2} ^ {2} - 2 \langle \boldsymbol {a}, \boldsymbol {b} \rangle = \| \boldsymbol {a} - \boldsymbol {b} \| _ {2} ^ {2} \leq \epsilon^ {2}.
$$
$$
\begin{array}{l} \sin^ {2} \theta (\boldsymbol {a}, \boldsymbol {b}) = (1 + \cos \theta (\boldsymbol {a}, \boldsymbol {b})) (1 - \cos \theta (\boldsymbol {a}, \boldsymbol {b})) \leq 2 (1 - \cos \theta (\boldsymbol {a}, \boldsymbol {b})) \\ = 2 \frac {\| \boldsymbol {a} \| _ {2} \| \boldsymbol {b} \| _ {2} - \langle \boldsymbol {a} , \boldsymbol {b} \rangle}{\| \boldsymbol {a} \| _ {2} \| \boldsymbol {b} \| _ {2}} \leq \frac {2 \epsilon^ {2}}{\| \boldsymbol {a} \| _ {2} ^ {2}}. \\ \end{array}
$$
F More Experiments
F.1 Synthetic Example
Figure 2: Synthetic Example: Vanilla NPM v.s. Mean-matched NPM v.s. FNPM (ours).
We randomly generated two different group-conditional distributions as 10-dimensional multivariate Gaussians with different mean vectors  $\pmb{\mu}_0$  and  $\pmb{\mu}_1$ , satisfying  $\pmb{\mu} = (1 - p)\pmb{\mu}_0 + p\pmb{\mu}_1 = \mathbf{0}$ , and different covariance matrices  $\Sigma_0$  and  $\Sigma_1$ . We choose the sampling probability parameter  $p$  as 0.2, which induces asymmetric sampling between two sensitive attributes. The covariance matrices are designed so that both of their eigenvalue spectra,  $\{\sigma_1,\dots ,\sigma_d\}$ , have power-law decay as many practical datasets do (Liu et al., 2015a), i.e.,  $\sigma_{j} = \Theta (j^{-\alpha})$  for some decay parameter  $\alpha \geq 1$ .
We first run and compare three different algorithms: vanilla NPM (without any fairness constraint), mean-matched NPM (with only constraint  $V^{\top}f = 0$ ), and FNPM with  $m = 3$ . To ease the visualization, we project the sampled distributions onto a 2-dimensional subspace (i.e., running 2-PCA). After running three algorithms for ten iterations and with a block size of  $b = B = 1000$ , we randomly sample 1000 data points and visualize the results of projecting the data points in Figure 2. In particular, for FNPM, we run 50 iterations for unfair subspace estimation (Algorithm 3) and run the other 50 iterations for PCA (Algorithm 4). We observe that FNPM does indeed enforce both mean-matching and (partial) covariance-matching, despite the setting being streaming and having asymmetric sampling probability.
F.2 Additional Results on CelebA Dataset
In Figures 3-5, we provide additional experimental results on the CelebA dataset (Liu et al., 2015b). We consider three sensitive attributes in the CelebA dataset: "Eyeglasses", "Mouth Slightly Open", and "Goattee". In all the figures, False is when the sensitive attribute is absent; True is otherwise.
Figure 3 presents the main results for the new attributes. The first row shows the original images, the second row shows the images after projecting them to  $\operatorname{col}(\mathbf{V}^{\star})$ , and the third row shows the images after projecting them to the estimated unfair subspace,  $\operatorname{col}(\hat{\mathbf{U}})$ . Here, both  $\hat{\mathbf{U}}$  and  $\mathbf{V}^{\star}$  are obtained from our FNPM; specifically speaking, they are obtained from Algorithm 3 and Algorithm 4, respectively. While we've used  $m = 2$  for Figure 1 in the main text, we use  $m = 5$  here.
We then perform the two ablation studies as described in the main text, one on the dimension  $m$  of unfair subspace and another on the block size  $b$  of Algorithm 3, for the additional sensitive attributes considered. In Figure 4, we vary  $m \in \{1,2,5,10\}$ : as  $m$  increases, more features of images are "erased", making the images from the two sensitive groups less distinguishable. At the same time, more semantically meaningful features are erased as well, resulting in rather "alien-like" images. In Figure 5, we vary  $b \in \{32000,8000,3200,1600\}$ : as  $b$  increases, we have a more accurate estimation of unfair subspace  $\operatorname{col}(U)$ , resulting in more indistinguishable images (in sensitive attributes). As soon as the batch size  $b$  exceeds a certain threshold (e.g., 8000 for "Goatee"),  $\operatorname{col}(U)$  is estimated very well, and the unfair features are cleanly recovered, as it can be seen in the bottom rows.
F.3 Full Results on UCI Dataset
The full experiment results of fair PCA and downstream tasks on UCI datasets are provided. Again, our method is competitive to the existing fair PCA algorithms across the considered UCI datasets.
Fair NPM
(Unfair subsp.)
(a) Attribute: "Eyeglasses" (Left four: False, Right four: True)
(b) Attribute: "Mouth Slightly Open" (Left four: False, Right four: True)
(c) Attribute: "Goatee" (Left four: False, Right four: True)
Figure 3: CelebA, Additional results  $(m = 5)$
(a) Attribute: "Eyeglasses" (Left: False, Right: True)
(b) Attribute: "Mouth Slightly Open" (Left: False, Right: True)
Figure 4: CelebA, Additional ablation on  $m \in \{1,2,5,10\}$
(c) Attribute: "Goatee" (Left: False, Right: True)
(b) Attribute: "Mouth Slightly Open" (Left: False, Right: True)
(c) Attribute: "Goatee" (Left: False, Right: True)
Figure 5: CelebA, Additional ablation on  $b \in \{32000, 8000, 3200, 1600\}$
Table 2: Dataset = Adult Income [feature dim=102, #(train data)=31,655]
k	Method	%Var(↑)	MMD2(↓)	%Acc(↑) kernel	ΔDP(↓) SVM	%Acc(↑) linear SVM	ΔDP(↓) MLP
2	PCA	6.88(0.14)	0.374(0.006)	82.4(0.2)	0.19(0.01)	81.23(0.16)	0.2(0.0)
Olfat and Aswani (2019) (0.1)	Memory Out
Olfat and Aswani (2019) (0.0)	Memory Out
Lee et al. (2022) (1e-3)	5.68(0.11)	0.0(0.0)	80.34(0.24)	0.05(0.01)	78.09(0.36)	0.01(0.0)
Lee et al. (2022) (1e-6)	5.42(0.11)	0.0(0.0)	79.41(0.23)	0.02(0.01)	77.52(0.69)	0.01(0.0)
Kleindessner et al. (2023) (mean)	5.74(0.11)	0.002(0.0)	80.6(0.2)	0.07(0.01)	78.57(0.26)	0.01(0.0)
Kleindessner et al. (2023) (0.85)	4.09(0.17)	0.001(0.001)	75.52(0.21)	0.0(0.0)	75.33(0.19)	0.0(0.0)
Kleindessner et al. (2023) (0.5)	2.63(0.07)	0.0(0.0)	75.38(0.18)	0.0(0.0)	75.33(0.19)	0.0(0.0)
Kleindessner et al. (2023) (kernel)	Takes too long time
Ravfogel et al. (2020)	1.91(0.08)	0.001(0.001)	75.67(0.31)	0.0(0.0)	75.33(0.19)	0.0(0.0)
Ravfogel et al. (2022a)	1.91(0.09)	0.006(0.011)	75.59(0.34)	0.0(0.0)	75.33(0.19)	0.0(0.0)
Samadi et al. (2018)	N/A	N/A	82.63(0.18)	0.15(0.01)	82.21(0.2)	0.17(0.0)
Ours (offline, mean)	5.74(0.11)	0.002(0.0)	80.6(0.2)	0.07(0.01)	78.57(0.26)	0.01(0.0)
Ours (FNPM, mean)	5.74(0.11)	0.002(0.0)	80.6(0.2)	0.07(0.01)	78.57(0.26)	0.01(0.0)
Ours (offline, m=15)	4.04(0.14)	0.001(0.001)	75.51(0.23)	0.0(0.0)	75.33(0.19)	0.0(0.0)
Ours (FNPM, m=15)	4.07(0.13)	0.001(0.0)	75.54(0.25)	0.0(0.0)	75.33(0.19)	0.0(0.0)
Ours (offline, m=50)	2.63(0.07)	0.0(0.0)	75.38(0.18)	0.0(0.0)	75.33(0.19)	0.0(0.0)
Ours (FNPM, m=50)	2.64(0.06)	0.0(0.0)	75.44(0.21)	0.0(0.0)	75.33(0.19)	0.0(0.0)
10	PCA	20.82(0.38)	0.195(0.002)	88.58(0.21)	0.18(0.01)	83.14(0.21)	0.19(0.0)
Olfat and Aswani (2019) (0.1)	Memory Out
Olfat and Aswani (2019) (0.0)	Memory Out
Lee et al. (2022) (1e-3)	Takes too long time
Kleindessner et al. (2023) (mean)	18.84(0.35)	0.01(0.0)	88.43(0.18)	0.18(0.01)	81.71(0.29)	0.03(0.01)
Kleindessner et al. (2023) (0.85)	14.54(0.23)	0.002(0.0)	86.99(0.32)	0.16(0.0)	75.33(0.19)	0.0(0.0)
Kleindessner et al. (2023) (0.5)	11.34(0.2)	0.0(0.0)	83.01(0.28)	0.12(0.01)	75.33(0.19)	0.0(0.0)
Kleindessner et al. (2023) (kernel)	Takes too long time
Ravfogel et al. (2020)	9.36(0.26)	0.001(0.0)	85.71(0.78)	0.14(0.02)	75.33(0.19)	0.0(0.0)
Ravfogel et al. (2022a)	9.6(0.25)	0.001(0.0)	87.61(0.57)	0.16(0.01)	75.37(0.22)	0.0(0.0)
Samadi et al. (2018)	N/A	N/A	85.69(0.24)	0.18(0.01)	83.25(0.18)	0.17(0.01)
Ours (offline, mean)	18.84(0.35)	0.01(0.0)	88.43(0.18)	0.18(0.01)	81.71(0.29)	0.03(0.01)
Ours (FNPM, mean)	18.83(0.35)	0.009(0.0)	88.53(0.22)	0.18(0.01)	81.7(0.29)	0.03(0.01)
Ours (offline, m=15)	14.48(0.22)	0.002(0.0)	86.86(0.3)	0.16(0.0)	75.33(0.19)	0.0(0.0)
Ours (FNPM, m=15)	14.38(0.2)	0.001(0.0)	86.64(0.29)	0.16(0.0)	75.33(0.19)	0.0(0.0)
Ours (offline, m=50)	11.34(0.2)	0.0(0.0)	83.0(0.25)	0.12(0.01)	75.33(0.19)	0.0(0.0)
Ours (FNPM, m=50)	11.34(0.2)	0.0(0.0)	82.85(0.22)	0.11(0.01)	75.33(0.19)	0.0(0.0)
Table 3: Dataset = COMPAS [feature dim=11, #(train data)=4,316]
k	Method	%Var(↑)	MMD2(↓)	%Acc(↑)
kernel SVM	ΔDP(↓)
linear SVM	ΔDP(↓)
MLP
2	PCA	2.09(0.11)	0.057(0.006)	64.84(0.78)	0.28(0.03)	58.91(0.52)
Olfat and Aswani (2019) (0.1)
Olfat and Aswani (2019) (0.0)
Lee et al. (2022) (1e-3)	1.8(0.1)	0.003(0.002)	62.41(1.09)	0.03(0.02)	57.57(1.18)
Lee et al. (2022) (1e-6)	1.78(0.11)	0.003(0.002)	62.65(1.54)	0.03(0.03)	57.56(2.06)
Kleindessner et al. (2023) (mean)	1.97(0.1)	0.008(0.003)	61.94(0.59)	0.07(0.02)	56.03(1.09)
Kleindessner et al. (2023) (0.85)	1.78(0.13)	0.005(0.002)	60.64(1.08)	0.08(0.05)	56.16(1.21)
Kleindessner et al. (2023) (0.5)	1.62(0.09)	0.004(0.001)	59.67(0.89)	0.1(0.03)	54.81(1.04)
Kleindessner et al. (2023) (kernel)	N/A	N/A	57.8(1.82)	0.08(0.06)	54.74(1.21)
Ravfogel et al. (2020)	0.62(0.18)	0.0(0.0)	56.35(0.71)	0.01(0.01)	54.81(1.04)
Ravfogel et al. (2022a)	0.49(0.03)	0.002(0.002)	57.54(0.74)	0.03(0.03)	54.81(1.04)
Samadi et al. (2018)	N/A	N/A	64.19(1.07)	0.15(0.03)	59.15(0.59)
Ours (offline, mean)	1.97(0.1)	0.008(0.003)	61.94(0.59)	0.07(0.02)	56.03(1.09)
Ours (FNPM, mean)	1.97(0.1)	0.008(0.003)	61.94(0.59)	0.07(0.02)	56.03(1.09)
Ours (offline, m=2)	1.92(0.11)	0.006(0.002)	61.22(0.85)	0.11(0.04)	55.43(0.9)
Ours (FNPM, m=2)	1.92(0.11)	0.006(0.002)	61.08(0.9)	0.09(0.04)	55.51(1.16)
Ours (offline, m=5)	1.89(0.1)	0.006(0.002)	61.71(0.78)	0.1(0.04)	55.49(0.79)
Ours (FNPM, m=5)	1.89(0.1)	0.006(0.002)	61.67(0.76)	0.11(0.05)	55.55(0.9)
10	PCA	5.85(0.36)	0.089(0.002)	82.78(0.43)	0.15(0.04)	65.39(1.15)
Olfat and Aswani (2019) (0.1)
Olfat and Aswani (2019) (0.0)
Lee et al. (2022) (1e-3)	5.36(0.31)	0.002(0.001)	80.76(0.46)	0.1(0.03)	63.89(1.15)
Lee et al. (2022) (1e-6)	4.83(0.45)	0.002(0.001)	78.93(0.84)	0.09(0.03)	61.68(1.54)
Kleindessner et al. (2023) (mean)	5.67(0.37)	0.004(0.001)	80.87(0.54)	0.09(0.03)	64.76(1.04)
Kleindessner et al. (2023) (0.85)	5.41(0.33)	0.004(0.001)	80.64(0.82)	0.09(0.04)	63.63(1.19)
Kleindessner et al. (2023) (0.5)	5.29(0.31)	0.003(0.001)	79.22(0.44)	0.1(0.03)	61.62(0.63)
Kleindessner et al. (2023) (kernel)	N/A	N/A	65.96(1.12)	0.26(0.07)	64.35(0.8)
Ravfogel et al. (2020)	2.64(0.47)	0.001(0.0)	64.17(1.46)	0.04(0.02)	54.85(1.03)
Ravfogel et al. (2022a)	2.45(0.06)	0.001(0.0)	71.75(0.69)	0.07(0.04)	55.26(1.25)
Samadi et al. (2018)	N/A	N/A	69.75(1.0)	0.17(0.02)	65.54(0.72)
Ours (offline, mean)	5.67(0.37)	0.004(0.001)	80.87(0.54)	0.09(0.03)	64.78(1.04)
Ours (FNPM, mean)	5.68(0.37)	0.004(0.001)	80.87(0.49)	0.09(0.03)	64.84(1.01)
Ours (offline, m=2)	5.57(0.35)	0.004(0.001)	80.93(0.63)	0.09(0.03)	64.72(0.93)
Ours (FNPM, m=2)	5.58(0.35)	0.004(0.001)	80.82(0.48)	0.09(0.03)	64.65(1.1)
Ours (offline, m=5)	5.55(0.35)	0.004(0.001)	81.03(0.61)	0.09(0.03)	64.47(1.01)
Ours (FNPM, m=5)	5.55(0.34)	0.004(0.001)	80.9(0.49)	0.09(0.03)	64.54(1.23)