SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Multiple rounds of pseudo-labeling. While SFB learns the optimal unstable classifier  $h_U^e$  in the test domain given enough unlabelled data, § 4.1 discussed how more accurate pseudo-labels  $\widehat{Y}$  improve the sample efficiency of SFB. In particular, in a restricted-sample setting, more accurate pseudo-labels result in an unstable classifier  $h_U^e$  which better harnesses  $X_U$  in the test domain. With this in mind, note that, after adapting, we expect the joint predictions of SFB to be more accurate than its stable-only predictions. This raises the question: can we use these improved predictions to form more accurate pseudo-labels, and, in turn, an unstable classifier  $h_U^e$  that leads to even better performance? Furthermore, can we repeat this process, using multiple rounds of pseudo-labelling to refine our pseudo-labels and ultimately  $h_U^e$ ? While this multi-round approach loses the asymptotic guarantees of § 4.2, we found it to work quite well in practice. In this (sub)section, we thus investigate the performance of SFB with and without multiple rounds of pseudo-labeling (PL rounds).
Table 4: SFB ablations on CMNIST. Means and standard errors are over 3 random seeds. BC: bias correction. CS: post-hoc calibration of the stable classifier. CU: post-hoc calibration of the unstable classifier. PL Rounds: Number of pseudo-labeling rounds used. GT adpt: “ground-truth” adaptation using true labels in the test domain.
Algorithm	Bias Correction	Calibration	PL Rounds	Test Acc.
Stable	Unstable
SFB no adpt.	1	70.6 ± 1.8
SFB	1	78.0 ± 2.9
+BC	✓	1	83.4 ± 2.8
+CS	✓	1	80.6 ± 3.4
+CU	✓	1	76.6 ± 2.4
+BC+CS+CU	✓	✓	✓	1	84.4 ± 2.2
+BC+CS	✓	✓	1	84.9 ± 2.6
+BC+CS	✓	✓	2	87.4 ± 1.9
+BC+CS	✓	✓	3	88.1 ± 1.8
+BC+CS	✓	✓	4	88.6 ± 1.3
+BC+CS	✓	✓	5	88.7 ± 1.3
SFB GT adpt.	✓	✓	1	89.0 ± 0.3
Results. Table 4 reports the ablations of SFB on ColorMNIST. Here we see that: (i) bias correction significantly boosts performance (+BC); (ii) calibrating the stable predictor also boosts performance without (+CS) and with (+BC+CS) bias correction, with the latter leading to the best performance; (iii) calibrating the unstable predictor (with respect to the pseudo-labels) slightly hurts performance without (+CU) and with (+BC+CS+CU) bias correction and stable-predictor calibration; (iv) multiple rounds of pseudo-labeling boosts performance, while also reducing the performance variation across random seeds; (v) using bias correction, stable-predictor calibration and 5 rounds of pseudo-labeling results in near-optimal adaptation performance, as indicated by the similar performance of SFB when using true labels Y to adapt  $h_U^e$  (denoted "SFB GT adpt." in Table 4).
F.1.2 Different stability penalties
In our experiments of § 6, we used IRM for the stability term of our SFB method, given in Eq. (5.1). However, as discussed in § 5, many other approaches exist for enforcing stability [35, 58, 47, 15, 67, 40, 77], and, in principle, any of these could be used. To illustrate this point, we now evaluate the performance of SFB when using different stability penalties, namely IRM [1], VReX [35], EQRM [15] and CLOvE [70]. For all penalties, we use SFB with bias correction, post-hoc calibration of the stable predictor, and 5 rounds of pseudo-labeling (see the ablation study of App. F.1.1).
Table 5: CMNIST test-domain accuracies for SFB with different stability penalties. Shown are the mean and standard error over 10 seeds.
Algorithm	Without Adaptation	With Adaptation
SFB w. IRM	70.6 ± 1.8	88.7 ± 1.3
SFB w. VReX	72.5 ± 1.0	88.7 ± 1.5
SFB w. EQRM	69.0 ± 2.8	88.2 ± 2.5
SFB w. CLOvE	67.0 ± 3.7	77.0 ± 6.6
F.1.3 Full results
We now provide extended/full results of those provided in the main text. In particular, Table 6 represents an extended version of Table 3 in the main text, comparing against many more baseline methods. In addition, Table 7 provides the full numerical results for all adaptive baseline methods (described in App. G.1), which correspond to the plots of Fig. 2 in the main text.
Table 6: CMNIST test-domain accuracies. Mean and standard error are over 10 seeds. Extended/full version of Table 3 in the main text.
Algorithm	Test Acc.
ERM	27.9 ± 1.5
GroupDRO [53]	29.0 ± 1.1
IRM [1]	69.7 ± 0.9
SD [46]	70.3 ± 0.6
IGA [57]	57.7 ± 3.3
Fisher [48]	70.1 ± 0.7
V-REx [35]	71.6 ± 0.5
EQRM [15]	71.4 ± 0.4
SFB no adpt.	70.6 ± 1.8
SFB	88.1 ± 1.8
Oracle no adpt.	72.1 ± 0.7
Oracle	89.9 ± 0.1
F.2 Camelyon17
We now provide results on the Camelyon17 [3] dataset. See App. E for a description of the dataset, and App. G.5 for implementation details.
Algorithm	Domain (Color-Label Correlation)
1.0	0.9	0.8	0.7	0.6	0.5	-0.6	-0.7	-0.8	-0.9	-1.0
ERM	97.5 ± 0.3	88.5 ± 0.4	79.7 ± 0.4	70.6 ± 0.5	61.4 ± 0.7	52.5 ± 0.4	43.5 ± 0.7	34.7 ± 0.7	25.1 ± 0.5	16.4 ± 0.4	7.6 ± 0.6
ERM+T3A	98.1 ± 0.2	88.9 ± 0.4	79.8 ± 0.4	70.4 ± 0.5	61.0 ± 0.8	51.7 ± 0.4	42.3 ± 0.7	33.0 ± 0.6	23.1 ± 0.4	13.8 ± 0.5	4.5 ± 0.5
ERM+PL (last)	97.6 ± 0.2	88.6 ± 0.3	79.7 ± 0.4	70.6 ± 0.5	61.4 ± 0.8	52.5 ± 0.4	43.4 ± 0.7	34.6 ± 0.7	25.0 ± 0.5	16.2 ± 0.3	7.4 ± 0.6
ERM+PL (all)	100.0 ± 0.0	90.0 ± 0.4	80.2 ± 0.4	70.1 ± 0.4	59.9 ± 0.8	50.1 ± 0.4	40.0 ± 0.5	30.1 ± 0.6	19.6 ± 0.3	9.9 ± 0.3	0.0 ± 0.1
IRM	70.6 ± 2.1	70.3 ± 1.9	70.4 ± 1.7	70.2 ± 1.1	69.9 ± 0.7	69.9 ± 0.7	70.1 ± 0.5	69.7 ± 0.6	69.8 ± 1.2	69.6 ± 1.6	69.4 ± 1.7
IRM+T3A	72.3 ± 1.7	71.2 ± 1.6	70.7 ± 1.6	70.2 ± 1.0	69.8 ± 0.7	69.9 ± 0.7	70.3 ± 0.6	70.6 ± 0.5	71.6 ± 1.1	72.4 ± 1.7	73.4 ± 1.9
IRM+PL (last)	70.8 ± 2.2	70.5 ± 1.9	70.5 ± 1.7	70.2 ± 1.1	69.9 ± 0.7	69.9 ± 0.6	70.0 ± 0.6	69.7 ± 0.6	69.9 ± 1.2	69.8 ± 1.6	69.7 ± 1.7
IRM+PL (all)	99.6 ± 1.2	89.4 ± 1.2	79.5 ± 1.3	68.7 ± 3.8	63.3 ± 4.7	63.5 ± 5.3	63.8 ± 4.5	67.5 ± 2.8	76.4 ± 4.2	87.2 ± 5.0	98.2 ± 3.0
SFB	100.0 ± 0.1	90.5 ± 0.5	79.8 ± 0.8	71.0 ± 1.1	70.9 ± 0.3	69.2 ± 0.4	68.4 ± 1.3	71.2 ± 0.3	79.3 ± 1.2	88.7 ± 1.3	98.9 ± 1.5
Table 7: CMNIST comparison with other test-time/source-free unsupervised domain adaptation methods. Means and standard errors are over 10 seeds. The largest mean per column/domain is in bold. "last": only last-layer updated. "all": all layers updated. Fig. 2 gives the corresponding plot.
Table 2 shows that ERM, IRM and SFB perform similarly on Camelyon17. In line with [24], we found that a properly-tuned ERM model can be difficult to beat on real-world datasets, particularly when the model is pre-trained on ImageNet and the dataset doesn't contain severe distribution shift. While we conducted this proper tuning for ERM, IRM, and SFB (see App. G.5), doing so for ACTIR was non-trivial. We thus report the result from their paper [31, Table 1], which is likely lower due to sub-optimal hyperparameters. In particular, we found that, for ERM and IRM, using a lower learning rate (1e-5 vs 1e-4) and early stopping (1 vs 25 epochs) improved performance by 20 percentage points, from around  $70\%$  [31, Table 1] to around  $90\%$  (Table 8 below). It remains to be seen whether or not ACTIR can improve over a properly-tuned ERM model on Camelyon17.
While it may seem disappointing that SFB does not outperform the simpler methods of IRM and ERM on Camelyon17, we note that SFB can only be expected to do well when there is some gain in out-of-distribution performance from enforcing stability, e.g., when IRM outperforms ERM. The identical performances of IRM and ERM in Table 8 indicate that, with ImageNet pre-training and proper hyperparameter tuning, this is not the case for Camelyon17. Finally, despite the similar performances, we note that adapting SFB on Camelyon17 still gives a small performance boost and reduces the variance across random seeds.
Table 8: Camelyon17 test-domain accuracies. Mean and standard errors are over 5 random seeds. †: Result taken from [31, Tab. 1] and is likely lower due to sub-optimal hyperparameters (they report ≈70% for ERM and IRM).
Algorithm	Accuracy
ERM	90.2 ± 1.1
IRM	90.2 ± 1.1
ACTIR	77.7 ± 1.7†
SFB no adapt.	89.8 ± 1.2
SFB	90.3 ± 0.7
Below we provide further implementation details for the experiments of this work. Code is available at: https://github.com/cianeastwood/sfb.
G.1 Adaptive baselines
For both the synthetic and CMNIST datasets, we compare against adaptive baseline methods by using pseudo-labeling (PL, [36]) and test-time classifier adjustment (T3A, [30]) on top of both ERM and IRM, choosing all adaptation hyperparameters using leave-one-domain-out cross-validation:
-  $ERM/IRM + PL$  (last): After training with ERM/IRM, we update the last layer using the model's own pseudo-labels [36].
-  $ERM/IRM + PL$  (all): After training with ERM/IRM, we update all layers using the model's own pseudo-labels [36].
-  $ERM/IRM + T3A$ : After training with ERM/IRM, we replace the classifier (final layer) with the template-based classifier of T3A [30]. This means: (i) computing template representations for each class using pseudo-labeled test-domain data; and (ii) classifying each example based on its distance to these templates.
G.2 Synthetic experiments
Following Jiang and Veitch [31], we use a simple three-layer network with 8 units in each hidden layer and the Adam optimizer, choosing hyperparameters using the validation domain.
For SFB, we sweep over  $\lambda_S$  in  $\{0.01, 0.1, 1, 5, 10, 20\}$  and  $\lambda_C$  in  $\{0.01, 0.1, 1\}$ . For SFB's unsupervised adaptation, we employ the bias correction of Alg. 1 and calibrate the stable predictor using post-hoc temperature scaling, choosing the temperature to minimize the expected calibration error (ECE, [25]) on the validation domain. In addition, we use the Adam optimizer with an adaptation learning rate of 0.01, choosing the number of adaptation steps in  $[1, 20]$  (via early stopping) using the validation domain. Finally, we report the mean and standard error over 100 random seeds.
G.3 ColorMNIST experiments
Training details. We follow the setup of Eastwood et al. [15, §6.1] and build on their open-source code<sup>5</sup>. In particular, we use the original MNIST training set to create training and validation sets for each domain, and the original MNIST test set for the test sets of each domain. For all methods, we use a 2-hidden-layer MLP with 390 hidden units, the Adam optimizer, a learning rate of 0.0001 with cosine scheduling, and dropout with  $p = 0.2$ . In addition, we use full batches (size 25000), 400 steps for ERM pre-training (which directly corresponds to the delicate penalty "annealing" or warm-up periods used by penalty-based methods on ColorMNIST [1, 35, 15, 74]), and 600 total steps. We sweep over stability-penalty weights in  $\{50, 100, 500, 1000, 5000\}$  for IRM, VREx and SFB and  $\alpha$ 's in  $1 - \{e^{-100}, e^{-250}, e^{-500}, e^{-750}, e^{-1000}\}$  for EQRM. As the stable (shape) and unstable (color) features are conditionally independent given the label, we fix SFB's conditional-independence penalty weight  $\lambda_C = 0$ . As is the standard for ColorMNIST, we use a test-domain validation set to select the best settings (after the total number of steps), and then report the mean and standard error over 10 random seeds on a test-domain test set. As in previous works, the hyperparameter ranges of all methods are selected by peeking at test-domain performance. While far from ideal, this is quite difficult to avoid with ColorMNIST and highlights a core problem with hyperparameter selection in DG—as discussed by many previous works [1, 35, 24, 74, 15].
SFB adaptation details. For SFB's unsupervised adaptation in the test domain, we use a batch size of 2048 and employ the bias correction of Alg. 1. In addition, we calibrate the stable predictor using post-hoc temperature scaling, choosing the temperature to minimize the expected calibration error (ECE, [25]) across the two training domains. Again using the two training domains for hyperparameter selection, we sweep over adaptation learning rates in  $\{0.1, 0.01\}$ , choose the best adaptation step in  $[5, 20]$  (via early stopping), and sweep over the number of pseudo-labeling rounds in  $[1, 5]$ . Finally, we report the mean and standard error over 3 random seeds for adaptation.
G.4 PACS experiments
We follow the setup of Jiang and Veitch [31, § 6.4] and build on their open-source code<sup>6</sup>. This means using an ImageNet-pretrained ResNet-18, the Adam optimizer with a learning rate of  $10^{-4}$ , and choosing hyperparameters using leave-one-domain-out cross-validation (akin to K-fold cross-validation, except with domains). In particular, for each held-out test domain, we train 3 models—each time leaving out 1 of the 3 training domains for validation—and then select hyperparameters based on the best average performance across the held-out validation domains. Finally, we use the selected hyperparameters to retrain the model using all 3 training domains.
For SFB, we sweep over  $\lambda_{S}$  in  $\{0.01, 0.1, 1, 5, 10, 20\}$ ,  $\lambda_{C}$  in  $\{0.01, 0.1, 1\}$ , and learning rates in  $\{10^{-4}, 50^{-4}\}$ . For SFB's unsupervised adaptation, we employ the multi-class bias correction of Alg. 2 and calibrate the stable predictor using post-hoc temperature scaling, choosing the temperature
$^{5}$ https://github.com/cianeastwood/qrm/tree/main/CMNIST
$^{6}$ https://github.com/ybjiaang/ACTIR.
to minimize the expected calibration error (ECE, [25]) across the three training domains. In addition, we use the Adam optimizer with an adaptation learning rate of 0.01, choosing the number of adaptation steps in [1, 20] (via early stopping) using the training domains. Finally, we report the mean and standard error over 5 random seeds.
G.5 Camelyon17 experiments
We follow the setup of Jiang and Veitch [31, § 6.3] and build on their open-source code<sup>7</sup>. This means using an ImageNet-pretrained ResNet-18, the Adam optimizer, and, following [33], choosing hyperparameters using the validation domain (hospital 4). In contrast to [31], we use a learning rate of  $10^{-5}$  for all methods, rather than  $10^{-4}$ , and employ early stopping using the validation domain. We found this to significantly improve all methods. E.g., the baselines of ERM and IRM improve by approximately 20 percentage points, jumping from  $\approx 70\%$  to  $\approx 90\%$ .
For SFB, we sweep over  $\lambda_{S}$  in  $\{0.01, 0.1, 1, 5, 10, 20\}$  and  $\lambda_{C}$  in  $\{0.01, 0.1, 1\}$ . For SFB's unsupervised adaptation, we employ the bias correction of Alg. 1 and calibrate the stable predictor using post-hoc temperature scaling, choosing the temperature to minimize the expected calibration error (ECE, [25]) on the validation domain. In addition, we use the Adam optimizer with an adaptation learning rate of 0.01, choosing the number of adaptation steps in  $[1, 20]$  (via early stopping) using the validation domain. Finally, we report the mean and standard error over 5 random seeds.
H Further Related Work
Learning with noisy labels. An intermediate goal in our work, namely learning a model to predict  $Y$  from  $X_{U}$  using pseudo-labels based on  $X_{S}$ , is an instance of learning with noisy labels, a widely studied problem [56, 44, 7, 60, 38, 64]. Specifically, under the complementarity assumption  $(X_{S} \perp X_{U}|Y)$ , the accuracy of the pseudo-labels on each class is independent of  $X_{U}$ , placing us in the so-called class-conditional random noise model [56, 44, 7]. As we discuss in Section 4, our theoretical insights about the special structure of pseudo-labels complement existing results on learning under this model. Our bias-correction (Eq. (4.3)) for  $P_{Y|X_{U}}$  is also closely related to the "method of unbiased estimators" [44]. However, rather than correcting the loss used in ERM, our post-hoc bias correction applies to any calibrated classifier. Moreover, our ultimate goal, learning a predictor of  $Y$  jointly using  $X_{S}$  and  $X_{U}$ , is not captured by learning with noisy labels.
Co-training. Our use of stable-feature pseudo-labels to train a classifier based on a disjoint subset of (unstable) features is reminiscent of co-training [10]. Both methods benefit from conditional independence of the two feature subsets given the label to ensure that they provide complementary information. The key difference is that while co-training requires (a small number of) labeled samples from the same distribution as the test data, our method instead uses labeled data from a different distribution (training domains), along with the assumption of a stable feature. Additionally, while co-training iteratively refines two pre-trained classifiers symmetrically based on each other's predictions, our method only trains the unstable classifier, in a single iteration, using the stable classifier's predictions.
Boosting. Our method of building a strong (albeit unstable) classifier using a weak (but stable) one is reminiscent of boosting, in which one ensembles weak classifiers to create a single strong classifier [54] and which inspires the name of our approach, "stable feature boosting (SFB)". However, whereas traditional boosting improves weak classifiers by examining how their predictions differ from true labels, our adaptation method utilizes only pseudo-labels and needs no true labels from the test domain. For example, while traditional boosting only refines functions of existing features, SFB can utilize new features that are only available in the test domain.
Learning theory for domain generalization. In addition to often assuming particular kinds of distribution shifts (e.g., covariate shift), existing error bounds for domain generalization often depend on some notion of distance between training and test domains (which does not vanish as more data is collected within domains) [9, 5, 76, 75] or assume that the test domain has a particular structural relationship with the training domains (e.g., is a convex combination of training domains [41]).
See Footnote 6.
See Krogel and Scheffer [34], Blum and Mitchell [10, Theorem 1] for discussion of this assumption.
In contrast, under the structure of invariant and complementary features, we show that consistent generalization (i.e., with generalization error vanishing as more data is collected within domains) is possible in any test domain. Additionally, whereas these prior works derive uniform convergence bounds (implying good generalization for ERM), our results demonstrate the benefit of an additional bias-correction step after training. We also note that, in much of this literature, "invariance" refers to invariance of the covariate marginal distribution  $P_{X}$  across domains; in contrast, our notion of stable features (Defn. 4.1) refers to invariance of the conditional  $P_{Y|X}$ .
I Performance When Complementarity is Violated
Thm. 4.4 justifies the bias correction of Eq. (4.3) under the assumption that stable  $X_{S}$  and unstable  $X_{U}$  features are complementary, i.e., conditionally independent given the label  $Y$ . In this section, we discuss what happens if this assumption is relaxed and provide some intuition for why the bias correction appears to help even when complementarity is violated (as we observed in some of our experiments). In particular, we provide an argument that, in most cases, the bias correction should improve the accuracy of a naive classifier by making it agree more often with the Bayes-optimal classifier. While not a rigorous proof, we believe that this argument provides some insight into SFB's strong performance even when complementarity is violated.
In the absence of complementarity, the quantity  $\operatorname*{Pr}[\widehat{Y} = 1|Y = 1,X_u = x_U]$  no longer reduces to the class-wise accuracy  $\operatorname*{Pr}[\widehat{Y} = 1|Y = 1]$ ; thus we write more generally  $\epsilon_1(x_U) = \operatorname*{Pr}[\widehat{Y} = 1|Y = 1,X_u = x_U]$ , and we write  $\overline{\epsilon_1} = \mathbb{E}_{X_U}[\epsilon_1(X_U)] = \operatorname*{Pr}[\widehat{Y} = 1|Y = 1]$  instead of simply  $\epsilon_{1}$  for the accuracy on class 1. Similarly, we write  $\epsilon_0(x_U) = \operatorname*{Pr}[\widehat{Y} = 0|Y = 0,X_u = x_U]$ , and we write  $\overline{\epsilon_0} = \mathbb{E}_{X_U}[\epsilon_0(X_U)] = \operatorname*{Pr}[\widehat{Y} = 0|Y = 0]$  instead of simply  $\epsilon_0$  for the accuracy on class 0.
Let  $f_{*}(x_{U}) = \operatorname{Pr}[Y = 1 | X_{U} = x_{U}]$  denote the true regression function, and let  $h_{*}(x_{U}) = 1 \{f_{*}(x_{U}) > 0.5\}$  denote the Bayes-optimal classifier. It is well known that the Bayes-optimal classifier  $h_{*}$  has the maximum possible accuracy out of all classifiers. Thus, the sub-optimality of a classifier  $h$  can be measured by the probability  $S(h) = \operatorname{Pr}_{X_{U}}[h(X_{U}) \neq h_{*}(X_{U})]$  that it disagrees with the Bayes-optimal classifier. Our next result expresses  $S(h)$  in terms of the true regression function  $f_{*}$ , the functions  $\epsilon_{0}$  and  $\epsilon_{1}$ , and the distribution of  $X_{U}$ , when  $h$  is the bias-corrected classifier
$$
h _ {B C} (x _ {U}) := 1 \left\{\frac {\Pr [ \widehat {Y} = 1 | X _ {U} = x _ {U} ] + \bar {\epsilon} _ {0} - \bar {\epsilon} _ {1}}{\bar {\epsilon} _ {0} + \bar {\epsilon} _ {1} - 1} > 0. 5 \right\}
$$
from Thm. 4.4 or when  $h$  is the "naive" classifier
$$
h _ {N a i v e} (x _ {U}) := 1 \left\{\operatorname * {P r} [ \widehat {Y} = 1 | X _ {U} = x _ {U} ] > 0. 5 \right\}
$$
that simply treats the pseudo-labels as true labels.
Proposition I.1.
$$
S \left(h _ {B C}\right) = \underset {X _ {U}} {\mathrm {P r}} \left[ \left| f _ {*} (X _ {U}) - 0. 5 \right| \leq \frac {\left| \epsilon_ {0} (X _ {U}) - \epsilon_ {1} (X _ {U}) - \mathbb {E} _ {X _ {U}} \left[ \epsilon_ {0} (X _ {U}) - \epsilon_ {1} (X _ {U}) \right] \right|}{2 (\epsilon_ {0} (X _ {U}) + \epsilon_ {1} (X _ {U}) - 1)} \right],
$$
$$
S \left(h _ {N a i v e}\right) = \Pr_ {X _ {U}} \left[ | f _ {*} (X _ {U}) - 0. 5 | \leq \frac {\left| \epsilon_ {0} \left(X _ {U}\right) - \epsilon_ {1} \left(X _ {U}\right) \right|}{2 \left(\epsilon_ {0} \left(X _ {U}\right) + \epsilon_ {1} \left(X _ {U}\right) - 1\right)} \right].
$$
These two formulae for  $S(h_{BC})$  and  $S(h_{Naive})$  differ only in the numerator of the right-hand side; letting  $Z \coloneqq \epsilon_0(X_U) - \epsilon_1(X_U)$ , the sub-optimality of  $h_{BC}$  scales with  $|Z - \mathbb{E}[Z]|$ , whereas the sub-optimality of  $h_{Naive}$  scales with  $|Z|$ . Intuitively, for all except very pathological random variables  $Z$ ,  $|Z - \mathbb{E}[Z]|$  is typically smaller than  $|Z|$ . Although not a rigorous proof that the bias correction is always better than the naive classifier, this analysis provides an argument that, in most cases, the bias correction should improve on the accuracy of the naive classifier, by making it agree more often with the Bayes-optimal classifier.
We conclude by sketching the proof of Proposition I.1:
Proof. By construction, a thresholding classifier  $h(x) = 1\{f(x) > 0.5\}$  disagrees with the Bayes-optimal classifier if and only if
$$
f (x) \leq 0. 5 <   f _ {*} (x) \quad \text {o r} \quad f _ {*} (x) \leq 0. 5 <   f (x).
$$
Expanding these inequalities in the cases  $f(x) = \frac{\operatorname*{Pr}[\widehat{Y} = 1|X_U = x] + \overline{\epsilon}_0 - \overline{\epsilon}_1}{\overline{\epsilon}_0 + \overline{\epsilon}_1 - 1}$  and  $f(x) = \operatorname*{Pr}[\widehat{Y} = 1|X_U = x]$  and solving for the quantity  $f_{*}(x) - 0.5$  in each case gives Proposition I.1.