Table 13. Average target domain Macro-F1 score of all algorithms on the Taxi, ACS Pub.Cov (Temporal) and ACS Income (Synthetic) datasets. In Taxi and ACS Pub.Cov (Temporal), we test all algorithms on the 3 target domains and report the mean and standard deviation across all 3 target domains. The standard deviation here reflects the stability of performances across different target states. In ACS Income (Synthetic), we simulate strong covariate shifts according to the "Age" feature, where  $10\%$  and  $20\%$  means the minor group ratio in training, respectively.
Dataset Source	Taxi	ACS Pub. Cov (Temporal)	ACS Income (Synthetic)
BOG	NYC	NY	CA	20%	10%
i.d.	o.o.d	i.d.	o.o.d	i.d.	o.o.d	i.d.	o.o.d	i.d.	o.o.d	i.d.	o.o.d
Base Methods	LR	0.754	0.695 ± 0.007	0.802	0.731 ± 0.014	0.641	0.599 ± 0.009	0.586	0.528 ± 0.024	0.797	0.799	0.766	0.780
SVM	0.757	0.761 ± 0.036	0.809	0.735 ± 0.019	0.663	0.613 ± 0.013	0.612	0.556 ± 0.016	0.795	0.764	N/A	N/A
MLP	0.757	0.684 ± 0.012	0.823	0.719 ± 0.008	0.692	0.652 ± 0.012	0.674	0.587 ± 0.030	0.817	0.796	0.792	0.762
Tree Ensemble Methods	Random Forest	0.760	0.762 ± 0.032	0.840	0.718 ± 0.018	0.692	0.648 ± 0.016	0.663	0.571 ± 0.032	0.810	0.797	0.782	0.777
LightGBM	0.866	0.717 ± 0.029	0.841	0.701 ± 0.011	0.711	0.668 ± 0.016	0.683	0.595 ± 0.027	0.817	0.813	0.787	0.788
XGBoost	0.839	0.695 ± 0.015	0.846	0.723 ± 0.017	0.715	0.679 ± 0.014	0.689	0.591 ± 0.034	0.815	0.801	0.787	0.788
DRO Methods	χ2-DRO	0.776	0.693 ± 0.016	0.830	0.726 ± 0.009	0.675	0.633 ± 0.016	0.663	0.580 ± 0.031	0.820	0.788	0.790	0.783
CVaR-DRO	0.767	0.744 ± 0.032	0.835	0.732 ± 0.010	0.668	0.625 ± 0.014	0.654	0.586 ± 0.025	0.820	0.799	0.791	0.783
χ2-DORO	0.750	0.752 ± 0.027	0.816	0.732 ± 0.009	0.568	0.540 ± 0.014	0.531	0.501 ± 0.010	0.810	0.796	0.778	0.766
CVaR-DORO	0.756	0.724 ± 0.023	0.829	0.730 ± 0.006	0.652	0.610 ± 0.017	0.618	0.556 ± 0.021	0.812	0.781	0.778	0.763
Group DRO	N/A	N/A	N/A	N/A	0.682	0.655 ± 0.010	0.667	0.605 ± 0.020	0.814	0.798	0.795	0.763
Imbalanced Learning Methods	SUBY	0.859	0.669 ± 0.014	0.835	0.708 ± 0.014	0.706	0.690 ± 0.009	0.678	0.660 ± 0.010	0.774	0.699	0.716	0.562
RWY	0.811	0.737 ± 0.033	0.840	0.713 ± 0.017	0.716	0.703 ± 0.009	0.693	0.656 ± 0.019	0.795	0.780	0.768	0.765
SUBG	0.858	0.712 ± 0.032	0.837	0.701 ± 0.013	0.707	0.692 ± 0.013	0.687	0.588 ± 0.033	0.816	0.802	0.788	0.783
RWG	0.802	0.731 ± 0.035	0.840	0.720 ± 0.018	0.713	0.654 ± 0.015	0.686	0.601 ± 0.034	0.816	0.810	0.798	0.791
JTT	0.842	0.686 ± 0.003	0.830	0.704 ± 0.017	0.677	0.654 ± 0.011	0.676	0.641 ± 0.016	0.801	0.780	0.772	0.750
DWR	0.845	0.671 ± 0.016	0.813	0.673 ± 0.015	0.713	0.675 ± 0.017	0.685	0.604 ± 0.029	0.817	0.812	0.785	0.785
Fairness Methods	DP	0.842	0.696 ± 0.020	0.843	0.702 ± 0.014	0.712	0.671 ± 0.016	0.689	0.600 ± 0.031	0.815	0.813	0.785	0.795
EO	0.813	0.727 ± 0.028	0.846	0.706 ± 0.015	0.713	0.670 ± 0.018	0.686	0.586 ± 0.031	0.814	0.798	0.782	0.789
EP	0.801	0.728 ± 0.027	0.835	0.710 ± 0.018	0.712	0.671 ± 0.016	0.687	0.597 ± 0.031	0.814	0.815	0.786	0.786
Exp	0.797	0.724 ± 0.034	0.838	0.702 ± 0.015	0.712	0.648 ± 0.018	0.685	0.597 ± 0.033	0.814	0.798	0.783	0.783
Threshold	0.806	0.660 ± 0.003	0.812	0.536 ± 0.027	0.711	0.670 ± 0.018	0.688	0.605 ± 0.030	0.785	0.786	0.752	0.761