SlowGuess/NeurIPS_2023_OCR huggingface dataset:
This procedure evaluates the gains of reporting by comparing the performance of a model assigned at a leaf node  $h$  and a model assigned at a parent node  $h'$  which does not use the reported information. Here, the null hypothesis  $H_0$  assumes that the parent model performs as well as the leaf model – and thus, we reject the null hypothesis when there is sufficient evidence to suggest that reporting will improve performance in deployment. Our routine allows practitioners to specify the hypothesis test to compute the gains. By default, we use the McNemar test for accuracy [21] and the Delong test for AUC [19, 50]. In general, we can use a bootstrap hypothesis test [20].
A.4 Greedy Induction of Sequential Reporting Interface
We present an additional routine to construct reporting interfaces for sequential systems in Algorithm 5. We include this routine as an alternative option that can be used to construct a reporting interface in settings where it may be impractical or undesirable to enumerate all possible reporting interfaces. The procedure results in a valid reporting interface that ensures gains. However, it does not guarantee an optimal tree in terms of maximizing the overall gain and does not allow practitioners to choose between reporting interfaces after training.
Algorithm 5 Greedy Induction Routine for Sequential Reporting Interfaces
1: procedure GreEDYTREE(R)  
2:  $T \gets$  empty reporting interface  
3: repeat  
4: for  $r \in$  leaves(T) do  
5:  $\{\mathcal{A}_r\} \gets G_i : r[i] = \emptyset$   
6:  $\mathcal{A}^* \gets \operatorname{argmax}_{\mathcal{A} \in \{\mathcal{A}_r\}} \min_{r' \in r \cdot \text{split}(\mathcal{A})} \Delta_{r'}(r', r)$   
7:  $r \cdot \text{split}(\mathcal{A}^*)$   
8: end for  
9: until no splits are added  
10: return  $T$ , reporting interface that ensures gains for reporting each  $\mathcal{R}$ .  
11: end procedure
Algorithm 5 takes as input a collection of reporting options  $\mathcal{R}$  and outputs a single reporting interface using a greedy tree induction routine that chooses the attribute to report to maximize the minimum gain at each step. The procedure uses the reporting options to iteratively construct a reporting tree that branches on all of the attributes in  $\mathcal{R}$ . The procedure considers each unused attribute for each splitting point and splits on the attribute that provides the greatest minimum gain for the groups contained at that node.
B Description of Datasets used in Section 4 - Experiments
We include additional information about the datasets used in Section 4.
Dataset	Reference	Outcome Variable	n	d	m	G
apnea	Ustun et al. [55]	patient has obstructive sleep apnea	1,152	28	6	{age, sex}
cardio_eicu	Pollard et al. [44]	patient with cardiogenic shock dies	1,341	49	8	{age, sex, race}
cardio_mimic	Johnson et al. [32]	patient with cardiogenic shock dies	5,289	49	8	{age, sex, race}
coloncancer	Scosyrev et al. [45]	patient dies within 5 years	29,211	72	6	{age, sex}
lungcancer	Scosyrev et al. [45]	patient dies within 5 years	120,641	84	6	{age, sex}
saps	Allyn et al. [3]	ICU mortality	7,797	36	4	{age, HIV}
Table 3: Datasets used to fit clinical prediction models in Section 4. Here:  $n$  denotes the number of examples in each dataset;  $d$  denotes the number of features;  $\mathcal{G}$  denotes the group attributes that are used for personalization; and  $m = |\mathcal{G}|$  denotes the number of intersectional groups. Each dataset is de-identified and available to the public. The cardio_eicu, cardio_mimic, lungcancer datasets require access to public repositories listed under the references. The saps and apnea datasets must be requested from the authors. The support dataset can be downloaded directly from the URL below.
apnea We use the obstructive sleep apnea (OSA) dataset outlined in Ustun et al. [55]. This dataset includes a cohort of 1,152 patients where  $23\%$  have OSA. We use all available features (e.g. BMI, comorbidities, age, and sex) and binarize them, resulting in 26 binary features.
cardio_eicu & cardio_mimic Cardiogenic shock is an acute condition in which the heart cannot provide sufficient blood to the vital organs [31]. These datasets are designed to predict cardiogenic shock for patients in intensive care. Each dataset contains the same features, group attributes, and outcome variables for patients in different cohorts. The cardio_eicu dataset contains records for a cohort of patients in the Collaborative Research Database V2.0 [44]. The cardio_eicu dataset contains records for a cohort of patients in the MIMIC-III [32] database. Here, the outcome variable indicates whether a patient in the ICU with cardiogenic shock will die while in the ICU. The features encode the results of vital signs and routine lab tests (e.g. systolic BP, heart rate, hemoglobin count) that were collected up to 24 hours before the onset of cardiogenic shock.
lungcancer We consider a cohort of 120,641 patients who were diagnosed with lung cancer between 2004-2016 and monitored as part of the National Cancer Institute SEER study [45]. Here, the outcome variable indicates if a patient dies within five years from any cause, and  $16.9\%$  of patients died within the first five years from diagnosis. The cohort includes patients from Greater California, Georgia, Kentucky, New Jersey, and Louisiana, and does not cover patients who were lost to follow-up (censored). Age and Sex were considered as group attributes. The features reflect the morphology and histology of the tumor (e.g., size, metastasis, stage, node count and location, number and location of notes) as well as interventions that were administered at the time of diagnosis (e.g., surgery, chemo, radiology).
coloncancer We consider a cohort of 120,641 patients who were diagnosed with colorectal cancer between 2004-2016 and monitored as part of the National Cancer Institute SEER study [45]. Here, the outcome variable indicates if a patient dies within five years from any cause, and  $42.1\%$  of patients die within the first five years from diagnosis. The cohort includes patients from Greater California. Age and Sex were considered as group attributes. The features reflect the morphology and histology of the tumor (e.g., size, metastasis, stage, node count and location, number and location of notes) as well as interventions that were administered at the time of diagnosis (e.g., surgery, chemo, radiology).
saps The Simplified Acute Physiology Score II (SAPS II) score predicts the risk of mortality of critically-ill patients in intensive care [37]. The data contains records of 7,797 patients from 137 medical centers in 12 countries. Here, the outcome variable indicates whether a patient dies in the ICU, with  $12.8\%$  patient of patients dying. The features reflect comorbidities, vital signs, and lab measurements.
C Results for Different Model Classes and Prediction Tasks
In this Appendix, we present experimental results for additional model classes and prediction tasks. We produce these results using the setup in Section 4.1, and summarize them in the same way as Table 2. We refer to them in our discussion in Section 4.2.
C.1 Logistic Regression for Ranking (AUC)
Dataset	Metrics	STATIC	IMPUTED	PARTICIPATORY
1Hot	mHot	KNN-1Hot	KNN-mHot	Minimal	Flat	Seq
apnea	Overall Performance	0.774	0.774	0.776	0.776	0.776	0.851	0.851
Overall Gain	-0.002	-0.002	0.000	-0.000	0.000	0.074	0.074
Group Gains	-0.002	-0.002	-0.002	-0.002	0.000	0.004	0.004
n = 1152, d = 26	Max Disparity	0.004	0.005	0.004	0.005	0.002	0.111	0.111
G = {age, sex}	Rat. Violations	2	2	2	2	0	0	0
Ustun et al. [55]	Imputation Risk	-0.002	-0.002
Options Pruned	0/6	0/6	0/12	0/12	5/7	4/12	4/12
Data Use	100.0%	100.0%	0.0%	0.0%	16.7%	100.0%	83.3%
cardio_eicu	Overall Performance	0.864	0.863	0.863	0.862	0.865	0.966	0.966
Overall Gain	0.002	0.001	0.000	-0.001	0.002	0.103	0.103
n = 1341, d = 49	Group Gains	-0.005	-0.010	-0.005	-0.010	0.000	0.170	0.170
G = {age, sex, race}	Max Disparity	0.009	0.019	0.009	0.019	0.003	0.170	0.170
|G| = 8 groups	Rat. Violations	3	3	3	3	0	0	0
Pollard et al. [44]	Imputation Risk	-0.005	-0.010
Options Pruned	0/8	0/8	0/27	0/27	6/9	13/27	11/27
Data Use	100.0%	100.0%	0.0%	0.0%	25.0%	100.0%	95.8%
cardio_mimic	Overall Performance	0.881	0.881	0.882	0.880	0.881	0.914	0.914
Overall Gain	0.000	0.000	0.002	-0.000	0.000	0.034	0.034
n = 5289, d = 49	Group Gains	-0.001	-0.001	-0.001	-0.001	0.000	0.008	0.008
G = {age, sex, race}	Max Disparity	0.002	0.002	0.002	0.002	0.001	0.049	0.049
|G| = 8 groups	Rat. Violations	3	3	3	3	0	0	0
Johnson et al. [32]	Imputation Risk	-0.001	-0.001
Options Pruned	0/8	0/8	0/27	0/27	6/9	9/27	8/27
Data Use	100.0%	100.0%	0.0%	0.0%	25.0%	100.0%	91.7%
coloncancer	Overall Performance	0.685	0.685	0.683	0.683	0.685	0.700	0.700
Overall Gain	0.001	0.002	-0.000	-0.000	0.001	0.016	0.016
n = 29211, d = 72	Group Gains	-0.001	-0.001	-0.001	-0.001	0.000	-0.021	0.001
G = {age, sex}	Max Disparity	0.003	0.002	0.003	0.002	0.001	0.020	0.020
|G| = 6 groups	Rat. Violations	3	2	3	2	0	0	0
Scosyrev et al. [45]	Imputation Risk	-0.001	-0.002
Options Pruned	0/6	0/6	0/12	0/12	5/7	2/12	5/12
Data Use	100.0%	100.0%	0.0%	0.0%	16.7%	100.0%	75.0%
lungcancer	Overall Performance	0.855	0.855	0.852	0.854	0.855	0.861	0.861
Overall Gain	0.001	0.001	-0.002	0.000	0.001	0.006	0.006
n = 120641, d = 84	Group Gains	-0.000	-0.000	-0.000	-0.000	0.000	-0.012	0.001
G = {age, sex}	Max Disparity	0.001	0.001	0.001	0.001	0.000	0.011	0.011
|G| = 6 groups	Rat. Violations	2	2	2	2	1	0	0
Scosyrev et al. [45]	Imputation Risk	-0.000	-0.000
Options Pruned	0/6	0/6	0/12	0/12	4/7	2/12	2/12
Data Use	100.0%	100.0%	0.0%	0.0%	33.3%	100.0%	91.7%
saps	Overall Performance	0.875	0.877	0.875	0.857	0.875	0.960	0.960
Overall Gain	0.010	0.011	0.010	-0.008	0.009	0.095	0.095
n = 7797, d = 36	Group Gains	-0.000	-0.002	-0.000	-0.002	0.000	-0.141	0.035
G = {HIV, age}	Max Disparity	0.017	0.021	0.017	0.021	0.016	0.106	0.106
|G| = 4 groups	Rat. Violations	1	1	1	1	0	0	0
Allyn et al. [3]	Imputation Risk	-0.000	-0.002
Options Pruned	0/4	0/4	0/9	0/9	1/5	2/9	3/9
Data Use	100.0%	100.0%	0.0%	0.0%	75.0%	100.0%	87.5%
Table 4: Overview of performance, data use, and consent for all personalized models and systems on all datasets as measured by test auc. We show the performance of models and systems built using logistic regression.
C.2 Random Forests for Decision-Making (Error)
Dataset	Metrics	STATIC	IMPUTED	PARTICIPATORY
1Hot	mHot	KNN-1Hot	KNN-mHot	Minimal	Flat	Seq
apnea	Overall Performance	26.3%	26.0%	25.9%	27.4%	26.3%	12.2%	12.2%
Overall Gain	1.5%	1.8%	1.9%	0.4%	1.5%	15.6%	15.6%
n = 1152, d = 26	Group Gains	-0.8% -4.2%	0.4% -3.8%	-0.8% -4.2%	0.4% -3.8%	0.0% -4.2%	5.3% -22.2%	5.3% -22.2%
G = {age, sex}	Max Disparity	5.0%	3.4%	5.0%	3.4%	4.2%	16.9%	16.9%
|G| = 6 groups	Rat. Violations	1	0	1	0	0	0	0
Ustun et al. [55]	Imputation Risk	-1.2%	-1.2%
Options Pruned	0/6	0/6	0/12	0/12	2/7	1/12	2/12
Data Use	100.0%	100.0%	0.0%	0.0%	66.7%	100.0%	91.7%
cardio_eicu	Overall Performance	18.6%	17.8%	18.2%	18.6%	18.4%	5.7%	6.0%
Overall Gain	-0.2%	0.6%	0.2%	-0.2%	0.0%	12.7%	12.4%
n = 1341, d = 49	Group Gains	-3.5% -1.4%	-2.2% -3.0%	-3.5% -1.4%	-2.2% -3.0%	0.0% -0.0%	6.0% -14.9%	6.0% -14.9%
G = {age, sex, race}	Max Disparity	4.9%	5.3%	4.9%	5.3%	0.0%	8.9%	8.9%
|G| = 8 groups	Rat. Violations	2	2	2	2	0	0	0
Pollard et al. [44]	Imputation Risk	-3.5%	-2.2%
Options Pruned	0/8	0/8	0/27	0/27	8/9	11/27	8/27
Data Use	100.0%	100.0%	0.0%	0.0%	0.0%	100.0%	91.7%
cardio_mimic	Overall Performance	19.9%	20.1%	19.9%	20.2%	19.6%	11.5%	11.4%
Overall Gain	-0.3%	-0.5%	-0.3%	-0.6%	0.0%	8.1%	8.1%
n = 5289, d = 49	Group Gains	-1.1% -1.3%	-1.3% -0.5%	-1.1% -1.3%	-1.3% -0.5%	0.0% -0.0%	1.0% -14.9%	1.0% -14.9%
G = {age, sex, race}	Max Disparity	2.4%	1.7%	2.4%	1.7%	0.0%	13.8%	13.8%
|G| = 8 groups	Rat. Violations	5	6	5	6	0	0	0
Johnson et al. [32]	Imputation Risk	-1.1%	-1.3%
Options Pruned	0/8	0/8	0/27	0/27	8/9	6/27	5/27
Data Use	100.0%	100.0%	0.0%	0.0%	0.0%	100.0%	87.5%
coloncancer	Overall Performance	37.2%	37.0%	37.2%	37.0%	37.0%	35.9%	35.9%
Overall Gain	-0.2%	0.0%	-0.2%	-0.0%	0.0%	1.0%	1.0%
n = 29211, d = 72	Group Gains	-0.7% -0.1%	-0.3% -0.2%	-0.7% -0.1%	-0.3% -0.2%	0.0% -0.0%	0.1% -3.2%	0.1% -3.2%
G = {age, sex}	Max Disparity	0.7%	0.5%	0.7%	0.5%	0.0%	3.1%	3.1%
|G| = 6 groups	Rat. Violations	4	1	4	1	0	0	0
Scosyrev et al. [45]	Imputation Risk	-0.7%	-0.3%
Options Pruned	0/6	0/6	0/12	0/12	6/7	3/12	5/12
Data Use	100.0%	100.0%	0.0%	0.0%	0.0%	100.0%	75.0%
lungcancer	Overall Performance	20.0%	20.2%	20.0%	20.3%	20.0%	19.3%	19.3%
Overall Gain	0.1%	-0.1%	0.1%	-0.2%	0.1%	0.8%	0.7%
n = 120641, d = 84	Group Gains	-0.3% -0.2%	-0.5% -0.0%	-0.3% -0.2%	-0.5% -0.0%	0.0% -0.2%	0.0% -2.3%	0.0% -2.2%
G = {age, sex}	Max Disparity	0.6%	0.5%	0.6%	0.5%	0.2%	2.3%	2.1%
|G| = 6 groups	Rat. Violations	1	4	1	4	0	0	0
Scosyrev et al. [45]	Imputation Risk	-0.3%	-0.5%
Options Pruned	0/6	0/6	0/12	0/12	3/7	1/12	3/12
Data Use	100.0%	100.0%	0.0%	0.0%	50.0%	100.0%	83.3%
saps	Overall Performance	14.1%	15.0%	14.1%	15.7%	13.9%	9.8%	9.8%
Overall Gain	0.9%	-0.0%	0.9%	-0.7%	1.1%	5.2%	5.2%
n = 7797, d = 36	Group Gains	-0.8% -3.4%	-0.5% -0.3%	-0.8% -3.4%	-0.5% -0.3%	0.0% -3.4%	0.0% -16.4%	0.0% -16.4%
G = {HIV, age}	Max Disparity	4.2%	0.8%	4.2%	0.8%	3.4%	16.4%	16.4%
|G| = 4 groups	Rat. Violations	1	1	1	1	0	0	0
Allyn et al. [3]	Imputation Risk	-0.8%	-0.7%
Options Pruned	0/4	0/4	0/9	0/9	2/5	1/9	1/9
Data Use	100.0%	100.0%	0.0%	0.0%	50.0%	75.0%	87.5%
Table 5: Overview of performance, data use, and consent for all personalized models and systems on all datasets as measured by test error. We show the performance of models and systems built using random forests.
C.3 Random Forests for Ranking (AUC)
Dataset	Metrics	STATIC	IMPUTED	PARTICIPATORY
1Hot	mHot	KNN-1Hot	KNN-mHot	Minimal	Flat	Seq
apnea	Overall Performance	0.825	0.824	0.822	0.806	0.823	0.944	0.942
Overall Gain	0.008	0.006	0.004	-0.012	0.005	0.126	0.124
Group Gains	-0.004 - 0.009	-0.005 - 0.012	-0.004 - 0.009	-0.005 - 0.012	0.000 - 0.009	0.058 - 0.157	0.058 - 0.157
G = {age, sex}	Max Disparity	0.012	0.017	0.012	0.017	0.009	0.098	0.098
Rat. Violations	2	3	2	3	0	0	0
|G| = 6 groups	Imputation Risk	-0.004	-0.005
Options Pruned	0/6	0/6	0/12	0/12	3/7	2/12	4/12
Ustun et al. [55]	Data Use	100.0%	100.0%	0.0%	0.0%	50.0%	100.0%	75.0%
cardio_eicu	Overall Performance	0.896	0.896	0.897	0.886	0.894	0.987	0.987
Overall Gain	0.003	0.003	0.004	-0.007	0.001	0.094	0.094
Group Gains	-0.008 - 0.011	-0.005 - 0.011	-0.008 - 0.011	-0.005 - 0.011	0.000 - 0.004	0.010 - 0.132	0.010 - 0.130
G = {age, sex, race}	Max Disparity	0.020	0.016	0.020	0.016	0.004	0.122	0.120
Rat. Violations	3	4	3	4	0	0	0
|G| = 8 groups	Imputation Risk	-0.008	-0.005
Options Pruned	0/8	0/8	0/27	0/27	7/9	10/27	10/27
Pollard et al. [44]	Data Use	100.0%	100.0%	0.0%	0.0%	12.5%	100.0%	87.5%
cardio_mimic	Overall Performance	0.884	0.883	0.884	0.881	0.885	0.955	0.954
Overall Gain	0.000	-0.001	0.001	-0.002	0.001	0.071	0.071
Group Gains	-0.005 - 0.006	-0.006 - 0.013	-0.005 - 0.006	-0.006 - 0.013	0.000 - 0.006	0.016 - 0.108	0.016 - 0.107
G = {age, sex, race}	Max Disparity	0.011	0.019	0.011	0.019	0.006	0.092	0.090
Rat. Violations	3	7	3	7	0	0	0
|G| = 8 groups	Imputation Risk	-0.005	-0.006
Options Pruned	0/8	0/8	0/27	0/27	5/9	6/27	6/27
Johnson et al. [32]	Data Use	100.0%	100.0%	0.0%	0.0%	37.5%	100.0%	83.3%
coloncancer	Overall Performance	0.684	0.682	0.681	0.680	0.683	0.696	0.696
Overall Gain	0.002	0.000	-0.001	-0.002	0.001	0.014	0.014
Group Gains	-0.002 - 0.004	-0.004 - 0.002	-0.002 - 0.004	-0.004 - 0.002	0.000 - 0.004	0.004 - 0.035	0.004 - 0.031
G = {age, sex}	Max Disparity	0.006	0.007	0.006	0.007	0.004	0.030	0.026
Rat. Violations	0	0	0	0	0	0	0
|G| = 6 groups	Imputation Risk	-0.002	-0.004
Options Pruned	0/6	0/6	0/12	0/12	3/7	2/12	5/12
Scosyrev et al. [45]	Data Use	100.0%	100.0%	0.0%	0.0%	50.0%	100.0%	75.0%
lungcancer	Overall Performance	0.849	0.849	0.848	0.849	0.848	0.856	0.856
Overall Gain	0.002	0.001	0.001	0.001	0.000	0.008	0.008
Group Gains	-0.001 - 0.003	-0.001 - 0.002	-0.001 - 0.003	-0.001 - 0.002	0.000 - 0.003	0.002 - 0.020	0.002 - 0.020
G = {age, sex}	Max Disparity	0.004	0.003	0.004	0.003	0.003	0.018	0.018
Rat. Violations	1	1	1	1	0	0	0
|G| = 6 groups	Imputation Risk	-0.001	-0.001
Options Pruned	0/6	0/6	0/12	0/12	2/7	1/12	2/12
Scosyrev et al. [45]	Data Use	100.0%	100.0%	0.0%	0.0%	66.7%	100.0%	91.7%
saps	Overall Performance	0.921	0.922	0.922	0.906	0.921	0.966	0.966
Overall Gain	0.003	0.004	0.003	-0.012	0.002	0.048	0.048
Group Gains	-0.002 - 0.010	-0.002 - 0.013	-0.002 - 0.010	-0.002 - 0.013	0.000 - 0.010	0.009 - 0.109	0.009 - 0.109
G = {HIV, age}	Max Disparity	0.012	0.015	0.012	0.015	0.010	0.100	0.100
Rat. Violations	2	2	2	2	0	0	0
|G| = 4 groups	Imputation Risk	-0.002	-0.002
Options Pruned	0/4	0/4	0/9	0/9	2/5	2/9	2/9
Allyn et al. [3]	Data Use	100.0%	100.0%	0.0%	0.0%	50.0%	100.0%	87.5%
Table 6: Overview of performance, data use, and consent for all personalized models and systems on all datasets as measured by test auc. We show the performance of models and systems built using random forests.
D Supporting Material for Performance Profiles
In the performance profiles, we measure the benefit of disclosure in terms of their expected performance gain and simulate the cost of reporting for each individual by sampling their reporting cost from a uniform distribution - i.e., for each individual  $i$ , we sample  $c_{i}$  as  $c_{i} \sim \mathrm{Uniform}(0, \gamma)$ , where  $\gamma \in [0, 0.2]$ . For each value of  $\gamma$ , we sample reporting costs ten times and average over the per group performance error for each sampled cost.