SlowGuess/NAACL_2021_OCR huggingface dataset:
Task	Description	Prevalence
IHM	In-Hospital Mortality	0.13
AAURF	Acute and unspecified renal failure	0.21
ACD	Acute cerebrovascular disease	0.07
AMI	Acute myocardial infarction	0.11
CD	Cardiac dysrhythmias	0.32
CKD	Chronic kidney disease	0.13
COPDAB	Chronic obstructive pulmonary disease and bronchiectasis	0.13
COSPOMC	Complications of surgical procedures or medical care	0.2
CD-2	Conduction disorders	0.07
CHFN	Congestive heart failure; nonhypertensive	0.28
CAAOHD	Coronary atherosclerosis and other heart disease	0.33
DMWC	Diabetes mellitus with complications	0.1
DMWC-2	Diabetes mellitus without complication	0.19
DOLM	Disorders of lipid metabolism	0.27
EH	Essential hypertension	0.41
FAED	Fluid and electrolyte disorders	0.25
GH	Gastrointestinal hemorrhage	0.07
HWCASH	Hypertension with complications and secondary hypertension	0.13
OLD	Other liver diseases	0.08
OLRD	Other lower respiratory disease	0.04
OURD	Other upper respiratory disease	0.04
PPPC	Pleurisy; pneumothorax; pulmonary collapse	0.08
PTCBTOSTD	Pneumonia (except that caused by tuberculosis or sexually transmitted disease)	0.14
RFIA	Respiratory failure; insufficiency; arrest (adult)	0.18
SIL	Septicemia (except in labor)	0.14
S	Shock	0.07
Table A.1: Clinical prediction tasks along with the respective prevalence
B Fine-tuning Experiments
To illustrate the impact that random seeds can have on measures of algorithmic fairness, we replicate the experiments concerning the fairness of finetuned clinical classifiers reported in (Zhang et al.,
Figure A.1: Correlations between overall performance and subgroup performance on the Shock phenotype classification task, evaluated on a subset of the test data with equal sample sizes for all demographic subgroups.
2020) $^4$ . The study compares 3 measures of multi-group fairness (recall gap, parity gap and specificity gap) with respect to protected attributes such as the gender, language, ethnicity and insurance status. The experiments were conducted over 57 downstream classification tasks: including the IHM and PC tasks we considered in this paper, 3 additional tasks derived from logical ORs on subsets of the PC tasks, and a variation of PC using only the first note. Table 4 in (Zhang et al., 2020) reports the number of tasks with statistically significant gaps and the percentage of significant tasks which favor a subgroup.
We use the same experimental setup and implementation<sup>5</sup> to replicate the experiments for IHM and PC using only the first note (29/57 tasks). We repeat each experiment with 100 different random seeds (using the same seed to shuffle the data and initialize parameters) and compute the mean and standard deviation of each measurement across seeds (Table B.1). We find that in general the number of tasks with significant differences is roughly half of those reported by (Zhang et al., 2020), which was expected since we considered half of the tasks. However, we also observe that changes in a single random seed can affect the disparities across protected groups, both in terms of the number and the magnitude of the gaps. We see that there can be variations of up to two tasks with significant gaps and differences of up  $31\%$  in the percentage of tasks favoring a specific group.
4see Section 5.2 and Tables 1 and 4.
<sup>5</sup>https://github.com/MLforHealth/rtfulWords
Acute and unspecified renal failure
Acute cerebrovascular disease
Acute myocardial infarction
Cardiac dysrhythmias
Chronic obstructive pulmonary disease and bronchiectasis
Complications of surgical procedures or medical care
Conduction disorders
Congestive heart failure; nonhypertensive
Coronary atherosclerosis and other heart disease
Diabetes mellitus with complications
Diabetes mellitus without complication
Disorders of lipid metabolism
Essential hypertension
Fluid and electrolyte disorders
Figure A.2: Differences relative to overall performance as a function of random seeds for each subgroup
Hypertension with complications and secondary hypertension
Pneumonia (except that caused by tuberculosis or sexually transmitted disease)
Figure A.3: Differences relative to overall performance as a function of random seeds for each subgroup
Figure A.4: Correlations between overall performance and subgroup performance  $\Delta$
Fluid and electrolyte disorders
Figure A.5: Correlations between overall performance and subgroup performance  $\Delta$
Other lower respiratory disease
Other upper respiratory disease
Pleurisy; pneumothorax; pulmonary collapse
Respiratory failure; insufficiency; arrest (adult)
Septicemia (except in labor)
In Hospital Mortality
Figure A.6: Correlations between overall performance and subgroup performance  $\Delta$
Acute cerebrovascular disease
Chronic kidney disease
Chronic obstructive pulmonary disease and bronchiectasis
Congestive heart failure; nonhypertensive
Complications of surgical procedures or medical care
Coronary atherosclerosis and other heart disease
Essential hypertension
Figure A.7: Differences relative to overall performance as a function of random seeds for each subgroup. Each point represents a run for a pair of seeds with validation performance similar to that of the best seeds.
Septicemia (except in labor)
Hypertension with complications and secondary hypertension
In Hospital Mortality
Pneumonia (except that caused by tuberculosis or sexually transmitted disease)
Figure A.8: Differences relative to overall performance as a function of random seeds for each subgroup. Each point represents a run for a pair of seeds with validation performance similar to that of the best seeds.
Recall Gap	Parity Gap	Specificity Gap
Gender	Male vs. Female (% Tasks Favoring Male)	3.0 ± 1.2 (67.8 ± 25.2%)	11.2 ± 2.0 (39.0 ± 8.1%)	9.5 ± 2.0 (76.8 ± 10.3%)
Language	English vs. Other (% Tasks Favoring Other)	3.1 ± 1.4 (50.3 ± 28.9%)	6.9 ± 1.9 (5.4 ± 7.0%)	4.2 ± 1.5 (88.9 ± 14.8%)
Ethnicity	White vs. Other (% Tasks Favoring White)	2.5 ± 1.5 (93.0 ± 22.0%)	7.3 ± 1.7 (92.6 ± 10.7%)	4.9 ± 1.5 (11.1 ± 12.1%)
Black vs. Other (% Tasks Favoring Black)	3.8 ± 1.5 (37.9 ± 21.2%)	6.6 ± 1.7 (65.6 ± 15.0%)	3.6 ± 1.4 (40.8 ± 22.4%)
Hispanic vs. Other (% Tasks Favoring Hispanic)	5.1 ± 1.6 (8.4 ± 10.9%)	7.6 ± 1.8 (0.0 ± 0.0%)	9.3 ± 1.8 (99.1 ± 8.9%)
Asian vs. Other (% Tasks Favoring Asian)	6.1 ± 1.5 (53.6 ± 15.8%)	2.3 ± 1.3 (77.1 ± 31.0%)	3.6 ± 1.5 (54.3 ± 25.0%)
Other vs. Other (% Tasks Favoring Other)	10.0 ± 1.7 (6.0 ± 4.9%)	2.6 ± 1.1 (1.0 ± 5.0%)	4.1 ± 1.2 (94.5 ± 12.5%)
Insurance	Medicare vs. Other (% Tasks Favoring Medicare)	15.0 ± 2.0 (93.8 ± 9.5%)	25.7 ± 2.5 (92.2 ± 8.6%)	23.9 ± 2.6 (2.9 ± 2.6%)
Private vs. Other (% Tasks Favoring Private)	7.1 ± 1.4 (10.5 ± 9.0%)	19.5 ± 2.2 (4.2 ± 3.3%)	19.7 ± 2.4 (95.5 ± 9.0%)
Medicaid vs. Other (% Tasks Favoring Medicaid)	9.0 ± 1.7 (8.7 ± 7.8%)	17.2 ± 2.1 (12.0 ± 3.3%)	15.0 ± 2.1 (92.8 ± 9.3%)
Table B.1: We replicated Zhang et al. (2020) analysis of multi-group fairness performance gaps for fine-tuned classifiers across gender, language, ethnicity, and insurance status. We evaluated 28 (out of 57) tasks and repeated the experiments with 100 different random seeds. We measured the average and standard deviation of the number of tasks with statistically significant differences, and the percentage of significant tasks which favor a subgroup.