Proportional Multicalibration
Authors: William La Cava
Elle Lett
Guangya Wan
Venue: Submitted to ICLR 2023 (2023)
Keywords: fairness
calibration
risk prediction
multicalibration
clinical decision support

Abstract: Multicalibration is a desirable fairness criteria that constrains calibration error among flexibly-defined groups in the data while maintaining overall calibration. However, when outcome probabilities are correlated with group membership, multicalibrated models can exhibit a higher percent calibration error among groups with lower base rates than groups with higher base rates. As a result, it remains possible for a decision-maker to learn to trust or distrust model predictions for specific groups. To alleviate this, we propose proportional multicalibration, a criteria that constrains the percent calibration error among groups and within prediction bins. We prove that satisfying proportional multicalibration bounds a model's multicalibration as well its differential calibration, a stronger fairness criteria inspired by the fairness notion of sufficiency. We provide an efficient algorithm for post-processing risk prediction models for proportional multicalibration and evaluate it empirically. We conduct simulation studies and investigate a real-world application of PMC-postprocessing to prediction of emergency department patient admissions. We observe that proportional multicalibration is a promising criteria for controlling simultenous measures of calibration fairness of a model over intersectional groups with virtually no cost in terms of classification performance.  

Under review as a conference paper at ICLR 2023
PROPORTIONAL MULTICALIBRATION
Anonymous authors Paper under double-blind review
ABSTRACT
Multicalibration is desirable fairness criteria that constrains calibration error among flexibly-defnned groups in the data while maintaining overall calibra- tion. However, when outcome probabilities are correlated with group member- ship, multicalibrated models can exhibit a higher percent calibration error among groups with lower base rates than groups with higher base rates. As result, it re- mains possible for a decision-maker to learn to trust or distrust model predictions for specific groups. To alleviate this, we propose proportional multicalibration, criteria that constrains the percent calibration error among groups and within prediction bins. We prove that satisfying proportional multicalibration bounds a model's multicalibration criteria inspired by the fairness notion of sufficiency. We provide an efficient algo- rithm for post-processing risk prediction models for proportional multicalibration and evaluate it empirically. We conduct simulation studies and investigate a real- world application of PMCC-potprrocessing to prediction of emergency department patient admissions. We observe that proportional multicalibration is a promising criteria for controlling simultenous measures of calibration fairness of a model over intersectional groups with virtually no cost in terms of classification perfor- mance.
well its differential calibration, stronger faimess
INTRODUCTION
Today, machine learning (ML) models have an impact on outcome disparities across sectors (health, lending, criminal justice) due to their wide-spread use in decision-makinn making. When applied in clin- ical meaking MI. models help care providers decide whom to prioritize to receive finite and time-sensitive resources among a population of potentially very ill patients. These resources include hospital beds (Barak-Corre et al., 2021a; Dinh & Berendsen Russell, 2021), organ trans- plants (Schnellinger et al., 2021), specialty treatment programs (Henry et al., 2015; Obermeyer et al., 2019), and, recently, ventilator and other breathing support tools to manage the COVID-19 pandemic (Riviello et al., 2022).
In scenarios like these, decision makers typically rely risk prediction models to be calibrated. Calibration measures the extent to which a model's ' risk scores, R, match the observed probability of the event, y. Perfect calibration implies that P(y R r) r. for all values of Calibration allows the risk scores to be used to rank patients in order of priority and informs care providers about the urgency of treatment. However, models that are not equally calibrated among subgroups defined by different sensitive attributes (race, ethnicity, gender, income, etc.) may lead to systematic denial of resources marginalized groups (e.g. (Obermeyer et al., 2019; Ashana al., 2021; Roberts, 2011; Zelnick et al., 2021; Ku et al., 2021)). Just this scenario was observed by Obermeyer et al. (2019) analyzed a large health system algorithm used to enroll high-risk patients into care management programs and showed that, at given risk score, Black patients exhibited significantly poorer health than white patients.
To address equity in calibration, Hebert-Johnson et al. (2018) proposed a faimess measure called multicalibration (MC), which asks that calibration be satisifed simultaneously over many flexibly- defined subgroups. Remarkably, MC can be satisfied efficiently by post-processing risk scores with- out negatively impacting the generalization error a model, unlike other fairness concepts like de- mographic parity (Foulds & Pan, 2020) and equalized odds (Hardt et al., 2016). This has motivated the use of MC in practical settings (c.g. Barda et al. (2021)) and has spurred several extensions (Kim al., 2019; Jung et al., 2021; Gupta et al., 2021; Gopalan et al., 2022). If we bin our risk predictions,

---

Under review as a conference paper at ICLR 2023
the MC criteria specifies that, for every group within each bin, the absolute difference between the mean observed outcome and the mean of the predictions should be small.
As Barocas et al. (2019) note, equity in calibration embeds the faimess notion called sufficieney, which states: for a given risk prediction, the expected outcome should be independent of group membership. Starting from this notion, can assess the conditions under which MC satisfies sufficiency. In this work, we derive a fairness criteria directly from sufficiency dubbed differential calibration for its relation to differential fairness (Foulds et al., 2019b). We show that satisfying differential calibration can ensure that model is equally "trustworthy" among groups in the data. By equally "trustworthy". we mean that a decision maker cannot reasonably come distrust the model's 's risk predictions for specific groups, which may help prevent differences in decision- between demographic groups, given the same risk prediction.
By relating sufficiency to MC, we describe shortcoming of MC that can occur when the outcome probabilities are strongly tied to group membership. Under this condition, the amount of calibration error relative to the expected outcome can be unequal between groups. This inequality hampers the ability of MC to (approximately) guarantee sufficiency, and thus guarantee equity trustworthiness for the decision maker.
We propose a simple variant of MC called preportional multicalibration (PMC) that ensures that the proportion of calibration error within each bin and group small. We prove that PMC bounds both multicalibration and differential calibration. We show that PMC can be satisfied with an efficinet post-processing method, similarly to MC.
1.1 OUR CONTRIBUTIONS
In this manuscript, we formally analyze the connection of MC the fairness notion of sufficiency. To do so, we introduce differential calibration (DC), sufficiency measure that constrains ratios of population risk between pairs of groups within prediction bins. We describe how DC, like suffi- ciency, provides sense of equal trustworthiness from the point of view of the decision maker. With this definition, we prove the following. First, models that are a-multicalibrated satisfy (In [m )-
DC, where Tmin is the minimum expected risk prediction among categories defined by subgroups and prediction intervals. We illustrate the meaning of this bound, which is that the proportion of calibration error in multicalibrated models may scale inversely with the outcome probability.
Based these observations, age error by group and risk strata (Definition 5). We show that models satisfying a-PMC are (1a.) muutiiccallibr and (In I -differentially calibrated. Proportionally multicalibrated mod- els thereby obtain robust fairness guarantees that are less dependent population risk categories. Furthermore, we define an efficient algorithm for learning predictors satisfying a-PMC.
propose variation of MC, PMC, that controls the percent-
Finally, we investigate the application of these methods to predicting patient admissions in the emer- gency department, a real-world resource allocation task, and show that post-procsssing for PMC results in models that accurate, multicalibrated, and differentially calibrated.
2 RECONCILING MULTICALBRATTON AND SUFFICIENCY
2.1 PRELIMINARIES
We consider the task of training a risk prediction model for population of individuals with outcomes, {0, 1}. and features, E X. Let D be the joint distribution from which individual samples (y, .) are drawn. We assume the outcomes y are random samples from underlying independent Bernoulli distributions, denoted p" p²xx) [o, 1]. Individuals can be further grouped into collections of subsets, C C xx such that S Cs is the subset of individuals belonging to S, and x E S indicates that individual x belongs to group S.
We denote our risk prediction model as R(x) X - [0. 1]. In order to consider calibration in prac- tice, the risk predictions are typically discretized and considered within intervals. The coarseness of this interval is parameterized by partitioning parameter, X E (0, 1). The X-discretization of (0, 1]
2

---

Under review as a conference paper at ICLR 2023
is denoted by a set of intervals, AA = [411/4011 where 1, = [id, (i + 1)A). For brevity, some proofs in the following sections are given in Appendix A.2.4.
2.2
MULTICALLIBRATION
MC (Hébert-Johnson et al., 2018) guarantees that the calibration error for any group from collection of subsets, C' will not exceed user-defined threshold, over the range of risk scores.
Definition 1 (a-calibration (Hébert-Johnson al., 2018)). Let S X. For a [0, 1). R is a- calibrated with respect to S if there exists some S S with |S 2 (1 a) |S| such that for all r E 1).
-
Definition 2 (a-MC (Hébert-Johnson et al., 2018)). Lei C C 2x be collection of subsets of X, a [0, 1]. A predictor R is a-multicalibrated on C if for all C, is a-calibrated with respect to S.
We note that, according to Definition 1, a model need only be calibrated over a sufficiently large subset of each group (S') in order to satisfy the definition. This relaxation is used to maintain satisfactory definition MC when working with discretized predictions. For simplicity, we conduct most of our analysis using the continuous versions of fairenss definitions like Definition 2 (see Appendix A.2.1 for an extended discussion).
MC is one of few approaches to achieving faimeess that does not require significant trade-off to be made between a model's generalization error and the improvement in fairness it provides (Hébert- Johnson et al., 2018). As Hébert-Johnson et al. (2018) show, this is because achieving multicali- bration is not at odds with achieving accuracy in expectation for the population as a whole. This separates calibration fairness from other fairness constraints like demographic parity and equalized odds (Hardt et al., 2016), both of which may denigrate the performance of the model on specific groups (Chouldechova, 2017; Pleiss et al., 2017). In clinical settings, such trade-off's may be difficult or impossible to justify. In addition to its alignment with accuracy in expectation, Hébert-Johnson al. (2018) propose an efficient post-processing algorithm for MC similar on boosting. We discuss additional extensions to MC in Appendix A.
2.3 SUFFICIENCY AND DIFFERENTIAL CALIBRATION
MC provides sense of faimesss by approximating calibration by group, which is perfectly satisfied when Pn R = r,t E S) r for all $ E C and E [0, 1]. Calibration by group is closely related the sufficiency fairness criterion (Barocas al., 2019). Sufficiency is the condition where the outcome probability is independent from C conditioned on the risk score. In the binary group setting (C (S, S,)), sufficiency can be expressed (, S,) = (y E 8,),
Pply x E S;)
Pp R, x E S,)
Unlike calibration by group, sufficiency does not stipulate that the risk scores be calibrated, yet from a fairness perspective, sufficiency and calibration-by-grot Consider that one can easily transform a model satisfying sufficiency into one that calibrated-by- group with single function f(R) + [o. 1]. for example with Platt scaling (Barocas et al., 2019). In both cases, the sense of fairnesy stems from the desire for capture everything about group membership that is relevant to predicting y-
are equivalent (Barocas et al., 2019).
Under sufficiency, the risk score is equally informative of the outcome, regardless of group mem- bership. In this sense, a model satisfying sufficiency provides equally trustworthy risk predictions to a decision maker, regardless of the groups to which an individual belongs. Below, we define an approximate measure of sufficiency that constrains pairwise differentials between groups, and accomodates binned predictions:

---

Under review as a conference paper at ICLR 2023
Definition 3 (e-Differential Calibration). Let C S 2 be collection of subsets of X. A model R(x) is e-differnntialy calibrated with respect to C if. for all pairs (,, E x C for which Pn((.). > 0. for any E [0, 1..
Eply R T, S.] Enly R r E Sj
Se
By inspection we see that in e-DC measures the extent which satisifies sufficiency. That is, when P(y R = r, E S,) s P(yR r, E Sj) for all pairs (S₁, Sy), z O. e-DC says that, within any bin of risk scores, the outcome y is at most ef times more likely among one group than another, and a minimum of e less likely. Definition 13 fits into the general definition of differential fairness measure proposed by Foulds et (2019a), although previously was used to define demographic parity criteria (Foulds & Pan, 2020). We describe the relation in more detail Appendix A.I.1, including Eq. (2)'s connection to differential privacy Dwork & Lei (2009) and pufferfish privacy Kifer & Machanavajjhala (2014).
2.4 THE DIFFERENTIAL CALIBRATION OF MULTICALIBRATED MODELS IS LIMITED BY LOW-RISK GROUPS
At basic level, the form of MC and sufficiency differ: MC constrainins absolute differences be- tween groups across prediction bins, whereas sufficiency constrains pairwise differentials between groups. To reconcile MC and DC/sufficiency more formally, we pose the following question: if a model satisfies a-MC, what, anything does this imply about the E-DC of the model? (In Ap- pendix A.3, L.S.Theremm4 4, we answer the inverse question). We now show that multicalibrated models have a bounded DC, but that this bound is limited by small values of R.
Theorem 1. Let R(x) be a model satisfying a-MC on collection of subsets 2* Let Tmin = minsec En [ r, E S) be the minimum expected risk prediction among S C and E [0, 1]. Then R(x) is (In to +o:) -diferrnniaaly calibrated.
Tmin -a
T E S] and p³ = Eply R = r, E S]. a-MC guarantees that a p" < r a for all groups E C and predictions r E [0, 1]. Plugging these lower and upper bounds into Eq. (2), we observe that the lower bound on e DCDC for R(x) is given by rie < e The maximum of the left-hand side for a fixed a occurs at the smallest value of r; therefore R(x) satisfies In tmintt < e. By switching the numeratator and denominator we obtain the minimum Posco -a
Proof. Let r = ED[
differential and the left-hand side constraint from Definition 13, i.e. e Tmin
Thus R(x) is
In Emiin Tmmin
differentially calibrated.
L
Theorem 1 illustrates the important point that, in terms of percentage error, MC does not provide equal protection to groups with different risk profiles. Imagine a model satisfying (0.05)-MC for groups S C. Consider individuals receiving model predictions R(x) = 0.9. MC guarantees that, for any category {x x E S, R(rr) = 0.9}. ). the expected outcome probability is at least 0.9 0.85 and at most 0.9 + a 0.95. This bounds the percent error among groups with this prediction to about 6%. In contrast, consider individuals for whom R(x) = 0.3; each group may have true outcome probability as low as 0.25, which is an error of 20% about 3.4x higher than the percent emor in the higher-risk group.
PROPORTIONAL MULTICALIBRATION
We are motivated to define a measure that is efficiently learnable like MC (Definition 2) but better aligned with the multiplicative interpretation of sufficiency, like DC (Definition 13). To do so, we define PMC, a variant of MC that constrains the proportional calibration error of a model among subgroups and risk strata. In this section, show that bounding model's PMC is enough to meaningfully bound DC and MC. Furthermore, we provide an efficient algorithm for satisfying PMC based on simple extension of MC/Multiaccuracy boosting (Kim al., 2019). We begin by
4

---

Under review as a conference paper at ICLR 2023
defining proportional calibration, which expresses calibration error percentage of the outcome probability among a group.
Definition 4 (a- (a-Proportional Calibration). Let S C X. For > 0, R is a-proportionally calibrated with respect to if there exists some S' CS with |S 2 (1 - a)) [S] such that for all E [0,
BbbR= = x E L r S a = = S
Proportional multicalibration is then defined by requiring Definition 4 be satisified among a collec- tion of groups:
Definition (a-Proportional Multicalibration). Lei C 2x be a collection of subsets of X, a > 0, A predictor R is c-proportionally multicalibrated on C if for all S C, R. is a-proportionally calibrated with respect to
We also define discretized version of PMC that is useful for implementing the measure in Algo- rithm and measuring PMC in our experiments.
Definition 6 ((a, X) Pr-Propolli Multicalibration). A model R(x) is (a, p)-proportionally mul- ticalibrated with respect to a collection of subsets C if. for all S E C and I E Ax satisfying Pn([(()) E I| E S) 2 ad,
Enlyl R E 1, E EDIRR 1, EEEEEN S|] aa
(3)
Note that, in practice, we must ensure Eply R E 1, S] 0 for Definition to be satisfied; we later introduce a lower bound, p. prevent the outcome probability from being too small. In Appendix 1, we show (a, A)-PMC meaningfully bounds a-PMC under different discretizations, such that we can minimize (a, A)-PMC achieve low a-PMC.
Comparison to Differential Calibration Rather than constraining the differentials of prediction- and group- specific outcomes among all pairs of subgroups in C x C as DC (Definition 13), PMC constrains the relative error of each group in C. In practical terms, this makes it more efficient to calculate PMC by a factor of O(CC) steps compared additional assumptions about the overall calibration of model in order to imply guarantees of MC, since PMC directly constrains calibration rather than constraining sufficiency alone.
DC. In addition, PMC does not require
Theorem 2. Let R(x) be model satisfying (a)-PMC on collection C. Then R(x) is (In 1ta differentially calibrated.
I-a
Proof. Let r [ = r.
E S) and p* Eplly = r, S). R(x) satisfies a-PMC
(Definition 5), then r1 + a)
IV In p" I r/(1 a).. Solving for the upper bound on c-DC, we
immediately have 2 In r(tt)
Theorem 2 demonstrates that c-proportionally multicalibrated models satisfy a straightforward no- tion of differential fairness that depends monotonically only on a. The relationship between PMC and DC is contrasted with the relationship of MC and DC in Fig. 3, left panel. The figure illustrates how MC's sensitivity to small risk categories limits its DC.
Comparison to Multicalibration Rather than constraining the absolute difference between risk predictions and the outcome in MC, PMC requires that the calibration error be small fraction of the expected risk in each category (S, I). In this sense, it provides a stronger protection than MC by requiring calibration eиror be small fraction regardless of the risk group. In many contexts, we would argue that this is also more aligned with the notion of fairness in risk prediction contexts. Under MC, the underlying probability of an outcome within a group affects the fairness protection that is received (i.e., the percentage error that Definition 2 allows). Because underlying probabili- ties of many clinically relevant outcomes vary significantly among subpopulations, multicalibrated models may systematically permit higher percentage error specific risk groups. The difference in relative calibration error among populations with different risk profiles also translates in weaker suf- ficiency guarantees, demonstrated in Theorem 1. In contrast, PMC provides faimess guarantee that is less dependent on subpopulation risks. In the following theorem, we show that MC is also constrained when model satisfies PMC.
5

---

Under review as a conference paper at ICLR 2023
Algorithm Proportional Multicalibration Post-processing
Require: Prodictor R(x)
1: C 2* such that for all S Pb(S) 2
2: a. A.
3: D= 11
4:
function D.
5: repeat
((v, sample
7:
for S C,I As such that Pb
ady do
8:
S. + --nn R(x) E
9:
18.1 Zzes, f((r)
average group prediction b average subgroop risk
0 - To.. Exes, v(x)
1I:
if S then continuc
12:
13:
14:
if Ar og then
R(a) R(z) Ar x E S,
16:
R(w) (R()))((())...) [0, I))
squash updates to [o.
17:
No Updates R(x) then
break
19:
Theorem 3. Let R(x) be model satisfying a-PMC on collection C. Then R(x) is (T
multicalibrated on C.
The proof of Theorem 3 is given in Appendix A.2.4. This theorem implies that a proportionally calibrated model with sufficiently low will satisfy a similarly low value of MC, whereas model with 0.5-PMC (i.e., 50% calibration error) or higher will not have a meaningful MC guarantee. We further diseuss and illustrate the bounds given by Theorems to and 4 in Appendix A.2.
3.1 LEARNING PROPORTIONALIS MULTICALIBRATED PREDICTORS
So far we have demonstrated that models satisfying PMC exhibit desirable guarantees relative to two previously defined measures of fair calibration, but have not considered whether PMC is possible to learn. Here, we answer in the affirmative by proposing Algorithm to satisfy PMC Algorithm a direct extension of MCBoost(PPistere et al., 2021) which works by checking for calibration errors among groups and prediction intervals that violate the user threshold and adjusting these predictions towards the target. PMC post-processing differs in two main ways: first, updates whenever calibration error is not within oy for all categories, opposed to simply within Second, it ignores updates for categories with low outcome probability (less than p). Next, we prove that PMC post-processing learns an PMC model in polynomial number of steps.
Proposition 1. Define a, A, 7.P > O. Let C C 2 be collection of subsets of X such that, for all S E C, PS(S) > 7. Let R(x) be risk prediction model to be post-processed. For all (S, EC Ax, let Ely|R I, E p. There exists algorithm that satisfies (a, PM-PMC
with respect to C in O (p))p"
steps.
We analyze Algorithm and show it satisfies Proposition in Appendix A.2.4. This more stringent threshold requires additional ((+)) steps, where p 0 is lower bound the expected outcome within a category ().
EXPERIMENTS
In our first set of experiments (Section 4), we study MC and PMC in simulated population data to understand and validate the analysis in previous sections. In the second section, we compare the performance of varied model treatments on a real world hospital admission task, using an imple- mentation of Algorithm 1. We make use of empirical versions of our faimess definitions which refer to as MC loss, PMC loss, and DC loss. In short, these measures calculate the maximum (pro- portional) calibration error or pairwise calibration differential among subgroups and risk categories in the data sample. Due to space constraints the formal definitions are given in Appendix A.2.2 (Definitions 14 to 16).

---

Under review as a conference paper at ICLR 2023
Simulation study We simulate data from a-multicalibrated models. For simplicity, we specify a data structure with a one-to-one correspondence between subset and model estimated risk, such that for all in S, R(z) = R(x| 5) R(S). Therefore all information for predicting the outcome based on the features in x: is contained in the attributes A that define subgroup S. Outcome probability is specified as p. = PD(y) lx E S.) 0.2 (i 1) and = 1,.... Ns. where N, is the number of subsets S. defined by AA and indexed by i with increasing p'. For each group, R. R(S.) R(x x E St) = P A. We randomly select A, for one group be ta and for the remaining groups, A; = =6, where å ~ (min = 0, max = a). In all cases, the sign of A; is determined by a random draw from a Bernoulli distribution. For these simulations we set Ns 61 and 0.1, such that P. E [0.2, 0.8] and R2 E 10.1, 0.9). We generate Naim 1000 simulated datasets, with n = 1000 observations per group, and for each S; we calculate the ratio of the absolute mean enror to p. i.e. the PMC loss function for this data generating mechanism.
We also simulate three specific scenarios where: 1) | is equivalent for all groups (Fixed); 2) |با increases with increasing p.; and 3) |A,| decreases with increasing P; with a 0.1 in each case. These scenarios compare when a is determined by all groups, the group with the lowest outcome probability, and the group with the highest outcome probability, respectively.
Hospital admission Next, we test PMC alongside other methods in application to prediction of inpatient hospital admission for patients visiting the emergency department (ED). The burden of overcrowding and long wait times in EDs is significantly higher among non-white, non- non-Hispanic marginalized patients (James et al., 2005; McDonald et al., 2020). Recent work has demonstrated risk prediction models that can expedite patient visits by predicting patient admission at an early stage of a visit with a high degree of certainty (AUC 2 0.9 across three large care centers) Searak-Corren et al., 2017b;a; 2021b;a). Our goal is to ensure no group of patients will be over- or under-prioritized over another by these models, which could exacerbate the treatment and outcome disparities that currently exist.
patients and
We construct prediction task similar to previous studies but using new data resource: the MIMIC- IV-ED repository (Johnson al., 2021). The overall intersectional demographic statistics for these data are given in Table 1. In Table we observe stark differences in admission rates by demographic group and gender, suggesting that the use of a proportional measure of calibration could be appropri- ate for this task. We trained and evaluated logistic regression (LR) and random forest (RF) models of patient admission, with and without post-procsssing for MC (Pfisterer et al., 2021) or PMC. For each of the parameter settings given in Table 2, we conducted 100 repeat experiments with different shuffles of the data. Comparisons are reported on a test set of 20% of the data for each trial. Ad- ditional experiment details are available in Appendix A.6 and code for the experiments is available here: https: /
wwt.1Tt22IISKKABBO pmc.
5 RESULTS
Fig. 1 shows the PMC loss of a-multicalibrated models under the scenarios described in Section 4. Proportional a-MC constrains the ratio of the absolute mean error (AME) to the outcome prevalence, for groups defined by a risk interval (R(x) E 7) and subset within collection of subsets (x E S, S C). Without the proportionality factor [y] r, E S]I= a-multicalibrated models allow a dependence between the group prevalence and the error or privacy loss permitted that is unfair for groups with lower outcome prevalence.
Results on the hospital admission prediction task are summarized in Fig. 2 and Tables 3 and 4. PMC post-processing has a negligible effect on predictive performance (<0.1% 1% A AUROC, LR and RF) while reducing DC loss by 27% for LR and RF models, and reducing PMC loss by 40% and 79%, respectively. In the case of RF models, PMC post- post-processing reduces MC loss by 23% a significantly larger improvement than MC post-procyssing itself (19%, 269.-26).
Senstivity Analysis We note PMC postprocessing has a lower tolerance for error than MC post- processing for given value of a, since ÿ < 1. Thus a natural question is whether MC can match the performance of PMC simply by specifying smaller a. Further, if a can be made small enough, the calibration error [ E I, E S Ep R E I, x E S] on all categories will be small compared the outcome prevalence, Ep(y R 1, S). meaning MC postprocessing could

---

Under review as a conference paper at ICLR 2023
Table 1: Admission prevalence (Admissions/Totes (%) among patients in the MIMIC-IV-ED data
repository, stratified by the intersection of ethnoracial group and gender.
Gender
F
M
Overall
Biluoorccia Group
American Indian/Alaska Native
70/57 (27%)
82/170 (48%)
152/427 (36%)
Asian
1043/3595 (29%)
1052/2384 (43%)
2075/5979 (35%)
Black/African American
3124/27486 (11%)
2603/14458 (18%)
5727/4 1944 (14%)
Hispanic/Laaie
1063/10262 (10%)
1168/5795 (20%)
2231/16057 (14%)
Other
1232/5163 (24%)
1479/3849 (38%)
2711/9012 (30S)
Unknown/Unmbble to Obtain
1521/2156 (71%)
2074/2337 (87%)
3595/4533 (795)
White
18147/50174 74 (36%)
18951/45435 (42%)
37098/95609 (39%)
Overall
26200999033 (26%)
27389/74468 (37%)
53589/173561 (31%)
Figure The relationship between MC, PMC, and outcome prevalence illustrated via simu- lation study in which the rates of the outcome are associated with group membership. Gray points denote the PMC loss of a (0.1,0.1)-MC model 1000 simulated datasets, and colored lines denote three specific scenarios in which each group's calibration error (|Al) follows specific rules. PMC loss is higher among groups with lower positivity rates in most scenarios unless the groupwise calibration error increases with posi- tivity rate.
MCC
Exod across groupe
pos
mncassss with livliily rale
PMD
0.2
03
0.4
0.6
0.6
Cutcome Prevelence (Poebiiity Rate)
satisfy the desired proportional guarantees of PMC. However, to achieve this performance by MC- postprocessing may require large number of unnecessary updates for high risk intervals, since the DC and PMC of multicalibrated models are limited by low-risk groups (Theorem 1). Further- more, the number of steps in MC-postprocessinn (and PMCC-pstprooeesing) scales as an inverse high-order polynomial of a (cf. Thm. 2 (Hebert-Johnson et al., 2018)).
We shed light this question in three ways. First, we quantify how often the use of each post- processing algorithm gives the best loss for each metric and trial in Table 3. To account for this, quantified the number of trials for which given method produced the best model according to a given metric, over all parameter configurations in Table 2. PMC post-procsssing (Algorithm 1) achieves the best faimnsss the highest percent of the time, according to DC loss (63%), MC loss (70%), and PMC loss (72%), while MCC-oosproceessed models achieve the best AUROC in 88% of cases. This provides strong evidence that, over large range of values. PMC post-processing is beneficial compared to MC-postprocessingg.
Next, empirically compare MC- and PMC postrossprocessinng by the number of steps required for each to reach their best performance in Fig. 8 and Table 4. We assess how many steps/updates MC and PMC take, and their empirical running time, in Table 4. We observe that PMC typically requires a larger number of updates to achieve its best performance on MC loss (about 2x wall clock time and number of updates), contrast MC-postprocessing requires an average of 5x more updates to achieve its best performance on PMC loss, due to its dependence on very small values of a.
Finally, in Appendix A.6, we look at detailed performance comparisons of MC and PMC postpro- cessing over values of and group definitions in Figs. 5 to 7. We observe that, while low values of a for MC postprocessing improve its PMC loss performance, PMC postprocessing is always able to outperform it for some value of a, and do so in fewer steps.
6 DISCUSSION AND CONCLUSION
In this paper have analyzed multicalibration through the lens of suffiency and differential calibration to reveal the sensitivity of this metric to correlations between outcome rates and group membership. We have proposed measure, PMC, that alleviates this sensitivity and attempts to
8

---

Under review as a conference paper at ICLR 2023
Table 2: Parameters for the hospital admission prediction experiment.
Table 3: The number of times each post- processing method achieved the best score among all methods, of 100 trials.
Parameter Values
(0.001, 0.01, 0.050 0.1) (0.05, 0.1) 0.1
postprocessing Base Model
MC
PMC
X
metric
(0.001, 0.01)
AUROC
88
6
Model Groups
LR, RF
DC loss
36
63
(rracc/ethnicity, gender), (race/ethnicity,
MC loss
21
70
gender, insurance product)]
PMC loss
27
72
AUROC
Multicalibration loss
Propartional Multicalibration loss
Differential Calibration loss
0.860
0.40
1.4
pestpracessing
1.2
0.875
2424
0.30
PNC
0.870
0.20
0.865
0.6
0.4
0.860
0.10
0.2
0 RF UR
RF
RF
LR
Figure 2: A comparison of LR and RF models, with and without MC and PMC post-processing, on the hospital admission task. From left to right, trained models are compared terms test set AUROC, MC loss, PMC loss, and DC loss. Points represent the median performance over 100 shuffled train/test splits with bootstrapped 99% confidence intervals. We test for significant differ- ences between post-processing methods using two-sided Wilcoxon rank-sum tests with Bonferroni correction. ns: p <= 1; *4; lc-03 < <= le-02; #**; lc-04 < <= le-03; #***; <= Ic-04.
capture the "best of both worlds" of MC and DC. PMC provides equivalent percentage calibration protections groups regardless of their risk profiles, and in doing, bounds model's differential calibration. We provide an efficient algorithm for learning PMC predictors by postprocessing given risk prediction model. On real-world and clinically relevant task (admission prediction), have shown that post-processing LR and RF models with PMC leads to better performance across all three fairness metrics, with little to no impact on predictive performance.
Our preliminary analysis suggests PMC can be valuable metric for training fair algorithms in resource allocation contexts. Future work could extend this analysis on both the theoretical and practical side. On the theoretical side, the generalization properties of the PMC measure should be established and its sample complexity quantified, as Rose (2018) did with MC. Additional exten- sions of PMC could establish bound on the accuracy of PMC-postprocessed models in similar vein to work by Kim et al. (2019) and Hébert-Johnson et al.. On the empirical side, future works should benchmark PMC on larger set of real-world problems, and explore cases in more depth.
Best Loss
of Updates
Wall Clock Time (s)
Table For MC- and PMC- Metric
ML.
Posiprocessing
postprocessing, we compare the median number of up- dates and median wall clock time (s) taken to train for the hyperparameter settings (e.g. a) that achieved the best per- formance on each metric.
MC
0.116
32.2
LRR
PMC
0.108
30
58.1
MC loss
MC
0.147
44.3
RF
PMC
0.135
172
79.6
MC
0.334
376
10.6
I.R
PMC
0.287
66.7
PMC
MC
0.356
504
106.3
RF
0.325

---

Under review as a conference paper at ICLR 2023
REFERENCES
Deepshikha Charan Ashana, George I. Anesi, Vincent X Liu, Gabriel Escobar, Christopher Ches- ley, Nwamaka Eneanya, Gary Weissman, William Dwight Miller, Michael O Harhay, and Scott D Halpern. Equitably allocating resources during crises: Racial differences in mortality prediction models. American journal of respiratory and critical care medicine, 204(2)178-186,, 2021.
Nikhil Bansal and Anupam Gupta. Potential-function proofs for gradient methods. Theory of Com- puting, 15(1):1-32, 2019.
Yuval Barak-Corren, Andrew M. Fine, and Ben Y. Reis. Early Prediction Model of Patient Hos- pitalization From the Pediatric Emergency Department. Pediatrics, 139(5), May 2017a. ISSN 1095-4275. doi: 10.1542/peds:2016-2777
Yuval
Shlomo Hanan Israelit, and Ben Y Reis. Progressive prediction of hospi- talisation in the emergency department: Uncovering hidden patterns to improve patient flow. Emergency Medicine Journal, 34(5):308-314, May 2017b. ISSN 1472-0205, 1472-0213. doi: 10. 1336mmmmrmeed-22112-23333
Yuval Barak-Corren, Isha Agarwal, Kenneth A Michelson, Todd W Lyons, Mark Neuman, Susan C Lipsett, Amir A Kimia, Matthew A Eisenberg, Andrew Capraro, Jason A Levy, Joel D Hudgins, Ben Y Reis, and Andrew M Fine. Prediction of patient disposition: Comparison of computer and human approaches and a proposed synthesis. Journal of the American Medical Informatics Association, 28(8):1736-1744 July 2021a. ISSN 1527-974X. doi: 10. 1i..0///aaaaaa/aab06
Yuval Barak-Corren, Pradip Chaudhari, Jessica Perniciaro, Mark Waltzman, Andrew M. Fine, and Ben Y. Reis. Prediction across healthcare settings: A case study predicting emergency de- partment disposition. upj Digital Medicine, 4(1):1-7, December 2021b. ISSN 2398-6352. doi:
10.10388/44774--01-000555
Noam Barda, Gal Yona, Guy N. Rothblum, Philip Greenland, Morton Leibowitz, Ran Balicer, Eitan Bachmat, and Noa Dagan. Addressing bias in prediction models by improving subpopulation calibration. Journal of the American Medical Informatics Association: JAMIA, 28(3):549-558, March 2021. ISSN 1527-974X. doi: 10. 10.1093/jamia//caa25..
Solon Barocas, Moritz Hardt, and Arvind Narayanan. Fairness and Machine Learning. fairml- book.org, 2019.
Richard Berk, Hoda Heidari, Shahin Jabbari, Matthew Joseph, Michael Kearns, Jamie Morgen- stern, Seth Neel, and Aaron Roth. A convex framework for fair regression. arXiv preprint 1.XY::1706.02409.. 2017.
Alessandro Castelnovo, Riccardo Crupi, Greta Greco, and Daniele Regoli. The zoo of Fairness metrics in Machine Learning. arXiv:2106.0044 00466 [cs, stat), June 2021.
Alexandra Chouldechova. Fair Prediction with Disparate Impact: A Study of Bias Recidivism Prediction Instruments. Big Data, ((2):153-163, June 2017. ISSN 2167-647X. doi: 10.1089/big.
2016.0047.
Alexandra Chouldechova and Aaron Roth. arXiv:1810.08810 [cs, stat], October 2018.
The Frontiers of Fairness in Machine Learning.
Patricia Hill Collins. Black Feminist Though: Knowledge, Consciousness, and the Politics of Em- powerment. Routledge, edition, September 1990.
Patricia Hill Collins and Sirma Bilge. Intersectionaliry. John Wiley & Sons, 2020. ISBN 1-5095- 3969-7.
Kimberle Crenshaw. Demarginalizing the Intersection of Race and Sex: A Black Feminist Critique of Antidiscrimination Doctrine, Feminist Theory and Antiracist Politics. University of Chicago Legal Forum, 1989(1):31, 1989.
10

---

Under review as a conference paper at ICLR 2023
Kimberle Crenshaw. Mapping the Margins: Intersectionalitty Identity Politics, and Violence against Women of Color. Stanford Law Review, 43(6):1241, July 1991. ISSN 00389765. doi: 10.2307/ 1229039. URL https::
/wwww jstor wttps//aaaee//223399e2ee2.
Michael M Dinh and Saartje Berendsen Russell. Overcrowding kills: How COVID-19 could reshape emergency department patient flow in the new normal. Emergency Medicine Australasia, 33(1): 175-177, 2021. ISSN 1742-6723.
doi: 10. 10.1111/1742-6722. 137077
Cynthia Dwork and Jing Lei. Differential privacy and robust statistics. In Proceedings of the Forty- First Annual ACM Symposium on Theory of Computing, pp. 371-380, 2009.
Cynthia Dwork, Moritz Hardt, Toniann Pitassi, Omer Reingold, and Richard Zemel. Fairness Through Awareness. In Proceedings of the 3rd Innovations in Theoretical Computer Science Conference, ITCS '2. pp. 214-226, New York, NY, USA, 2012. ACM. ISBN 978- 11881-45003-115-
1. doi: 10.1 241144990236.209000000
Cynthia Dwork, Michael P. Kim, Omer Reingold. Guy N. Rothblum, and Gal Yona. Learning from Outcomes: Evidence- Based Rankings. In 2019 IEEE 60th Annual Symposium on Foundations of Computer Science (FOCS), pp. 106-125, November 2019. doi: 10.1 10.100/FOCSS.00 2019.00016.
James Foulds, Rashidul Islam, Kamrun Naher Keya, and Shimei Pan. An Intersectional Definition of Fairness. arXiv:/807.0833 007.0862700 [cs, stat], September 2019a.
James Foulds, Rashidul Islam, Kamrun Naher Keya, and Shimei Pan. An Intersectional Definition of Fairness. arXiv:1807.08330 [cs, stat], September 2019b. URL htttt://axxi....omm /bb// 1807. 08362.
arXiv: 1807.08362.
James R. Foulds and Shimei Pan. Are Parity-Basee Notions of AI Fairness Desirable? Engineering, pp. 51, 2020.
Data
Parikshit Gopalan, Michael P. Kim, Mihir Singhal, and Shengjia Zhao. Low-Degree Multicalibra- tion. arXiv:2203.01255 [cs], March 2022.
Varun Gupta, Christopher Jung, Georgy Noarov, Mallesh M. Pai, and Aaron Roth. Online Multivalid Learning: Means, Moments, and Prediction Intervals, January 2021.
Alex Hanna, Emily Denton, Andrew Smart, and Jamila Smith-Loud. Towards critical race method- ology in algorithmic fairness. Proceedings of the 2020 Conference on Fairness, Accountabiliy, and 7 Transparency, pp. 501-512, Barcelona Spain, January 2020. ACM. ISBN 978- 177--14503366936 7. doi: 10.1145/3351095.30 333.222
Moritz Hardt, Eric Price, Eric Price, and Nati Srebro. Equality of Opportunity in Supervised Learn- ing. In D. D. Lee, M. Sugiyama, U. V. Luxburg, I. Guyon, and R. Garnett (eds.), Advances in Neural Information Processing Systems 29, pp. 3315-3323, Curran Associates, Inc., 2016.
Úrsula Höbert-Johnson, Michael P Kim, Omer Reingold, and Guy N Rothblum. Multicalibration: Calibration for the cmmpuuaaionalll-Iddenffaai Masses. pp. 10.
Ursula Hebert-Johnson. Michael Kim, Omer Reingold, and Guy Rothblum. Multicalibration: Cal- Masses. In Proceedings of the 35th International
ibration for the (Computationally-ldeniii Conference on Machine Learning, pp. 1948. PMLR, July 2018.
Úrsula Hébert-Johnson, Michael P. Kim, Omer Reingold, and Guy N. Rothblum. Calibration for the (Computatoonally-dennfffi
Masses.
fcs, stat], March 2018.
Katharine E. Henry, David N. Hager, Peter J. Pronovost, and Suchi Saria. targeted real-time early warning score (TREWScore) for septic shock. Science Translational Medicine, 2999a):299ra12) August 2015. ISSN 1946-6242. doi: 10. 126/scitrans//me..aa33711
Catherine A. James, Florence T. Bourgeois, and Michael W. Shannon. Association of Race/Ethnicity with Emergency Department Wait Times. Pediatrics, 115(3))e11-=3315 March 2005. ISSN 0031-4005. doi: 10.1542/peds.2004 2044-1111..

---

Under review as a conference paper at ICLR 2023
Heinrich Jiang and Ofir Nachum. Identifying and Correcting Label Bias in Machine Learning, January 2019.
Alistair Johnson, Lucas Bulgarelli, Tom Pollard, Leo Anthony Celi, Roger Mark, and Steven Horng. MIMIC-IV-ED, 2021.
Christopher Jung, Changhwa Lee, Mallesh Pai, Aaron Roth, and Rakesh Vohra. Moment Multi- calibration for Uncertainty Estimation. In Proceedingg of Thirty Fourth Conference on Learning Theory, pp. 2634-2678, PMLR, July 2021.
Nathan Kallus, Xiaojie Mao, and Angela Zhou. Assessing algorithmic fairness with unohserved protected class using data combination. In Proceedings of the 2020 Conference on Fairness, Ac- countability, and Transparency, FAT* '20, pp. 110, Barcelona, Spain, January 2020. Association for Computing Machinery. ISBN 978-1-4503-6936-7 doi: 10. 1575335500053333355555
Michael Kearns, Seth Neel, Aaron Roth, and Zhiwei Steven Wu, Preventing Fairness Gerrymander- ing: Auditing and Learning for Subgroup Fairness. arXiv: [cs]. December 2018.
Daniel Kifer and Ashwin Machanavajhall. Pufferfish: framework for mathematical privacy definitions. ACM Transactions on Database Systems (TODS), 39(1):1-36, 2014.
Michael P. Kim, Amirata Ghorbani, and James Zou. Multiaccuracy: Black-box post-processing for fairness in classification. Proceedings of the 2019 AAAT/ACM Conference on AI, Ethics, and Sociefy, pp. 247-254, 2019.
Jon Kleinberg. Sendhil Mullainathan, and Manish Raghavan. Inherent trade-offs the fair determi- nation of risk scores. arXiv preprint arXiv: 1609.05877, 2016.
Elaine Ku, Charles E McCulloch, Deborah B Adey, Libo Li, and Kirsten L Johansen. Racial dis- parities in eligibility for preemptive waitlisting for kidney transplantation and modification of eGFR thresholds to equalize waitlist time. Journal of the American Society of Nephrology. 32(3): 677-685, 2021.
Erica J. McDonald, Matthew Quick, and Mark Oremus. Examining the Association between Community-Leve Marginalization and Emergency Room Wait Time in Ontario, Canada. Health- care Policy, 15(4):64-76, May 2020. ISSN 1715-6572. doi: 10. 10.12977hcpo0.222022222.
Ziad Obermeyer, Brian Powers, Christine Vogeli, and Sendhil Mullainathan. Dissecting racial bias in an algorithm used manage the health of populations. Science, 366(6464):447 453, October 2019. ISSN 0036-8075, 1095-9203. doi: 10. 16//science.aax23422
Florian Pfisterer, Christoph Kern, Susanne Dandl, Matthew Sun, Michael P. Kim, and Bernd Bischl. Mcboost: Multi- caltiiCaaibrationn Boosting for R. Journal of Open Source Software, 6(64):3453, 2021.
Geoff Pleiss, Manish Raghavan, Felix Wu, Jon Kleinberg, and Kilian Q Weinberger. On Fairness and Calibration. In I. Guyon, U. V. Luxburg. S. Bengio, H. Wallach, Fergus. S. Vishwanathan, and R. Garnett (eds.), Advances in Neural Information Processing Systems 30, pp. 5680-5689. Curran Associates, Inc., 2017.
Elisabeth D. Riviello, Tenzin Dechen, Ashley L. O'Donoghue, Michael N. Cocchi, Margaret M. Hayes, Rose L. Molina, Nicole H. Moraco, Anne Mosenthal, Michael Rosenblatt, Noa Talmor, Daniel P. Walsh, David N. Sontag, and Jennifer P. Stevens. Assessment of Crisis Standards of Care Scoring System for Resource Prioritization and Estimated Excess Mortality by Race, Ethnicity, and Socially Vulnerable Area During Regional Surge in COVID-19. JAMA Network Open, 5(3) 2244,4, March 2022. ISSN 2574-3805. doi: 10. laaaatt/jaaaanttoropppennn.c 202r 1744.
Dorothy Roberts. Fatal invention: How science, politics, and big business re-create race in the /wenty-first century. New Press/ORIM, 2011. ISBN 1-59558-691-1.
Sheri Rose. Machine learning for prediction electronic health data. JAMA network open, 1(4); e181 11144441114440 1404, 2018.
12

---

Under review as a conference paper at ICLR 2023
Erin M Schnellinger, Edward Cantu, Michael O Harhay, Douglas E Schaubel, Stephen E Kimmel, and Alisa J Stephens- Shields. Mitigating selection bias in organ allocation models. BMC medical research methodology. 21(1):1-9, 2021.
Paula Tanabe, Rick Gimbel, Paul R. Yarnold, Demetrios N. Kyriacou, and James G. Adams. Reli- ability and validity of scores on The Emergency Severity Index version 3. Academic Emergency Medicine: Officcall Journal of the Society for Academic Emergency Medicine, 11(1):59-655 65, Jan-
uary 2004. ISSN 1069-6563. doi: 10.11977j.em.2000000011..
Leila R Zelnick, Nicolae Leca, Bessie Young, and Nisha Bansal. Association of the estimated glomerular filtration rate with vs without a coefficient for race with time to eligibility for kidney transplant. JAMA network open, 4(1):e2034004 1L0:22033004--233404,, 2021.
13

---

Under review as a conference paper at ICLR 2023
A APPENDIX
In this section, include additional comparisons related work, additional definitions, proofs to the theorems in the main text, and additional experimental details. The code to reproduce the figures and experiments available here: https: : / /githtgtt/.///// com wwm/byyltTttIskkkOO... pmc.
A.1 RELATED WORK
There are myriad ways to measure fairness that are covered in more detail in other works (Barocas et al., 2019; Chouldechova & Roth, 2018; Castelnovo et al., 2021). We briefly review three notions here. The first, demographic parity, requires the model's predictions to be independent of patient demographics (A). Although model satisfying demographic parity can be desirable when the outcome should be unrelated to sensitive attributes (Foulds & Pan, 2020), it can be unfair if important risk factors for the outcome are associated with those attributes (Hardt al., 2016). For example, it may be more fair to admit socially marginalized patients to a hospital at a higher rate if they are assessed less able manage their care at home. Furthermore, if the underlying rates of illness vary demographically. requiring demographic parity can result in a healthier patients from one group being admitted more often than patients who urgently need care.
Definitions of Fairness
When the base rates of admission are expected to differ demographically. we can instead ask that the model's errors be balanced across groups. One such notion is equalized odds, which states that for a given Y. the model's 's predictions should be independent of A. Satisfying equalized odds is equivalent to having equal FPR and FNR for every group in A.
When the model is used for patient risk stratification, as in the target use case in this paper, it is important to consider a model's calibration for each demographic group in the data. Because risk prediction models influence who is prioritized for care, an unfairly calibrated model can systemati- cally under-predict risk for certain demographic groups and result under-allocation patient care to those groups. Thus, guaranteeing group- -wiee calibration via an approach such multicalibration also guarantees fair patient prioritization for health care provision. In some contexts, risk predic- tions not directly interpreted, but only used rank patients, which in some contexts is sufficient for resource allocation. Authors have proposed various ways of measuring the faimess of model rankings, for example by comparing AUROC between groups (Kallus et al., 2020).
Approaches to Fairness Many approaches to achieving fairness guarantees according to demo- graphic parity, equalized odds and its relaxations have been proposed (Dwork et al., 2012; Hardt al., 2016; Berk et al., 2017: Jiang & Nachum, 2019; Kearns et al., 2018). When choosing an approach, is important to carefully weigh the relative impact of false positives, false negatives, and miscalibration on patient outcomes, which differ by use case. When group base rates differ (i.e.. group--pecific positivity rates), equalized odds and calibration by group cannot both be satis- fied (Kleinberg et al., 2016). Instead, one can often equalized multicalibration while satisfying relax- ations of equalized odds such equalized accuracy, where Accuracy nr PR+ (1- for a group with base rate μ.. However, to do so requires denigrating the performance of the model on specific groups (Chouldechova, 2017: Pleiss al., 2017), which is unethical in our context.
As mentioned in the introduction, we are also motivated to utilize approaches to fairness that 1) dovetail well with intersectionality theory, and 2) provide privacy guarantees. Most work in the computer science/ machine learning space does not engage with the broader literature on socio- cultural concepts like intersectionality. which we see as a gap that makes adoption in real-world settings difficult (Hanna et al., 2020). One exception to this statement is differential fairess (Foulds et al., 2019a), a measure designed with intersectionality in mind. In addition to being a definition of fairness that provides equal protection to groups defined by intersections of protected attributes, models satisfying e-differential fairness also satisfy e-pufferfish privacy. This privacy guarantee is very desirable in risk prediction contexts, because it limits the extent to which the model reveals sensitive information to a decision maker that has the potential to influence their interpretation of the model's 's recommendation. However, prior work differential faimeess has been limited to using it to control for demographic parity, which is not an appropriate fairness measure for our use case (Foulds & Pan, 2020).
14

---

Under review as a conference paper at ICLR 2023
Multicalibration has inspired several extensions, including relaxations such multiaccuracy (Kim et al., 2019), low-degree multicalibration (Gopalan et al., 2022), and extensions to conformal pre- diction and online learning (Jung et al., 2021; Gupta al., 2021). Noting that multicalibration is guarantee over mean predictions on a collection of groups C, Jung et al. (2021) propose to extend multicalibration higher-order moments (e.g., variances), which allows dence interval for the calibration error for each category. Gupta et al. (2021) extend this idea and generalize the online learning context, in which an adversary chooses sequence of examples for which one wishes to quantify the uncertainty of different statistics of the predictions. Recent work has also utilized higher order moments to "interpolate" between the guarantees provided by multiaccuracy, which only requires accuracy in expectation for groups in C, and multicalibration, which requires accuracy in expectation at each prediction interval (Kim et al., 2019). Like propor- tional multicalibration (Definition 5), definitions of multicalibration for higher order moments pro- vide additional criteria for quantifying model performance over many groups: in general, however, much of the focus in other work is on statisties for uncertainty estimation. Like these works, one may view our proposal for proportional multicalibration as alternative definition of what it means to be multicalibrated. The key difference is that proportional multicalibration measures the degree to which multicalibration depends on differences in outcome prevalence between groups, and in doing so provides guarantees of pufferfish privacy and differential calibration.
to estimate a confi-
Dwork et al. (2019) study the relation of fair rankings to multicalibration, and, in a similar vein to differential fairness measures, formulate fairness measure for group rankings using the relations between pairs of groups. However, these definitions are specific to the ranking relation between the groups, whereas differential calibration cares only about the outcome differential (conditioned on model predictions) between pairs of groups
A.1.1 DIFFERENTIAL FAIRNESS
DF was explicitly defined to be consistent with the social theoretical framework of intersectional- ity. This framework dates back as early as the social movements of the '60s and '70s (Collins & Bilge, 2020) and was brought into the academic mainstream by pioneering work from legal scholar Kimberlé Crenshaw (Crenshaw, 1989; 1991) and sociologist Patricia Hill Collins (Collins, 1990). Central to intersectionality is that hicrarchies of power and oppression are structural elements that fundamental to our society. Through an intersectional lens, these power structures viewed as interacting and co-constituted, inextricably related to one another. To capture this viewpoint, DF (Foulds et al., 2019a) constrains the differential of general data mechanism among all pairs of groups, where groups are explicitly defined as the intersections of protected attributes in A.
Definition 7 (E-differential fairness (Foulds et al., 2019a)). Let θ denote set of distributions and let x ~ 0 for 0 E e. A mechanism M (x) is e-differentially fair with respect to (C,0) for all 6 e e with ~ 8, and m Range(M (M if. for all (S, E C x C where P(S.(0) 0, P(S 0,
e E5 PMM.((M()) = m S)
(MM)(M(()== S) S
(4)
Definition 8 (Pufferfish Privacy). Let the collection of subsets C represent sets of secrets. A mecha- nism M (z) is e-pufferfish private (Kifer & Machanavaijhala, 2014) with respect to (C, 0) for all 0 E θ with 0, for all secret pairs (8,, S3) E C Cand y Range(M).
e c= MM. (M (x) = =y 0)
6
(5)
PM. M y)5,,
when St and Sj are such that P(S, > 0, P(S)()) > O.
Note on pufferfish and differential privacy Although Eq. (4) is notable in its similarity to dif- ferential privacy (Dwork& & Lei, 2009), they differ in important ways. Differential privacy aims to limit the amount of information learned about any one individual in a database by computations per- formed on the data (e.g. (x)). Pufferfish privacy only limits information learned about the group membership of individuals defined by C. Kifer & Machanavajjhala (2014) describe in detail the conditions under which these privacy frameworks are equivalent.
Efficiency Property Foulds et al. (2019a) also define interesting property of e-differential fair- ness that allows guarantees of higher order (i.c., marginal) groups to be met for free; the property is given in Appendix A.2.2.
15

---

Under review as a conference paper at ICLR 2023
Definition 9 (Efficiency Property (Foulds al., 2019a)). Let (x) be an e-differentially fair mech- anism with respect to @). Let the collection of subsets group individuals according to the Cartesian product of attributes A S A. Let G be any collection of subsets that groups individuals by the Cartesian product of attributes in A', where C and A' + 0. Then M(x) is s-differentially fair in (g, ()
The authors call this the "intersectionality property". although its implication is the reverse: if model satisfies e-DF for the low level (i.e. intersectional) groups in C, then satisfies c-DF for every higher-level (i.e. marginal) group. For example, if model is (e)-differentially fair for inter- sectional groupings of individuals by race and sex, then is e-DF for the higher-leves race and sex groupings well. Whereas the number of intersections grows exponentially as additional attributes are protected (Kearns et al., 2018), the number of total possible subgroupings grows at a larger com- binatorial rate: for p protected attributes, we have =k=1 (2)m² groups, where ma is the number of levels of attribute a.
Limitations To date, analysis of DF for predictive modeling has been limited defining R(x) as the mechanism, which is akin to asking for demographic parity. Under demographic parity, one requires that model predictions be independent from group membership entirely, and this limits the utility of it as a faimess notion. Although a model satisfying demographic parity can be desirable when the outcome should be unrelated (Foulds & Pan, 2020), it can be unfair important risk factors for the outcome are associated with demographics (Hardt et al., 2016). For example, if the underlying rates of an illness vary demographically, requiring demographic parity can result a healthier patients from one group being admitted more often than patients who urgently need care.
A.2 EXTENDED THEORETICAL ANALYSIS
Illustrating Relationships between Definitions Fig. shows how the definitions of MC, DC, and PMC are related. In each subplot, the and y coordinates map the guarantee from one metric (x axis) to the implied guarantee in the other metric (y axis).
The right panel of Fig. 3 illustrates this relation in comparison to the DC-MC relationship described in Appendix A.3, Theorem 4. At small values of E and a and when the model is perfectly calibrated overall, a-PMC and E-DC behave similarly. However, given > E-differentially calibrated models suffer from higher MC error than proportionally calibrated models when a-PMC < 0.3. The right graph also illustrates the feasible range of a for a-PMC is 0 < a < 0.5, past which it does not provide meaningful a-MC. The steeper relation between a-PMC and MC may have advantages or disadvantages, depending on context. It suggests that, by optimizing for a-PMC, small improvements to this measure can result in relatively large improvements to MC; conversely, e-DC models that are well calibrated may satisfy a lower value a-MC over a larger range of E.
A.2.1 DISCRETIZATION
To clarify and simplify our analysis, we work mainly with the continuous versions of multicalibra- tion and proportional multicalibration, under the assumption that minimizing the discretized versions (i.e., binning R(x)) will translate low values the continous version. this section we provide detailed bounds on the continuous versions of PMC and DC that are implied by the discretized versions.
First, we will formally define two different discretization schemes. The first, X-discretization, de- fines equally spaced bins on the interval [0, 1]. as follows.
Definition 10 (>-discretization).. Let [0, 1], p E [0, 1]. The X-discretization of [0, 1] is denoted by of intervals, A {{1, 3/2-1 where I, JjA 1)A).
For ensuring multiplicative closeness under PMC, it can be useful to instead discretize the prediction bins so that the bins are equally spaced on a log scale. We define such a discretization below.
Definition 11 ((), peggmmet discretization.. Let À [0, 1].
[0.1]. The (A, p)-gcmeetric
discretization of [o, 1] is denoted by a set of intervals, AS { (1))//11 where 1; (o(1-1 ((1-)1-))
16

---

Under review as a conference paper at ICLR 2023
1.0
10
a-MC model model
c-DC model
0-PMC model
CiOfterenttall Calibration Bound
a coliiitalllattioonn Bound
0.6
G-calibration
0.4
0.2
0.6
0.2 Valuue a.
Figure 3: A comparison of E-DC, a-MC, and a-PMC terms of their parameters a and E. both panes, the value is given value of one metric for model, and the axis is the implied value of the other metric, according to Theorem 4-Theorem 3. The left filled area denotes the dependence of the privacy/DC of o-multicalibrated models the minimum risk interval, Tmin E [0.01, 1.0]- The right filled area denotes the dependence of the MC of e-differentially calibrated models on their overall calibration, ò E [0.0, 0.5]. a-PMC does not have these sensitivities.
Hebert-Johnson et al. (2018) define discretized version of MC in which R(x) is binned according discretization parameter, X:
Definition 12 ((a, N)-multicalibrattonn) Ler 2 be a collection of subsets of X. For any a, x >> 0, predictor R is (o, Al-matticalibratee on C if. for all I E Ax and S E C where PD E Ix E 8) 2 ad,
1, - 1 $a.sa.
Hébert-Johnson et al. (2018) establish that (a, m)-multtiaallb models are at most (a + A)- multicalibrated.
In an analagous fashion, we show below that (a, A)-PMC implies (a + A/p)- for bins defined by >-discretization. When using (A, p)-gmmeerric discretization, (a, PMC implies (ap +
p-> - 1)-PMC, which can be tighter bound than the former.
Claim 1. Define p. az
O and let C C 2 be collection of subsets of X. Let Eply
J E S) 2 for all S E C. and T & Ax- Let R(x) be a model satisfying (a, propporoti multicalibration. Then R(x) is at most (a + >) pr-prpportionally multicalibratte.
Proof. By Definition 6, R(x) satisfies
Ep Ep 1, RD S]|
<a
EDW
for categories (S, E C A satisfying PD(R(x) E T| E 5) 2 ad. Given 1/A bins, the subset where Pp E 1| E 5) < aA has size at most aSS). Therefore there is a subset |S" (1 S) where for all E As, a-PMC (Definition 5) is satisfied.
Let be the constaint on 8-PMC. Let p- = Ep[y R = r, E S] and = Ep[R = r, x E S]. Consider the case p" and let a (r p²)/p" X-discretization shifts by most X. Let
8 5 (r + X pp'//p
Substituting r S ap" + p" yields
8 < A
p*
Plugging in p as the minimum of p", we complete the proof.
17

---

Under review as a conference paper at ICLR 2023
1.75
0.01
1.50
1.25
0.25
0,000
0.025
0.050
0.075
0.100
0.125
0.150
0.175
0.200
Figure 4: Relationship between (a, PMC and a-PMC given a geometric discretization. Illus- trated for (a. - PMC =0.1, for various values of and A.
The term A can be potentially large when < A. One way to avoid this issue to make the change in R(x) between bins scale with R(x) using Definition 11. What makes Definition 11 different from A-discretization that the intervals are multiplicative, rather than additive, distance apart. Hence, for given [0, 1]. model satisfying (a, AM- can have its prediction shift by at most factor of p This leads us to the following proposition.
Claim 2. Define p. x= > O and let C C 2 be collection of subsets of X. Let Eplul R I E S) 2 p for all S E and Ax- Let R(x) be a model satisfying (a, X) x)- multicalibration. Given (X, -)geometric discretization, R(x) is at most (ap + p- 1)-
proportionally multicalibrated.
Proof. By Definition 6, R(x) satisfies
EDypR E E ED [ E 1, E S1
<a
Ro
for categories (S E C x Ax satisfying Pp(R(x) E Ie E 5) 2 aX. Given 1/A bins, the subset where (D((()) E I| E 8) ad has size of at most alS) Therefore there is subset |S| 2 (1 - a)|S where for all r E Ax, a-PMC (Definition 5) is satisfied.
Let 8 be the constaint 8-PMC. Let p" Eoly r, S] and S). Consider the case > p' and let a = (r p')/p", i.. the tight bound. (A, p)-geometric discretization shifts by at most factor of p This implies
Ep R| r,
8 (rp pp')/p"
Substituting ap" + p" yields
8 op + pp-x -
We illustrate the relationship between (a, AM-PMC and a-PMC given geometric discretization in Fig. 4, which quantifies the relationship for different settings of and p.
A.2.2 ADDITIONAL DEFINITIONS
Definition 13 ((s, A)DDifferenttal Calibration). Let C. C 2 be collection of subsets of X. A model R(x) is difffferrnntillly calibrated with respect to C if, for all pairs (St, E C for which (D(S.), (D(S)) > 0, for any I E Ay
Eply I,
Se
(6)
Eply
R E 1, S,]
18

---

Under review as a conference paper at ICLR 2023
A.2.3 LOSS FUNCTIONS
The following loss functions are empirical analogs of the definitions of MC, PMC, and DC, and are used in the experiment section to measure performance.
Definition 14 (MC loss). Let D {ly, x) 1==0
D, and let A, 0. Define collection of subsets C 2 such that for all S C, [S] 2 qN. Let S₁ {x: R(x) 1, S) for (S, 1) E C x Ax. Define the collection S containing all S1 satisfying S1 2 oλN. The MC loss of model R(x) on D is
Σ 1/
Σn. R iES;
S,ES
Si
ieSı
Definition 15 (PMC loss). Ler {(y, 2) D, and a 7.P 0. Define collection
of subsets C E 2 such that for all S E C, 2 NN. Let S₁ fx R(x) E 11x S} for (8,1)
Ax. Define the collection S containing all S1 satisfying S 2 aXN. Let [8 Lies, 2 p.
The PMC loss of model R(x) on D is
Eses, Vi -Eies, R
max S
ΣiEs, Vi
Definition 16 (DC loss). Let D {(,, x)),
D, and let a, 7, Define collection of
subsets C 2* such for all S S C, 2 yN. Given a risk model R(x) (z) and prediction intervals
I, Let S₁ {æ R(z)= E I, S} for (S,1) E C x Ax. Define the collection S containing all S₁
satisfying S 2 aAN. The DC loss of model R(x) on D is
(9f S88
log
1S
Σ
Uk
S
Vj
EEST
jESt
A.2.4 THEOREM PROOFS
Theorem 3 Let R(x) be model satisfying a-PMC collection C. Then R(x) is (-aa)= multicalibrated on C.
Proof. To distinguish the parameters, let R(x) a model satisfying 8-PMC. Let ED|RR = r, E S] and pd Ep ly r E S1. Then r/(1+ + S p² < <r/(1-0 5). We solve for the upper bound on a-MC from Definition 2 for the case when p" > r. This yields
8
1-8" 8"
We can also solve for the lower bound on a-MC from Definition 2 for the case when p" < r. This yields
a S - p
Sr
1+ 8
r 1 1+ 5
8
1+8
For any 8 0, 5
8
Therefore the first case (p" r) limits the multicalibration of R(x).
1-8 1
19

---

Under review as a conference paper at ICLR 2023
Proposition 1 for all S E C, PD(S) > 7. Let R(x) be risk prediction model to be post-processed. For all (S EC x Ax let Ely|R I, E S] p. There exists algorithm that satisfies (a. A--PMC
Define A, 7 0. Let C 2 be collection of subsets of X such that,
with respect to C in O ((ax5-p>)) steps.
Proof. We show that Algorithm converges using potential function argument (Bansal Gupta, 2019), similar to the proof techniques for the MC boosting algorithms in Hebert-Johnson et al. (2018); Kim et al. (2019). Let p be the underlying risk, R, be our initial model, and R be our updated prediction model for individual i E S,, where S.- fz|æ E S, R(x) E I and (S, I) Ax. We use p", R, and R' without subscripts denote these values over S,. We cannot easily construct potential argument using progress towards (o,A)-PMC, since its derivative is undefined at E Epyy R I, x E S}=0. Instead, we analyze progress towards the difference in the lz norm at each step.
1|'" -R| -1lp" - RR'1l = Σ (pi - RR)²
Σ (p) squash(R, (R, + Ar))²
iES
iES,
2
Σ
((r) 11)² (pī (R: + Ar))²)
iE8.
(2p Ar 2R,Ar Ar?))
iES,
2Ar
Σ
(0: R2) A
iES.
From Algorithm we have
Ar
S,p Σ (pi R,)
EE
Substituting into Eq. (7) gives
lpp" R|| 11p*
R'll
|
We know that [S,] 2 N and that the smallest update Ar is ap. Thus,
I|p" - R - 1|p -
R' > a pPpPAyN
Since initial loss, for category S,,
I|p" R||, is at most N, Algorithm 1 converges in most O 13
updates
To understand the total number of steps, including those without updates, we consider the worst case, in which only single category S, is updated in cycle of the for loop (if no updates are made, the algorithm exits). Since each repeat consists of at most |C) / loop iterations, this results in O [C p
total steps.
L
Li
A.3 ADDITIONAL THEOREMS
A.3.1 DIFFERENTIALLY CALIBRATED MODELS WITH GLOBAL CALIBRATION ARE MULTICALIBRATED
Here show that, under the assumption that model is globally calibrated (satisfies 8-calibration), models satisfying E-DC are also multicalibrated.
Theorem 4. Let R(x) be a model satisfying (E. and 8-calibration. Then R(x) is (1 - - + N-mmllicaalirrted.
20

---

Under review as a conference paper at ICLR 2023
Proof. From Eq. (2) observe that E is bounded by the two groups with the largest and smallest group- and prediction- specific probabilities of the outcome. Let IM be the risk stratum maximizing (e, A)-DC, and let Pn. maxsec Pp (y) IMs. z E S) and Pa minsec PD(y R IM E S). These groups determine the upper and lower bounds of e as e S pa/Pa and Pn/ hn/pa ee.
We note that Pa S R IM) S Pa. since P(y|R E MM) N Esec P IMs S). and Pv, and Pa are the extreme values of P( E IMr E S) among S. So, a-MC is bound
by the group outcome that most deviates from the predicted value, which either Pn or Pa. Let
R IMS) There are then two scenarios to consider:
1. < Pn r| Pn when (pm + pa); and
2. a < Pa -rl = Pa when r 2 (pm pa).
We will look at the first case. Let p = Pb(y R E Im). Due to d-calibration, p - - St +6 Then
S Pn
$ Pn (pz -
Pn + 6
(1 -) +
- - + 6.
Above we have used the facts that 8 p scenario is complementary and produces the identical bound.
Pa. ps S e Pns and Pn S The second pns
Theorem 4 formally describes how 5-calibration controls the baseline calibration error contribution to a-MC, while eDDC limits the deviation around this value by constraining the (log) maximum and minimum risk within each category.
A.4 MULTICALIBRATED MODELS SATISFY INTERSECTIONAL GUARANTEES
In contrast DF, MC (Hehert-Johnson al., 2018) was not designed to explicitly incorporate the principles of intersectionality. However, show that it provides an identical efficiency property to DF in the theorem below. Given an individual's s attributes x = (I1,
Ia). will be useful to refer to subsets wish to protect, e.g. demographic identifiers. To do so, define A {A1, Ap}. I d, such that A₁ is the set of values taken by attribute I1.
Theorem 5. Let the collection of subsets C C 2 define groups of individuals according to the Cartesian product of attributes A C A. Let G E 2 be any collection of subsets that groups individuals by the Cartesian product of altributes in A', where A' C A and A' + 0. If R(x) satisfies a-MC on C, then R(x) is mumuliicalibrtedd on G.
In proving Theorem 5, we will make use of the following lemma.
Lemma 6. The a-MC criteria be rewritten as: for collection of subsets C X, E [0, and r E [0, 1).
max E ([)))((() = r, r, Sr+a
aEC D
and
min Elyl((() r, 1 E aEC
Proof. The lemma follows from Definition 2, and simply restates it as a constraint on the maximum and minimum expected risk among groups each prediction level.
Proof of Theorem 5. We use the same argument Foulds et al. (2019a) in proving this property for DE. Define Q as the Cartesian product of the protected attributes included in A, but not A'. Then for any (y, m) ~ D.
21

---

Under review as a conference paper at ICLR 2023
max gEg D
R(xx) T, E g]
max
(8)
9E9
BEQ
max
= EraeE
max D'EQ
(9)
959
PEQ
max max B [ 9E9 q'EQ
(10)
max aEC D =
(11)
Moving from (5) to (6) follows from substituting the maximum value of Eply R(x) r, r, for observations in the intersection of subsets in 9 and Q which is the upper limit of the expression in (5). Moving from (6) to (7) follows from recognizing that the sum P|z E qlæ E g] for all subsets in Q is I. Finally, moving from (7) to (8) follows from recognizing that the intersections of subsets in and Q that satisfy (7), must define a subset C. Applying the same argument, we can show that
min R(y) r, SEg D
2 min B(ylR(x) r, EC..
cEC D
Substituting into Lemma
max Elyl r E gl aa gEs D
and
min Eyyl R(x) = rr, = r-a
gEB
r, = g - S a
for all g G. Therefore R(x) is a-multicalibrated with respect G.
As a concrete example, imagine we have the protected attributes A {race (B,W), gender E {M, F}).
According to Theorem 5, C would contain four sets: M), (B, F",, M), (W, F)}. In contrast, there are eight possible sets in G: {(B, (B, F", (W, M), (), (),+), (W,+), (+, M), (*, F)), where the wildeard indicates match to either attribute. As noted in Appendix A.1.1, the efficiency property is useful because the number of possible sets in grows large combinatorial rate, rate as additional attributes are added; meanwhile C grows holds, consider that the maximum calibration error two subgroups at least as large as the maximum expected error of those groups combined; e.g., the maximum calibration error in higher order groups such as (B, +) will be covered by the maximum calibration error in either (B, M) or (B,F). F).
slower, yet exponential, rate. For an intuition for why this property
A.5 ADDITIONAL EXPERIMENT DETAILS
Models were trained on a heterogenous computing cluster. Each training instance was limited to single core and 4 GB of RAM. We conducted full parameter sweep of the parameters specified in Table A single trial consisted of a method, parameter setting from Table 2, and random seed. Over 100 random seeds, the data was shuffled and split 75%/25% into train/test sets. Results in the manuscript are summarized over these test sets.
Code Code for the experiments available by
https: githuuu.... com/ Code is licensed under GNU Public License v3.0.
here:
22

---

Under review as a conference paper at ICLR 2023
Table 5: Features used in the hospital admission task.
Description
Features
Vitals
temperature. heartrate, resprate, o2sat, systolic blood pressure, diastolic blood press, Emergency Severity Index (Tanabe et al,, 2004)
Triage Acuity
Check-in Data
chief complaint, self-reported pain score no. previous visits, no. previous admissions ethnoracial group, gender, age, marital status, insurance, primary language
Health Record Data
Demographic Data
Data We make use of data from the MIMIC-IV-ED repository, version 1.0, to train admission risk prediction models (Johnson et al., 2021). This resource contains more than 440,000 ED admissions from Beth Isreal Deaconness Medical Center between 2011 and 2019, We preprocessed these data to construct an admission prediction task in which our model delivers risk of admission estimate for each ED visitor after their first visit to triage, during which vitals are taken. Additional historical data for the patient was also included (c.g., number of previous visits and admissions). A list of features given in Table
A.6 ADDITIONAL RESULTS
Table 2 lists few parameters that may affect the performance post-processing for both MC and PMC. Of particular interest when comparing MC versus PMC post- post-processing is the parameter a, which controls how stringent the calibration error must be across categories terminate, and the group definition (A), which selects which features of the data will be used to asses and optimize fairness. We look at the performance of MC and PMC postprocessing over values of a and group definitions in Figs. 5 to 7. Finally, we empirically compare MC- and PMC-postprocessing by the number steps required for each to reach their best performance in Fig. and Table
From Fig. 5, it is clear that post-processing has minimal effect on AUROC in all cases; note the differences disappear we round to two decimal places. When post processing with RF, we do note relationship between lower values of and very slight decrease in performance, particularly for MC-postprocessingg
Figs. 6 and 7 show performance between methods MC loss and PMC loss, respectively. In terms of MC loss, PMC-postprocesssing tends to produce models with the lowest loss, at a values greater than 0.01. Lower values of do not help MC-postprocesssing in most cases, suggesting that these smaller updates may be overfitting to the post-processing data. In terms of PMC loss (Fig. 7), we observe that performance by MC-postprocessing is highly sensitive to the value of a. For smaller values of a, MC-postproceessing is able to achieve decent performance by these metrics, although in all cases, PMC potproostro generates a model with a better median loss value at some configuration of a.
We assess how many steps/updates MC and PMC take for different values of a in Fig. 8, and sum- marize empirical measures of running time in Table 4. On the figure, we annotate the point for which each post-processing algorithm achieves the lowest median value of PMC loss across trials. Fig. validates that PMC-postprocessing is more efficient than MC-oostprocessing at producing models with low PMC loss, on average requiring 4.0x fewer updates to achieve its lowest loss on test. From Table 4 observe that PMC typically requires larger number of updates achieve its best perfor- mance on MC loss (about 2x wall clock time and number of updates), whereas MC- postppoocceesn requires larger number of updates to achieves its best performance to its dependence on very small values of a. We accompany these results with the caveat that they based on performance on one real-world task, and wall clock time measurements are influenced by the heterogenous cluster environment; future work could focus on a larger empirical comparison.
PMC loss and DC loss, due
23

---

Under review as a conference paper at ICLR 2023
ML LR groups = aennchy.ggerder
ML = LR| groups ethnicty gendrrrunnneeeeee
0.15
0.1
0.01
0.001
postaracessing
groups ethnicttyggedde
groups eehnicry.gendde insurance
aphha
0.05
0.001
0.860
0.012
0.870
0.800
0.012
0.800
AUROC
AUROC
Figure 5: AUROC test performance versus experiment settings. Rows different ML base models, and columns are different attributes used to define C. The color denotes the post- processing method.
M:. - 18 groups - ahnicity.eender
ML - 18 groups - othnicity.gendeeee insuracee
0.05
0.001
0.32
0.16
0.175 0.200
8/255
ML RF groups ethnicty.gender
ML RF groups www.iittyyggdde..iei. insuraces
0.05
0.01
0.001
0.38
MC loss
MC loss
Figure 6: MC loss test performance versus across experiment settings. Rows different ML base models, and columns are different attributes used to define C. The color denotes the post- processing method.
24

---

Under review as a conference paper at ICLR 2023
ML LR] groups ethnicky.goom gender
ML LR groups ethnicity gender insurance
0.05
0.01
0.001
0.3
0.4
0.5
0.6
0.8
0.25
0.30
0.35
0.40
0.45
postaracessing
Hodal
ML RF groups othricc
ML RP groups athnicity gender insurance
alpha
0.05
0.01
0.001
0.5
1.0
1.5
2.0
3.0
3.5
0.5
PMC loss
PMC loss
Figure 7: PMC loss performance base models, and columns are different attributes used to define C. The color denotes the post- processing method.
experiment settings, Rows different ML
HL - Li groups athnicity.eendee
ML groups atiiiitty anniecaaneede..nuuaancce
0.15
PMC-0022
PMC--0.22
abi
0.05
PNC-0.31
1.00
200
300
400
500
600
700
800
200
400
600
820
postorocessing
ethnicity-gender
NL ethnictty gender,
PMC
0.35
P
alpha
PMCC-0.28
0.05
PMCC00.41
PMC=0.30
0.001
200
400
600
200
400
600
500
Updattss
Updates
Figure 8: Number post-pros post-processing updates by MC and PMC versus a across experiment settings. Rows are different ML base models, and columns are different attributes used to define C. The color denotes the post-processing method. Each result is annotated with the median PMC loss for that method and parameter combination.
25


Reviews:
[{"review_id": "imm-fx0315q", "reviewer": "Reviewer_RcCa", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The authors introduce the notion of **proportional multicalibration** (PMC): a criteria that measures whether a model is both multicalibrated (MC) and fair (in the sense of differentiable calibration (DC) which is an extension of sufficiency). As a reminder, multicalibration refers to a notion of calibration by group and by bin (of confidence scores). And differential calibration, which is a notion introduced by the authors, ensures that any two groups in a collection have the same expected score for each bin (of confidence scores).\n\nThey show theoretically (Th.2 and Th. 3) that PMC implies **both** multicalibration and differential calibration. A post-processing algorithm is then provided to proportionally multicalibrate confidence scores. Finally, experiments on an hospital admission task using MIMIC IV show that the proposed algorithm decreases the MC, DC as well as PMC losses without decreasing the AUROC. \n", "summary_of_the_review": "The paper is interesting but penalised by too many unclear points (notably related to the correctness of the theorems, choice of hyperparameters, usefulness of PMC vs MC), that prevent me from being convinced by the paper.", "strength_and_weaknesses": "**Strength**\n- The paper adresses a problem related to fairness with important practical implications.   \n- Completeness: the proposed measure is motivated theoretically, an algorithm is given to implement it practically, and experiments on simulated and real data are provided.\n\n**Weaknessess**\n-  I have a major concern regarding the theoretical results (Th. 1, 2, and 3). I am not sure that they are correct.   \n Let’s take theorem 1, which states that multicalibration implies differential calibration. In the proof, $r$ is the expected confidence score over a group S in a bin I, and $p^*$ is the expected outcome over a group S in a bin I. In the proof, it seems to me that $p^\\star$ and $r$ are considered equal for all bins and groups. However, we have dependencies $r = r(I, S)$ and $p^* = p^*(I, S)$. So plugging into equation (2) would involve the quantities $r(I, S_i)$, $r(I, S_j)$, $p^*(I, S_i)$ and $p^*(I, S_j)$, so that the conclusion does not follow.  \nExactly the same comments apply to Theorem 2, which states that proportional multicalibration implies differential calibration.  \nFor Theorem 3, which states that proportional multicalibration implies multicalibration, the proof is done for the case $p^\\star > r$. I think the case $p^* < r$ should also be considered, otherwise the proof is not complete and the Theorem could be incorrect.\n\n- It is unclear to me how the hyperparameters have been chosen for the experiments ($\\alpha$, $\\lambda$, $\\gamma$, $\\rho$). This is however important to be able to evaluate the experiments correctly.\n\n- The main paper lacks a more thorough discussion/comparison of MC vs PMC in the experimental part. As it seems that MC with small values of $\\alpha$ can do as well as PMC for the various performance losses, I understand that in practice, the main advantage of PMC vs MC would be in terms of computational time budget. Do I understand correctly? It would be useful to discuss this in more depth in the main paper.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "c8_7nOmwcmF", "reviewer": "Reviewer_oBPB", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper proposes an interesting new notion of \"Proportional multicalibration\" for algorithmic fairness. Proportional multicalibration (PMC) builds on the notion of multicalibration (MC). MC requires that the predictor is calibrated for each (identifiable) group within the data, as opposed to simply requiring calibration over the entire population. The notion of MC has been well-studied at this point, and has been found to be useful in practical contexts as well.\n\nThe authors point out a potential flaw in MC. MC is defined to allow an additive calibration error (following on the standard notion of calibration, which is also defined additively). The authors claim that this could be problematic since it allows for a larger relative calibration error for groups which have smaller probability estimates. PMC aims to correct for this, by requiring that the relative calibration error be bounded for all the groups. PMC also satisfies a notion of differential-calibration with better guarantees than MC.\n\nThe authors substantiate their theoretical results with an interesting experiment on real-world hospital data.", "summary_of_the_review": "In summary, I quite liked reading this paper and I think it has interesting ideas. Even at the current stage, it would be an interesting addition to the ICLR program. Perhaps the authors could comment a bit more on the weaknesses I wrote earlier?", "strength_and_weaknesses": "Strengths:\n\n1. The authors point out a potentially important issue with calibration scores (the fact that they could lead to larger errors for groups with lower probability estimates). PMC seems like a good way to correct for this,\n\n2. The authors establish an interesting connection between MC and notions of sufficiency and the new notion of differential calibration. The notion of sufficiency seems quite related to the concept of \"Outcome indistinguishability\" (https://arxiv.org/abs/2011.13426) and it would be nice if the authors elaborate a bit more on this connection.\n\n3. I quite liked the fact that the authors evaluated their results on real-world data which comes close to capturing the problem the paper aims to solve.\n\nWeaknesses:\n\n1. The paper has a number of strengths, but I'm not fully convinced of the papers claim about the superiority of PMC to MC. To me, the choice of relative error vs additive errors seems to be closer to a normative judgement, or at the very least the two need to be compared with respect to how they affect decision making in the use cases. Can I not make the claim that the proposed relative error notion is unfair, since it allows for a larger (additive) error for groups which do have higher probabilities? Could a decision maker not end up having less trust in the model's predictions if it has higher errors for those groups? Additive error almost seems like a simpler to understand notion (since the error remains the same for all predicted probabilities), and I could be convinced to believe that it could lead to better downstream decision making because of this property. Perhaps the authors can point out contexts where calibration itself is more meaningfully defined as a relative notion rather than an additive notion? If these settings exists, then PMC naturally comes out as a better solution than MC.\n\n2. I think it would be good if the authors commented a bit more on the fact that PMC seems to have lower accuracy than MC in the real-world experiment (I appreciate that the authors include this result). Is PMC trading-off increasing fairness guarantees for accuracy?", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "_D55Twpagze", "reviewer": "Reviewer_PsRB", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This work identifies a limitation of multicalibration, relates multicalibration and differential calibration, and proposes a new metric of calibration, dubbed proportional multicalibration (PMC). Then it proposes an algorithm to achieve PMC and illustrate the performance of the proposed algorithm in simulated and hospital admission settings.  ", "summary_of_the_review": "While this paper is interesting and the contribution is novel and important, the current work requires another major review to make sure the proofs and math work out and there are no mistakes. Therefore, I recommend rejection. \n\nIf I misunderstood the proofs, please correct me. I would be glad to change my score.  ", "strength_and_weaknesses": "Strengths:\n\nThis work is clearly written and addresses an important problem, calibration, that is relevant to model performance evaluation, trustworthiness, and algorithmic fairness. It is a blend of theoretical and computational contributions. The work is novel and provides new insight into the relationship between different definitions of calibration and addresses some of their limitations. The figures are very clear, and professional, and confidence intervals are computed. In general a strong contribution to the ICLR conference. \n\nWeaknesses:\nWhile due to limited time, I could not go through all the derivations and proofs in the appendices, and I found several mistakes in the proofs provided in the main text and computations. I might have made a mistake, but if I am not mistaken this work need to be reviewed comprehensively to make sure that there are no other mistakes in the proofs. \n\n\nMajor issues:: \n\n1. \"Proportionally multicalibrated models thereby obtain robust fairness guarantees that are independent of population risk categories.\" This is actually not strictly correct. In the proof of Theorem 3, we have $=\\frac{r\\delta}{1-\\delta} \\le \\frac{\\delta}{1-\\delta}$. While this is not incorrect, this is not the strongest upper bound, one can alternatively write $=\\frac{r\\delta}{1-\\delta} \\le \\frac{r_{max} \\delta}{1-\\delta}$ that is both stronger than the claim made in Theorem 3 and makes this bound dependent on population risk categories. Hence, under $\\alpha$-PMC, error in MC can depend on the population risk category. \n\n2. Theorem 1. \nAfter  plugging the lower and upper bounds into Eq. (2), we get $e^{\\epsilon} \\le \\frac{r_i + \\alpha}{r_j - \\alpha} $ not $e^{\\epsilon} \\ge \\frac{r + \\alpha}{r - \\alpha}$. Therefore, the maximum of this ratio will be  ${\\epsilon}  \\le \\ln(\\frac{r_{\\rm max} + \\alpha}{r_{\\rm min} - \\alpha})$  not ${\\epsilon} \\le \\log(\\frac{r_{\\rm min} + \\alpha}{r_{\\rm min} - \\alpha})$. Please also use $\\ln$ as $\\log$ is often reserved for log-10 based. Also, the inequalities should be $ \\le$, not $\\ge$. If it is $\\ge$ then $\\epsilon$ can be any large number and does not provide a meaningful bound on differential calibration. \n\n3. In a similar vein, in Theorem 2, the r in the denominator and nominator can be different and do not need to be the same, hence they do not cancel and the bound is still a function of $r$.\n\n4. I cannot replicate the numbers $6$% and $20$%, neither with the new formula nor with the old formula. Can you walk me through how these numbers are computed? \n \n\nMinor Issues: \n\n4. In Figure 3 (and results), the results are sensitive to the choice of $\\alpha$, and as the authors mentioned the optimal choice of $\\alpha$ might differ. I could not understand how the $\\alpha$ is chosen for each model (MC and MPC).\n\n\n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "_3TJRVzdC4", "reviewer": "Reviewer_Csmw", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "Multicalibration requires calibration of a predictor even conditioned on membership in S where S comes form a family of protected subsets. Formally, we require $E[y] = E[f(x)]$ conditioned on x belonging to S and $f(x) \\approx p$. If we relax this to allow $E[(y - f(x))] = \\alpha$, conditioned on $f(x) \\approx p$, we have $E[y] = p(1 \\pm \\alpha/p\\mu(p))$ where $\\mu(p)$ is the probability that $f(x) \\approx p|x \\in S$. \n\nThis paper posits that to achieve the fairness notion of sufficiency, one cares that we have a good multiplicative guarantee across all protected sets. Doing so requires\n1. The set $S$ to be reasonably large, so that $\\Pr(x \\in S) \\geq \\gamma$.\n2. The event $f(x) \\approx p|x \\in S$ to be reasonably large, in other words that $\\mu(p) \\geq \\delta$.\n3. The value $p$ itself is not too small.\n\nUnder these  conditions, multicalibration guarantees that condition of $S$, for most values $p$ (all but $\\alpha$ fraction), the expected value of $y$ is multiplicatively close to our predictions.\n\nThis paper proposes a new notion called proportional MC that achieves this guarantee. But it seems that it can  be achieved by setting parameters differently in the original definition, so I am not convinced that this is really a new notion (or that one is needed).  Indeed, if you look at Definition 1 and 3, you can infer 3 from 1 by taking $\\alpha = f(\\alpha, \\lambda,\\rho)$. In other words, sufficiently small error in the standard notion ensures the multiplicative notion that is desired here.\n\nAt a higher level, having only sample access means that the best guarantees one could hope for are of the form $$ E[I(x, f(x))(y -f(x))] \\leq \\alpha$$\nwhere $I$ is some event that depends on $x, f(x)$. Here it is $x \\in S, f(x) \\approx p$. If we wish to get multiplicative closeness under conditioning, we need:\n1. A lower bound $\\mu $ on how large E[I(x, f(x))] can be.\n2. Take the multi calibration error $\\alpha$ small enough so that $\\alpha/\\mu$ is meaningful. \n\nThis suffices the ensure that $E[y|I]$ and $[f(x)|I]$ are close. Here, since they want to error to be comparable to $p$, we also lower bound $p$. \n\n\n\n\n\n\n\n", "summary_of_the_review": "The message that this paper makes that if one wants strong sufficiency guarantees, one has to be mindful of possibly low probability events is a good one. But I don't think this calls for a new definition, rather just a judicious setting of parameters in the existing one. \n", "strength_and_weaknesses": "See above.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject