On the Epistemic Limits of Personalized Prediction
Anonymous Author(s)
AffiliationAddress email
Abstract
Predictive models often include group attributes that encode personal characteristics like sex, blood type, or HIV status.
Personalized models must ensure fairuse – i.e., groups who provide personal data should expect to receive a tailoredimprovement in accuracy compared to a non-personalized model.
In this paper,we derive conditions under which one can detect fair use violations in predictivemodels and characterize when estimating fair use is impossible.
We propose ametric to evaluate the worst-case accuracy gain across groups called the benefitof personalization (BoP).
We obtain bounds on the error probability for testing ifthe BoP is above a target threshold given finite samples.
Remarkably, our boundsprovide an information-theoretic limit on the number of group attributes that amodel can use to allow for verification – beyond this limit, it is impossible toreliably detect if personalization harms or benefits all groups.
We also derivestatistical limits for the minimax mean-square error of estimating the BoP. Ourresults show that there is no way to reliably determine if a personalized modelwith k ≥ 19 attributes benefits every group that provides personal data, even whenwe are given a dataset with N = 8 × 10 9 samples (i.e., one observation for eachperson in the world).
Introduction
Machine learning models are increasingly personalized 1 .
In applications such as healthcare, modelsoften use group attributes to encode personal characteristics (e.g., sex, age group), sensitive information (HIV status) or information that must be acquired at prediction time (blood type, malignantbiopsy, see e.g., Taylor et al., 2017; Bertsimas et al., 2019).
The collection and use of group-denotingfeatures in prediction models is adopted to improve performance (Fan and Poole, 2006; McAuley,2022).
Categorical group features divide the population into subgroups distinguished by the combination of feature values (e.g., women with ages above 70 years).
At first, one might assume that apersonalized model that uses group data produces predictions that are at least as accurate comparedto a non-personalized model for all subgroups.
Otherwise, the personalization is unfair: a subgroup isharmed when group data is included in the prediction model.
Thus, an essential goal for fairness inpersonalized models is to ensure fair use — i.e., to ensure that models using group attributes producea tailored improvement in accuracy for all
Standard practices in training machine learning models do not ensure fair use in personalization.
Personalized models are trained to minimize empirical risk on testing data, which does not necessarily lead to better prediction for all subgroups.
In fact, health technology companies have faced backlashover mounting concerns about the fairness of their algorithms on different racial groups (Vartan, 2019).
However, testing personalized models for fair use on all (potentially exponentially) many subgroupsinduced by different values of group attributes is challenging (Kearns et al., 2018; Hebert-Johnsonet al., 2018).
It may be the case that, for specific subgroups, a group-blind model that does not usegroup attributes may perform better than a personalized one (Suriyakumar et al., 2022).
We propose a metric called the benefit of personalization (BoP) that measures the minimum gain inaccuracy across all subgroups by using a personalized model instead of a group-blind model.
BoPmust be positive whenever a personalized model is used—otherwise, at least one group is harmed bychoosing a personalized model.
Estimating the BoP empowers individual stakeholders to ensure theirpersonal group information is used fairly and that they receive a more accurate prediction in returnfor their data.
Moreover, reporting the BoP enables practitioners to justify the collection of groupinformation for personalized models.
The goal of this paper is to understand the conditions under which one can check for fair use ofgroup data in prediction models and, more interestingly, the conditions where estimating fair useis impossible.
We consider a setting where only a finite number of samples of the underlyingpopulation is available to audit a set of personalized models for a BoP.
The number of subgroupsgrows exponentially with the number of group attributes used for personalization.
As we will show,when there are many attributes and not enough samples, reliable estimation and testing for BoPbecome impossible.
This is, of course, expected: an exponential increase in group-denoting attributesinevitably requires more samples per subgroup—in fact, this is the motivation behind auditing forcomputationally identifiable groups (e.g., Hebert-Johnson et al., 2018).
What is surprising in ourresults is that, given a finite number of samples (at most one sample for each person in the world≈ 8 × 10 9 ), the information-theoretic maximum of group attributes that can be collected and stillused to verify for a BoP is, in fact, small: with more than 19 binary attributes, it is (statistically)impossible to test a personalized model for a BoP without further assumptions on the underlying datadistribution.
The main contributions of this work include:
We propose a metric, the benefit of personalization , which quantifies the minimum gain in accuracyby using group attributes across all subgroups for a prediction model.2.
We design a hypothesis test framework to audit if a personalized model violates fair use of groupattributes.
We pair our test with an information-theoretic limit on the error probability that holdsfor any hypothesis test that audits a model for fair use.
A rule of thumb is that the number ofsamples per subgroup should be greater than the number of subgroups for testing error to be small.
We characterize statistical limits on the Mean Square Error (MSE) of any estimate of BoP. Ourresults show that we cannot reliably estimate the benefit of personalization across subgroups formodels that use more than 25 personal attributes.
We conduct experiments to demonstrate the behavior of the probability of error in testing for fairuse.
Our experiments show that the MSE of a BoP estimation increases exponentially fast whenthe number of samples per group drops.
Finally, we evaluate our theoretical minimax bounds onthe MSE of the BoP estimation on real-world datasets.
1.1 Related Work
Algorithmic Fairness.
Benefit of personalization , our proposed metric, embodies a fairness principle that is distinct from fairness metrics that aim to quantify disparate impact.
Fairness metricscan be broadly characterized as individual fairness and group fairness.
The motivating philosophybehind individual fairness is to treat similar individuals similarly (Dwork et al., 2012), while groupfairness aims to ensure similar performance across different groups (Ustun et al., 2019).
An influentialprinciple that captures popular metrics like Equalized Odds and Equal Opportunity (Hardt et al.
,2016) is parity, which seeks to achieve proportional allocation of resources conditioned on the group(Dwork and Ilvento, 2018).
Although parity is an appropriate notion of fairness for applications in criminal justice, hiring, and college admissions since it signals disparate treatment among groups(Zafar et al., 2017), it is less suitable for domains with different ethical principles.
For example inmedical applications, the relevant principles are non-maleficence (do no harm) and beneficence (dogood) for all groups rather than achieving parity on a statistical metric (Beauchamp et al., 2001).
Insuch settings, accuracy is paramount.
Our proposed metric ( benefit of personalization ) fills the gap.
Ensuring a positive BoP is essential for non-maleficence in model personalization.
Moreover, BoP is complementary to existing group-fairness metrics.
BoP may mask imbalancedperformance across groups, while group-fairness metrics do not capture the gain (or loss) in usinggroup attributes.
Hence, there are situations where both metrics are called for.
Auditing for Fairness.
Auditing whether an ML model is fair is a challenging problem withimportant consequences.
In criminal justice, for example, Neil and Winship (2019) argue that testingfor racial discrimination in policing can yield diverging results using different testing methods.
Berket al.
(2021) show that it may be impossible to simultaneously satisfy different fairness constraints.
To regulate algorithmic filtering on social media, Cen and Shah (2021) develop a statistical test thatenforces a regulation.
In greater generality, many recent works explore causal approaches that testfor unfairness in models (Kilbertus et al., 2017; Nabi and Shpitser, 2018).
Si et al.
(2021) propose atesting algorithm via Optimal Transport Projections, only considering one protected attribute at atime without intersectional subgroups.
In this work, we prove an information-theoretic impossibilityresult on testing that works for all tests.
Multicalibration & Fairness Gerrymandering.
Testing for fairness is complicated in settingswith a richer class of subgroups.
Specifically, the framework of our work is similar to the Multi-Xapproach considered by Hebert-Johnson et al.
(2018) and Kearns et al.
Fairness constraints,such as equal opportunity and statistical parity, are imposed on all computationally identifiable –potentially exponentially-many – subgroups within a population.
Under this framework, Kearns et al.(2018) show that auditing for fairness is computationally-intractable for worst-case distributions.
Ourimpossibility result in Corollary 1 answers a different question.
In particular, we characterize whenthe probability of error of an arbitrary auditing algorithm is too large to flag a concrete outcome.
Wederive a fundamental limit on the number of attributes a model can have to reliably test for fairnesswith any auditing algorithm.
Our findings differ from those of Kearns et al.
(2018) resulting from a key difference in the assumptionof sample size.
Specifically, in (Kearns et al., 2018), the authors assume that it is possible to havemore data points than the VC dimension of the model hypothesis class.
Note that for models likedeep neural networks, the VC dimension approaches infinity (Sontag, 1998).
Our work builds on thekey observation that with a finite population on earth, the number of available independent samples isfinite.
Thus, the available data points for each subgroup are limited.
Personalization, Privacy & Data Minimization.
Data is collected and used for building personalized algorithms.
When training personalized algorithms using personal data, prior work in both
Computer Science and Law advocate for including individual rights in data privacy laws (Kaminski,2022) (Isaak and Hanna, 2018).
Ustun et al.
(2019) argue that given the rights, a rational individualwill only provide information on personal data if the decoupled classifiers lead to higher performance.
This concept is called beneficence and is formalized in fair use.
Framework
In this section, we introduce notations and definitions used in this paper.
In particular, we define anew metric to evaluate fair use called the benefit of personalization (BOP).
2.1 Preliminaries
Consider a binary classification task where categorical attributes represent group membership.
Weaim to predict the label Y by training a classifier on the dataset D ≜ { ( X i , S i , Y i ) } Ni =1 , where
X i ∈
X is a vector of non-group features, and S i ∈ S ≜ { 0 , 1 } k is a vector of k group attributes(e.g.
sex, blood_type, HIV_status), and Y i ∈ { 0 , 1 } is a binary label.
The elements of S are binaryk -dimensional vectors, which means that there are d ≜ 2 k groups that provide personal data.
Hence,we denote S as a one-hot encoding of the binary group attributes, with d = | S | = 2 k .
To simplify theanalysis, we consider a scenario where each subgroup has the same number of data points.
Let N bethe total number of samples, then the number of samples for each group is m ≜
⌊ N/d ⌋ data points.
We argue that this is the best-case scenario when prior knowledge of the dataset distribution is notknown: intuitively, the group with fewer samples will dominate testing or estimation performance(see supplementary material (SM) for a more detailed discussion).
We frame our impossibility resultsfor this best-case setting; an imbalance in samples will only render more difficult auditing for thebenefit of personalization.
Let the distribution of the dataset be P D .
At the group level, suppose that the distributions for eachgroup are independent, and we draw m i.i.d.
data points for each subgroup from the correspondingdistribution P j :
[Equation]
In this paper, we compare two classifiers, a group-blind classifier and a personalized classifier .
Specifically, the group-blind classifier h 0 only takes non-group attributes as input, while the personalized classifier h g takes both non-group and group attributes, i.e., h 0 :
X → { 0 , 1 } andh g : X × S → { 0 , 1 } .
h 0 is taken to be a baseline model trained in good faith, e.g. by empirical riskminimization on a dataset with group attributes removed.
A core principle that motivates our work is the Fair Use of group attributes (Ustun et al., 2019), wemake this definition mathematically precise next.
Definition 1 (Fair Use) .
Let h 0 be a group-blind classifier, h g a personalized classifier, and P j as in(1).
We say that h g follows the fair use principle if
[Equation]
2.2 The Benefit of Personalization
A personalized classifier outperforms a group-blind classifier when the metric is the overall accuracyof a model, i.e., average accuracy across all data points.
In contrast, a group-blind classifier can havehigher accuracy for specific subgroups in comparison with a personalized classifier (Suriyakumaret al., 2022).
We next introduce a metric called the benefit of personalization .
This metric captures the minimumgain in accuracy across all intersectional groups by using a personalized classifier and can be used toidentify fair use violations.
Definition 2.
(Benefit of Personalization).
Given a fixed data distribution P S,X,Y .
The benefit ofpersonalization is given by:
[Equation]
In addition, we call P j ( Y = h g
( X, S ))
− P j ( Y = h 0 ( X ))
the per-group benefit.
The benefit ofpersonalization is a function of both classifiers and the data distribution.
When the parameters areclear from the context, we denote γ ( h 0 , h g , P D ) = γ .
In this paper we choose accuracy as a benchmark metric for the BoP.
Our results still hold if accuracy By the definition in (2), the BoP is the gain in accuracy by using a personalized classifier.
Forexample, γ = 0 .
4 means that, for every subgroup, a personalized classifier is at least 40% moreaccurate than the group-blind classifier.
Testing for Fair Use
In this section, we formulate fair use as a hypothesis test on the benefit of personalization.
We providean information-theoretic bound on the reliability of testing in terms of a trade-off between the numberof group attributes and the probability of error of the test for a fixed sample size.
Our bound leads tothe maximum number of group features that ensures reliable testing for fair use of group attributesand an interesting phase transition phenomenon on the probability of error in the hypothesis test.
Allproofs are provided in the supplementary material.
3.1 Hypothesis Test Famework
The disclosure of group information to a model should incur a gain in accuracy.
It is critical toreliably test if, on average, a personalized model yields a gain in accuracy across all subgroups.
Thefollowing hypothesis test framework verifies if the benefit of personalization given in Def. 2 is abovea certain threshold ϵ .
• H 0 :
γ < 0 ⇔ min j ∈ [ d ] P j ( Y = h g ( X, S ))
− P j ( Y = h 0 ( X ))
< 0• H 1 : γ ≥ ϵ ⇔ min j ∈ [ d ] P j ( Y = h
g
( X, S ))
− P j ( Y = h
( X )) ≥ ϵ
In practice, a personalized model should be deployed if there is evidence from data to reject the nullhypothesis ( H 0 ).
Otherwise, a personalized model should not be deployed, since there is not enoughevidence to conclude that all groups receive an average gain in exchange for revealing their personalgroup attributes.
Note that, when ϵ = 0 , the above hypothesis test is equivalent to verifying fair useof group attributes according to Defn.
3.2 Limits of Testing
In real-world applications, the reliability of testing for fair use is governed by sample size: reliabletesting requires a minimal number of samples per group.
In the extreme case where each group hasonly one data point, for example, it is impossible to reliably test for fair use of group attributes.
Suppose that a practitioner has access to a dataset D with only N ∈ N samples, and uses it to traina classifier with k group attributes.
In this case, the number of data points per group is given bym = N/ 2 k .
In Theorem 1, we characterize how the exponential reduction of data at the group levelaffects the precision of the hypothesis test in the minimax sense.
To characterize the trade-off between error probability, define V h as the indicator function that h doesnot satisfy the null hypothesis, i.e., for each classifier h , V h
=
if and only if h ∈ H 0 .
Theorem 1 (Error in Hypothesis Test) .
Let ϵ ∈
[ − 1 / 2 , 1 / 2] , h 0 be a group-blind classifier, and P Dbe the distribution of the dataset D .
Also, suppose that the dataset is described in Sec. 2.1.
Considerthe hypothesis regions H
= { h | E D [ γ ( h, h 0 )] > ϵ } , and H
= { h | E D
[ γ ( h, h 0 )] ≤ 0 } .
LetΨ :
H → { 0 , 1 } be a decision function, and define the error probability in the hypothesis test to be
P e given by:
[Equation]
Then,
[Equation]
Given that practitioners do not have access to the dataset distribution, Theorem 1 presents a minimaxlower bound on the probability of error in a hypothesis test.
The minimax lower bound implies thatfor every decision function Ψ , there exists a dataset distribution such that the probability of error forthe hypothesis test is greater than the lower bound in (5).
To prove this Theorem, we lower bound the probability of error of any decision function by thetotal variation distance of two distributions (belonging to different hypotheses) that are “difficult” todistinguish.
We use a generalized version of Le Cam’s
Lemma stated in Lemma 1 that might be ofindependent interest.
The original Le Cam Lemma (Le Cam, 1973) states that (6) holds when θ 1 , θ 2are elements of Θ .
We generalize this lemma for the case where θ 1 , θ 2 are elements of Θ chosen atrandom.
Lemma 1 (Generalized Le Cam) .
Let { Λ , d } be a metric space, and D be a dataset.
Let P be afamily of probability distributions with parameter θ ∈ Θ .
Suppose that f is a statistics with image inΛ ( f ( θ ) for θ ∈ Θ means the statistics calculated for P θ ), and that there exists Λ 1 , Λ 2 subsets of Λthat are 2 δ -separated.
Also, suppose that Θ 1 and Θ 2 are subsets of Θ
such that f ( θ ) ∈ Λ 1 for allθ ∈ Θ 1 and f ( θ ) ∈ Λ 2 for all θ ∈ Λ 2 .
Then we have that:
[Equation]
(6) where θ i ∈ σ (Λ) with image in Θ i , and P θ
i indicates a probability distribution in P with parameterθ i for i ∈ [2] .
Threshold Test.
Given sufficient samples per group as outlined in Theorem 1, how should testingbe done in practice?
The most natural choice is a threshold test:
Given h g and h 0 , we simplyevaluate the performance of each classifier using an auditing dataset and check if there is an ϵ gain inaccuracy across all the groups. Intuitively, the threshold test consists in estimating the BoP (e.g., usingempirical mean in the group or bootstrapping) and then checking if it is greater than the threshold ϵ . Note that the theorem holds for all possible tests (not only the threshold test). 3.3 Implications of the Limits of Testing
Given that the size of the dataset is smaller or equal to 8 × 10 9 (number of people on Earth), theorem1 yields a fundamental limit on the number of group attributes in order for the probability of error inthe hypothesis test to be smaller than 1 / 2 (a random choice).
Corollary 1 formalizes this limit.
Corollary 1 (Limit for Testing) .
With the same setup as in Theorem 1.
Take N = 8 × 10 9 , and letϵ = 0 .
If min max P e ≤ 1 / 2 then k ≤ 19 .
[Figure]
Phase Transition.
The lower bound in Theorem 1 gives a characterization for a phase transition in the hypothesis test error.
Specifically,consider that N = 8 × 10 9 , ϵ = 0 .
01 , d = 2 k ,and m = N/d .
For k = 18 attributes, the minimax lower bound for the probability of error is0 .
13 , in contrast for k = 19 attributes the minimax lower bound is 0 . 97 .
Hence, there existsa notable phase transition in the probability oferror in hypothesis test between k = 18 andk = 19 .
Figure 1 shows that, for a fixed numberof groups d = 141 and varying the number ofsamples per group, there is also a phase transition for the minimax error probability in a hypothesis test.
This phase transition can also beverified in the semi-synthetic data case in Figure2 Left .
[Figure]
a t a p o i n t s p
e r g r o u p
[Figure]
Error Probability in Semi-Synthetic Data.
Figure 2 ( Left ) displays the error probability in ahypothesis test using the threshold test.
The error probability decreases when the number of datapoints per group increases.
Moreover, the figure shows that when the number of samples per groupis greater than the number of groups, the error probability is smaller or equal to 50% for the firsttime.
This is, indeed, a simple rule of thumb as a consequence of our results: the number of groupsshould never be greater than the number of samples per group; otherwise, testing for fair use isinformation-theoretically infeasible without prior assumptions on the underlying distributions.
In
Figure 2 ( Right ), we plot a heat map for the error probability in a hypothesis test using as thresholdthe values on the x-axis and as the number of data points per group the values of in the y-axis.
Forhigher values of ϵ , there is more "tolerance" for mistakes in the test, and the phase transition happensfor a smaller number of samples per group.
Estimating the Benefit of Personalization
When the benefit of personalization is positive but small, a user may opt-out of providing groupfeatures to a personalized model.
If such features are sensitive or costly to produce, their use mayonly be justified when a personalized model provides a significant accuracy gain.
Thus, testing for thebenefit of personalization (e.g., if γ > ϵ ) is, on its own, insufficient: stakeholders must be made awareof the estimated increase in accuracy from sharing personal data and only provide their personalinformation when there is a sufficient gain in accuracy (Ustun et al., 2019).
In this section, we consider the problem of estimating the benefit of personalization.
We deriveinformation-theoretic limits for estimation and show that, for a finite dataset, there is a threshold forthe number of group attributes required to reliably estimate the BoP with a small error.
4.1 Estimation Framework
We consider the problem of designing an estimator γ ∗ for the BoP given a dataset D with n samples.
Our metric of choice is the mean-square error (MSE) E (cid:2) ( γ − γ ∗ ) 2 (cid:3) .
Naturally, the value of theminimum MSE (MMSE) depends on the dataset distribution and the estimator of choice.
Insteadof restricting our analysis to specific estimators and datasets, we aim to understand the minimaxestimation error, i.e., the best MMSE achievable across all dataset distribution, defined as
[Equation]
To motivate the definition of R ∗ , suppose that there exist functions g 1 ( d, m ) , g 2 ( d, m ) > 0 such that
R ∗ ≤ g 1 ( d, m ) , and R ∗ ≥ g 2 ( d, m ) .
Recall that d is the number of subgroups and m is the number ofsamples per group.
This implies that for every estimator γ ∗ , there exists a dataset distribution P X,S,Ysuch that g 2 ( d, m ) ≤ E (cid:2) ( γ − γ ∗ ) 2 (cid:3) ≤ g 1 ( d, m ) .
In practice, this means that for any estimator thereis a distribution such that the MSE of the resulting BoP estimate will be bounded away from zero byg 2 ( d, m ) .
The main result in this paper is Theorem 2, which gives explicit expressions for g 1 , g 2 .
Theorem 2 (Minimax bounds for the Estimation MSE) .
Given a dataset D , constants d, m ∈ N suchthat
d ≥ 4 , d ≤ 2 m +2 , a group-blind and a personalized classifier, the minimax estimation error forthe benefit of personalization in (7) is such that:
[Equation]
Detailed proof for this Theorem is given in the supplementary material.
The proof relies on two-pointmethods for minimax lower bounds; specifically, we use the Generalized Le Cam (Lemma 1) togetherwith standard concentration inequalities.
We discuss the implications of this Theorem next.
4.2 Implications of The Limits of Estimating the BoP
For a dataset with a fixed amount of data points ( N ), given two personalized models with k 1 andk 2 group attributes ( k 1 > k 2 ), then in the best case scenario, the number of data points per groupwill be, respectively, N/ 2 k 1 < N/ 2 k 2 , hence the MSE for estimation will be higher on the "more"personalized classifier.
Theorem 2 gives a characterization for the order of the estimation error for the
BoP. The bounds in Theorem 2 are illustrated in Figure 3 ( Right ) in log scale.
This figure suggeststhat even knowing that the bounds in Theorem 2 are in the minimax sense, they are close to theestimated MSE, especially the lower bound.
We apply Theorem 2 to derive a fundamental limit on the number of binary group attributes toreliably estimate the BoP γ .
This lower (converse) bound in (8) captures the trade-off between theminimax MSE of an estimator, the number of sensitive attributes, and the dataset size.
This trade-offis instantiated in Corollary 2, resulting in an upper bound in the number of group attributes requiredto achieve a small minimax MSE in estimating the BoP given finite samples.
Corollary 2 (Limits of Estimation) .
Given a dataset D with N data points, d ≥ 4 groups, m = N/ddata points per group, an 0 < δ < 1 / 16 , a group-blind and a personalized classifier.
If d ≤ 2 m +2 ,and
[Equation]
then, the number of binary group attributes ( k ) satisfies
[Equation]
Specifically, for N = 8 × 10 9 (approximately the number of humans on the Earth) and δ = 1 / 400 =(5%) 2 , then the maximum number of binary group attributes is 25 , i.e, k ≤ 25 in order to ensurereliable estimation.
The proof of Corollary 2 is a direct application of the Theorem 2.
A more detailed proof is given inthe SM.
By choosing δ = 1 / 400 the square root of the MSE is at most 1 / 20 = 5% .
Hence, when theestimator γ ∗ is unbiased, Corollary 2 gives an upper bound of 5% in the estimation variance.
Corollary 2 gives a rule for practitioners to design algorithms that reliably determine the minimumgain from personalization.
During a decision-making process, such as deciding to take an examfor a medical diagnosis, the user should take advantage of an accurate estimate of the benefit ofpersonalization and make the best-informed decision.
In general, one would expect that the maximum number of group attributes for a reliable test shouldexceed the maximum number of group attributes for reliable estimation.
Our bounds (5), (8) showotherwise. This is an artifact of the fact that | γ |∈ [0 , 1] .
Easier estimation for statistics with normsmaller than 1 is simply an artifact of scale and follows from the Le Cam Lemma.
Final Remarks and Limitations
Personalized ML models should provide performance benefits to all individuals who provide personaldata.
We call this gain the benefit of personalization (BoP).
In this work, we have proposed tools toestimate this quantity and determine if it is large enough to warrant the collection of personal data.
Our results suggest that we are limited in our ability to determine if personalization leads to equitablegains in finite-sample settings.
In particular, we show that there is no way to reliably determine if apersonalized model with k ≥ 19 attributes benefits all groups that provide personal data, even whenwe are given a dataset composed of N = 8 × 10 9 samples (i.e., one observation per person in theworld).
Limitations.
The minimax bounds hold for all decision functions and estimators and inherentlyconsider the worst-case distribution for each case.
However, its generality implies that some knowledge of the underlying data distribution may result in a more favorable trade-off between samplesize, estimation (or testing) error, and the number of group attributes.
In particular, in Figure 3, the MSE in estimating BoP falls below the minimax lower bound, suggesting that the model’s underlyingdistributions for each subgroup is not quite the worst-case scenario.
Regardless, the upper boundof the MSE estimation and the impossibility result always hold in the general case when no priordistribution knowledge is known, as usual for minimax results.
A seeming constraint in our workis that we consider every group to have the same number of data points, but in Sec.
2.1 we arguethat this is the best-case scenario for testing and estimation.
Finally, fair use is a complement togroup fairness metrics, not a substitute.
If the base and personalized model are trained in bad-faithto underperform on a target group, then a positive BoP may mask a loss of accuracy on this group.
Conversely, group fairness does not capture the gain (or loss) in using group attributes.
Recommendations.
At the very least, ML practitioners should use the rules of thumb to controlthe number of group features to ensure reliable estimation and testing.
In addition to not exceeding19 attributes to ensure reliable testing, there should be more samples per group than the number ofsubgroups.
Failing to do so will require additional assumptions on the data to reliably estimate the BoP or ensure fair use.
When one group has very few data points and estimating the BoP is unreliable,practitioners should reduce the number of groups or use the baseline model only for some groups. Our bounds can guide on how the number of groups can be reduced to enable reliable estimationof the BoP. We encourage the report of estimated per-group gains in accuracy for all groups.
Thisenables stakeholders to make an informed decision on whether to share their personal information,particularly if this may incur a data collection or privacy cost.
Finally, if the BoP is accuratelyestimated and is negative, groups without a gain in personalization should revert to group-blindmodels. A per-group estimate of the gain in accuracy can also be used to guide feature selection(Miao and Niu, 2016).
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main ex-
perimental results (either in the supplemental material or as a URL)?
[Yes] Check
supplementary material.
All codes are provided.
(b) Did you specify all the training details (e
g., data splits, hyperparameters,
how they
were chosen)?
[Yes] Check supplementary material.
All training details are specified
in the code.
(c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
[Yes] Check supplementary material.
All plots with randomness
include confidence interval bars.
(d) Did you include the total amount of compute and the type of resources used (e.g., type
of GPUs, internal cluster, or cloud provider)?
[Yes] Check supplementary material.
All
experiments run on Google Collab
If you are using existing assets (e
g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators?
[Yes]
(b) Did you mention the license of the assets?
[Yes]
(c) Did you include any new assets either in the supplemental material or as a URL?
[Yes]
(d) Did you discuss whether and how consent was obtained from people whose data you’re
[Table]
using/curating?
[No] We are using standard and public
[Table]
(e) Did you discuss whether the data you are using/curating contains personally identifiable
[Table]
information or offensive content?
[No] We are using standard and public dataset
[Table]
If you used crowdsourcing or conducted research with human subjects...
[Table]
(a) Did you include the full text of instructions given to participants and screenshots, if
[Table]
applicable?
N/A]
[Table]
) Did you describe any potential participant risks, with links to Institutional Review
[Table]
Board
IRB) approvals, if applicable?
[N/
[Table]
c) Did you include the estimated hourly wage paid to participants and the total amount
[Table]
spent on participant compensation?
[N/A] [Table]