On the Epistemic Limits of Personalized Prediction
Anonymous Author(s)
AffiliationAddress email
Abstract
Predictive models often include group attributes that encode personal characteristicslike sex, blood type, and HIV status.
Personalized models must ensure fair use– i.e., each group who provides personal data should expect to receive a tailoredimprovement in accuracy compared to a non-personalized model.
In this paper, wederive conditions under which one can detect fair use violations, and characterizewhen it is impossible to detect fair use reliably.
We introduce a metric calledthe benefit of personalization (BoP) to measure the worst-case accuracy gainacross groups.
We bound the error probability for testing if the BoP exceeds atarget threshold in a finite sample regime.
These bounds delineate an informationtheoretic limit on the number of group attributes that a model can use to allow forreliable verification – i.e., beyond this limit, it is impossible to tell if personalizationharms or benefits all groups.
We also derive statistical limits for the minimax meansquare error of estimating the BoP. Our results show that there is no way to reliablydetermine if a personalized model with k ≥ 19 attributes benefits every group whoprovides personal data using a dataset with N = 8 × 10 9 samples (i.e., one samplefor each person in the world).
Introduction
Machine learning models are increasingly personalized 1 .
In applications such as healthcare, modelsoften use group attributes – categorical features to encode personal characteristics (e g., sex, agegroup), sensitive information (HIV status) or information that must be acquired at prediction time(blood type, malignant biopsy, see e.g., Taylor et al., 2017; Bertsimas et al., 2019).
The collection anduse of group attributes stems from the need to improve performance (Fan and Poole, 2006; McAuley,2022).
Group attributes divide the population into subgroups distinguished by the combination offeature values (e.g., women with ages above 70 years).
At first, one might assume that a personalizedmodel that uses group data produces predictions that are at least as accurate compared to a nonpersonalized model for all subgroups.
Otherwise, the personalization is unfair: a subgroup isharmed when group data is included in the prediction model.
Thus, an essential goal for fairness inpersonalized models is to ensure fair use — i.e., to ensure that models using group attributes producea tailored improvement in accuracy for all subgroups (Ustun et al., 2019; Suriyakumar et al., 2022).
Standard practices in empirical risk minimization do not ensure fair use in personalization.
Personalized models are trained to minimize empirical risk at a population-level, which does not necessarilylead to better prediction for all subgroups.
In fact, health technology companies have faced backlash over mounting concerns about the fairness of their algorithms on different racial groups (Vartan, 2019).
However, testing personalized models for fair use on all (potentially exponentially) many subgroupsinduced by different values of group attributes is challenging (Kearns et al., 2018; Hebert-Johnsonet al., 2018).
It may be the case that, for specific subgroups, a group-blind model that does not usegroup attributes may perform better than a personalized one (Suriyakumar et al., 2022).
We propose a metric called the benefit of personalization (BoP) that measures the minimum gain inaccuracy across all subgroups by using a personalized model instead of a group-blind model.
BoPmust be positive whenever a personalized model is used—otherwise, at least one group is harmed bychoosing a personalized model.
Estimating the BoP empowers individual stakeholders to ensure theirpersonal group information is used fairly and that they receive a more accurate prediction in returnfor their data.
Moreover, reporting the BoP enables practitioners to justify the collection of groupinformation for personalized models.
The goal of this paper is to understand the conditions under which one can check for fair use ofgroup data in prediction models and, more interestingly, the conditions where estimating fair useis impossible.
We consider a setting where only a finite number of samples of the underlyingpopulation is available to audit a set of personalized models for a BoP.
The number of subgroupsgrows exponentially with the number of group attributes used for personalization.
As we will show,when there are many attributes and not enough samples, reliable estimation and testing for BoPbecome impossible.
This is, of course, expected: an exponential increase in group-denoting attributesinevitably requires more samples per subgroup—in fact, this is the motivation behind auditing forcomputationally identifiable groups (e.g., Hebert-Johnson et al., 2018).
What is surprising in ourresults is that, given a finite number of samples (at most one sample for each person in the world≈ 8 × 10 9 ), the information-theoretic maximum of group attributes that can be collected and stillused to verify for a BoP is, in fact, small: with more than 19 binary attributes, it is ( statistically impossible to test a personalized model for a BoP without further assumptions on the underlying data distribution.
The main contributions of this work include:
We propose a metric, the benefit of personalization , which quantifies the minimum gain in accuracyby using group attributes across all subgroups for a prediction model.2.
We design a hypothesis test framework to audit if a personalized model violates fair use of groupattributes.
We pair our test with an information-theoretic limit on the error probability that holdsfor any hypothesis test that audits a model for fair use.
A rule of thumb is that the number ofsamples per subgroup should be greater than the number of subgroups for testing error to be small.3.
We characterize statistical limits on the Mean Square Error (MSE) of any estimate of BoP. Ourresults show that we cannot reliably estimate the benefit of personalization across subgroups formodels that use more than 25 personal attributes.
We conduct experiments to demonstrate the behavior of the probability of error in testing for fairuse.
Our experiments show that the MSE of a BoP estimation increases exponentially fast whenthe number of samples per group drops.
Finally, we evaluate our theoretical minimax bounds onthe MSE of the BoP estimation on real-world datasets.
1.1 Related Work
Algorithmic Fairness.
Benefit of personalization , our proposed metric, embodies a fairness principle that is distinct from fairness metrics that aim to quantify disparate impact.
Fairness metricscan be broadly characterized as individual fairness and group fairness.
The motivating philosophybehind individual fairness is to treat similar individuals similarly (Dwork et al., 2012), while groupfairness aims to ensure similar performance across different groups (Ustun et al., 2019).
An influentialprinciple that captures popular metrics like Equalized Odds and Equal Opportunity (Hardt et al.
,2016) is parity, which seeks to achieve proportional allocation of resources conditioned on the group(Dwork and Ilvento, 2018).
Although parity is an appropriate notion of fairness for applications inhiring and lending (Zafar et al., 2017), it is less suitable for domains with different ethical princi- ples.
For example in medical applications, the relevant principles are non-maleficence (do no harm)and beneficence (do the best) (Beauchamp et al., 2001).
In such settings, accuracy is paramount,and our proposed metric ( benefit of personalization ) can be used to personalize in a way that isnon-malefience.
Moreover, BoP is complementary to existing group-fairness metrics.
BoP may mask imbalancedperformance across groups, while group-fairness metrics do not capture the gain (or loss) in usinggroup attributes.
Hence, there are situations where both metrics are called for.
Auditing for Fairness.
Auditing whether an ML model is fair is a challenging and importantproblem.
In criminal justice, for example, Neil and Winship (2019) argue that testing for racialdiscrimination in policing can yield diverging results using different testing methods.
Berk et al.(2021) show that it may be impossible to simultaneously satisfy different fairness constraints.
Toregulate algorithmic filtering on social media, Cen and Shah (2021) develop a statistical test thatenforces a regulation.
In greater generality, many recent works explore causal approaches that testfor unfairness in models (Kilbertus et al., 2017; Nabi and Shpitser, 2018).
Si et al.
propose atesting algorithm using Optimal Transport Projections, only considering one protected attribute at atime without intersectional subgroups.
In this work, we prove an information-theoretic impossibilityresult on testing that works for all tests.
Multicalibration & Fairness Gerrymandering.
Testing for fairness is complicated in settingswith a richer class of subgroups.
Specifically, our framework resembles the Multi-X approach of Hebert-Johnson et al.
(2018) and Kearns et al.
Fairness constraints, such as equal opportunityand statistical parity, are imposed on all computationally identifiable – potentially exponentially-many– subgroups within a population.
In this setting, Kearns et al. (2018) show that auditing for fairnessis computationally-intractable for worst-case distributions.
Our impossibility result in Corollary1 answers a different question.
In particular, we characterize when the probability of error of anarbitrary auditing algorithm is too large to flag a concrete outcome.
We derive a fundamental limit onthe number of attributes a model can have to reliably test for fairness with any auditing algorithm.
Our findings differ from those of Kearns et al.
(2018) resulting from a key difference in the assumptionof sample size.
Specifically, in (Kearns et al., 2018), the authors assume that it is possible to havemore samples than the VC dimension of the model hypothesis class.
Note that for models like deepneural networks, the VC dimension approaches infinity (Sontag, 1998).
Our work builds on the keyobservation that with a finite population on earth, the number of available independent samples isfinite.
Thus, the available samples for each subgroup are limited.
Personalization, Privacy & Data Minimization.
Data is collected and used for building personalized algorithms.
When training personalized algorithms using personal data, prior work in both
Computer Science and Law advocate for including individual rights in data privacy laws (Kaminski,2022) (Isaak and Hanna, 2018).
Ustun et al.
(2019) argue that given the rights, a rational individualwill only provide information on personal data if the decoupled classifiers lead to higher performance.
This concept is called beneficence and is formalized in fair use.
Framework
In this section, we introduce notations and definitions used in this paper.
In particular, we define anew metric to evaluate fair use called the benefit of personalization (BOP).
2.1 Preliminaries
Consider a binary classification task where categorical attributes represent group membership.
Weaim to predict the label Y by training a classifier on the dataset D ≜ { ( X i , S i , Y i ) } Ni =1 , where
X i ∈
X is a vector of non-group features, and S i ∈ S ≜ { 0 , 1 } k is a vector of k group attributes(e.g.
sex, blood_type, HIV_status), and Y i ∈ { 0 , 1 } is a binary label.
The elements of S are binary k -dimensional vectors, which means that there are d ≜ 2 k groups that provide personal data.
Hence,we denote S as a one-hot encoding of the binary group attributes, with d = | S | = 2 k .
To simplify theanalysis, we consider a scenario where each subgroup has the same number of data points.
Let N bethe total number of samples, then the number of samples for each group is m ≜
⌊ N/d ⌋ data points.
We argue that this is the best-case scenario when prior knowledge of the dataset distribution is notknown: intuitively, the group with fewer samples will dominate testing or estimation performance(see supplementary material (SM) for a more detailed discussion).
We frame our impossibility resultsfor this best-case setting; an imbalance in samples will only render more difficult auditing for thebenefit of personalization.
Let the distribution of the dataset be P D .
At the group level, suppose that the distributions for eachgroup are independent, and we draw m i.i.d.
data points for each subgroup from the correspondingdistribution P j :
[Equation]
In this paper, we compare two classifiers, a group-blind classifier and a personalized classifier .
Specifically, the group-blind classifier h 0 only takes non-group attributes as input, while the personalized classifier h g takes both non-group and group attributes, i.e., h 0 :
X → { 0 , 1 } andh g : X × S → { 0 , 1 } .
h 0 is taken to be a baseline model trained in good faith, e.g. by empirical riskminimization on a dataset with group attributes removed.
A core principle that motivates our work is the Fair Use of group attributes (Ustun et al., 2019), wemake this definition mathematically precise next.
Definition 1 (Fair Use) .
Let h 0 be a group-blind classifier, h g a personalized classifier, and P j as in(1).
We say that h g follows the fair use principle if
[Equation]
2.2 The Benefit of Personalization
A personalized classifier outperforms a group-blind classifier when the metric is the overall accuracyof a model, i.e., average accuracy across all data points.
In contrast, a group-blind classifier can havehigher accuracy for specific subgroups in comparison with a personalized classifier (Suriyakumaret al., 2022).
We next introduce a metric called the benefit of personalization .
This metric captures the minimumgain in accuracy across all intersectional groups by using a personalized classifier and can be used toidentify fair use violations.
Definition 2.
(Benefit of Personalization).
Given a fixed data distribution P S,X,Y .
The benefit ofpersonalization is given by:
[Equation]
In addition, we call P j ( Y = h g
( X, S ))
− P j ( Y = h 0 ( X ))
the per-group benefit.
The benefit ofpersonalization is a function of both classifiers and the data distribution.
When the parameters areclear from the context, we denote γ ( h 0 , h g , P D ) = γ .
In this paper we choose accuracy as a benchmark metric for the BoP.
Our results still hold if accuracyis replaced with False Positive Rates (FPR) or False Negative Rates (FNR) (see supplementarymaterial for a more detailed discussion).
By the definition in (2), the BoP is the gain in accuracy by using a personalized classifier.
Forexample, γ = 0 .
4 means that, for every subgroup, a personalized classifier is at least 40% moreaccurate than the group-blind classifier.
Testing for Fair Use
In this section, we formulate fair use as a hypothesis test on the benefit of personalization.
We providean information-theoretic bound on the reliability of testing in terms of a trade-off between the numberof group attributes and the probability of error of the test for a fixed sample size.
Our bound leads tothe maximum number of group features that ensures reliable testing for fair use of group attributesand an interesting phase transition phenomenon on the probability of error in the hypothesis test.
Allproofs are provided in the supplementary material.
3.1 Hypothesis Testing Framework
The disclosure of group information to a model should incur a gain in accuracy.
It is critical toreliably test if, on average, a personalized model yields a gain in accuracy across all subgroups.
Thefollowing hypothesis test framework verifies if the benefit of personalization given in Def. 2 is abovea certain threshold ϵ .
[Equation]
[-( X, S )) − P j ( Y = h 
 ( X )) ≥ ϵ-] In practice, a personalized model should be deployed if there is evidence from data to reject the nullhypothesis ( H 0 ).
Otherwise, a personalized model should not be deployed, since there is not enoughevidence to conclude that all groups receive an average gain in exchange for revealing their personalgroup attributes.
Note that, when ϵ = 0 , the above hypothesis test is equivalent to verifying fair useof group attributes according to Defn.
3.2 Limits of Testing
In practice, the reliability of testing for fair use is governed by sample size: reliable testing requires aminimal number of samples per group.
In the extreme case where each group has only one sample,for example, it is impossible to test for fair use reliably.
Suppose that a practitioner trains a classifier with k group attributes using a dataset D with N ∈ Nsamples – i.e., m = N/ 2 k samples per group.
In Theorem 1, we characterize how the the precisionof the hypothesis test changes as a result of the exponential reduction of data at the group level in aminimax sense.
To characterize the trade-off between error probability, define V h as the indicator function that h doesnot satisfy the null hypothesis, i.e., for each classifier h , V h
=
if and only if h ∈ H 0 .
Theorem 1 (Error in Hypothesis Test) .
Let ϵ ∈
[ − 1 / 2 , 1 / 2] , h 0 be a group-blind classifier, and P Dbe the distribution of the dataset D .
Also, suppose that the dataset is described in Sec. 2.1.
Considerthe hypothesis regions H
= { h | E D [ γ ( h, h 0 )] > ϵ } , and H
= { h | E D
[ γ ( h, h 0 )] ≤ 0 } .
LetΨ :
H → { 0 , 1 } be a decision function, and define the error probability in the hypothesis test to be
P e given by:
[Equation]
Then,
[Equation]
[Equation]
Given that practitioners do not have access to the dataset distribution, Theorem 1 presents a minimaxlower bound on the probability of error in a hypothesis test.
The minimax lower bound implies thatfor every decision function Ψ , there exists a dataset distribution such that the probability of error forthe hypothesis test is greater than the lower bound in (5).
The proof of Theorem 1 constructs a lower bound the probability of error of any decision functionusing the total variation distance of two distributions (belonging to different hypotheses) that are“difficult” to distinguish.
This result uses a generalized version of Le Cam’s Lemma stated in Lemma1 (which might be of independent interest).
The original Le Cam Lemma (Le Cam, 1973) states that(6) holds when θ 1 , θ 2 are elements of Θ .
We generalize this lemma for the case where θ 1 , θ 2 areelements of Θ chosen at random.
Lemma 1 (Generalized Le Cam) .
Let { Λ , d } be a metric space, and D be a dataset.
Let P be afamily of probability distributions with parameter θ ∈ Θ .
Suppose that f is a statistics with image inΛ ( f ( θ ) for θ ∈ Θ means the statistics calculated for P θ ), and that there exists Λ 1 , Λ 2 subsets of Λthat are 2 δ -separated.
Also, suppose that Θ 1 and Θ 2 are subsets of Θ
such that f ( θ ) ∈ Λ 1 for allθ ∈ Θ 1 and f ( θ ) ∈ Λ 2 for all θ ∈ Λ 2 .
Then we have that:
[Equation]
(6) where θ i ∈ σ (Λ) with image in Θ i , and P θ
i indicates a probability distribution in P with parameterθ i for i ∈ [2] .
Threshold Test.
Given sufficient samples per group as outlined in Theorem 1, how should testingbe done in practice?
The most natural choice is a threshold test:
Given h g and h 0 , we simply evaluatethe performance of each classifier using an auditing dataset, and then check if there is an ϵ gain inaccuracy across all the groups.
Intuitively, the threshold test consists in first estimating the BoP(e.g., using empirical mean in the group or bootstrapping), and then checking if it is greater than thethreshold ϵ .
Note that the theorem holds for all possible tests (not only the threshold test).
3.3 Implications of the Limits of Testing
Given that the size of the dataset is smaller or equal to 8 × 10 9 (number of people on Earth), theorem1 yields a fundamental limit on the number of group attributes in order for the probability of error inthe hypothesis test to be smaller than 1 / 2 (a random choice).
Corollary 1 formalizes this limit.
Corollary 1 (Limit for Testing) .
With the same setup as in Theorem 1.
Take N = 8 × 10 9 , and letϵ = 0 .
If min max P e ≤ 1 / 2 then k ≤ 19 .
[Figure]
Phase Transition.
The lower bound in Theo- rem 1 gives a characterization for a phase transition in the hypothesis test error.
Specifically,consider that N = 8 × 10 9 , ϵ = 0 .
01 , d = 2 k ,and m = N/d .
For k = 18 attributes, the minimax lower bound for the probability of error is0 .
13 , in contrast for k = 19 attributes the minimax lower bound is 0 . 97 .
Hence, there existsa notable phase transition in the probability oferror in hypothesis test between k = 18 andk = 19 .
Figure 1 shows that, for a fixed numberof groups d = 141 and varying the number ofsamples per group, there is also a phase transition for the minimax error probability in a hypothesis test.
This phase transition can also beverified in the semi-synthetic data case in Figure2 Left .
Error Probability in Semi-Synthetic Data.
Figure 2 ( Left ) displays the error probability in ahypothesis test using the threshold test.
The error probability decreases when the number of datapoints per group increases.
Moreover, the figure shows that when the number of samples per groupis greater than the number of groups, the error probability is smaller or equal to 50% for the first
[Figure]
a t a p o i n t s p
e r g r o u p
[Figure]
time.
This is, indeed, a simple rule of thumb as a consequence of our results: the number of groupsshould never be greater than the number of samples per group; otherwise, testing for fair use isinformation-theoretically infeasible without prior assumptions on the underlying distributions.
In
Figure 2 ( Right ), we plot a heat map for the error probability in a hypothesis test using as thresholdthe values on the x-axis and as the number of data points per group the values of in the y-axis.
Forhigher values of ϵ , there is more "tolerance" for mistakes in the test, and the phase transition happensfor a smaller number of samples per group.
Estimating the Benefit of Personalization
When the benefit of personalization is positive but small, a user may opt-out of providing groupfeatures to a personalized model.
If such features are sensitive or costly to produce, their use mayonly be justified when a personalized model provides a significant accuracy gain.
Thus, testing for thebenefit of personalization (e.g., if γ > ϵ ) is, on its own, insufficient: stakeholders must be made awareof the estimated increase in accuracy from sharing personal data and only provide their personalinformation when there is a sufficient gain in accuracy (Ustun et al., 2019).
In this section, we consider the problem of estimating the benefit of personalization.
We deriveinformation-theoretic limits for estimation and show that, for a finite dataset, there is a threshold forthe number of group attributes required to reliably estimate the BoP with a small error.
4.1 Estimation Framework
We consider the problem of designing an estimator γ ∗ for the BoP given a dataset D with n samples.
Our metric of choice is the mean-square error (MSE) E (cid:2) ( γ − γ ∗ ) 2 (cid:3) .
Naturally, the value of theminimum MSE (MMSE) depends on the dataset distribution and the estimator of choice.
Insteadof restricting our analysis to specific estimators and datasets, we aim to understand the minimaxestimation error, i.e., the best MMSE achievable across all dataset distribution, defined as
[Equation]
To motivate the definition of R ∗ , suppose that there exist functions g 1 ( d, m ) , g 2 ( d, m ) > 0 such that
R ∗ ≤ g 1 ( d, m ) , and R ∗ ≥ g 2 ( d, m ) .
Recall that d is the number of subgroups and m is the number ofsamples per group.
This implies that for every estimator γ ∗ , there exists a dataset distribution P X,S,Ysuch that g 2 ( d, m ) ≤ E (cid:2) ( γ − γ ∗ ) 2 (cid:3) ≤ g 1 ( d, m ) .
In practice, this means that for any estimator there is a distribution such that the MSE of the resulting BoP estimate will be bounded away from zero byg 2 ( d, m ) .
The main result in this paper is Theorem 2, which gives explicit expressions for g 1 , g 2 .
Theorem 2 (Minimax bounds for the Estimation MSE) .
Given a dataset D , constants d, m ∈ N suchthat
d ≥ 4 , d ≤ 2 m +2 , a group-blind and a personalized classifier, the minimax estimation error forthe benefit of personalization in (7) is such that:
[Equation]
The proof of Theorem 2 relies on two-point methods for minimax lower bounds. Specifically, we usethe Generalized Le Cam (Lemma 1) together with standard concentration inequalities.
We provide adetailed proof in the supplementary material, and discuss its implications next.
4.2 Implications of The Limits of Estimating the BoP
For a dataset with a fixed amount of samples ( N ), given two personalized models with k 1 and k 2group attributes ( k 1 > k 2 ), then in the best case scenario, the number of samples per group willbe, respectively, N/ 2 k 1 < N/ 2 k 2 , hence the MSE for estimation will be higher on the “more"personalized classifier.
Theorem 2 gives a characterization for the order of the estimation error for the
BoP. The bounds in Theorem 2 are illustrated in Figure 3 ( Right ) in log scale.
This figure suggeststhat even knowing that the bounds in Theorem 2 are in the minimax sense, they are close to theestimated MSE, especially the lower bound.
We apply Theorem 2 to derive a fundamental limit on the number of binary group attributes toreliably estimate the BoP γ .
This lower (converse) bound in (8) captures the trade-off between theminimax MSE of an estimator, the number of sensitive attributes, and the dataset size.
This trade-offis instantiated in Corollary 2, resulting in an upper bound in the number of group attributes requiredto achieve a small minimax MSE in estimating the BoP given finite samples.
Corollary 2 (Limits of Estimation) .
Given a dataset D with N data points, d ≥ 4 groups, m = N/ddata points per group, an 0 < δ < 1 / 16 , a group-blind and a personalized classifier.
If d ≤ 2 m +2 ,and
[Equation]
then, the number of binary group attributes ( k ) satisfies
[Equation]
Specifically, for N = 8 × 10 9 (approximately the number of humans on the Earth) and δ = 1 / 400 =(5%) 2 , then one can only ensure reliable estimation using at most k ≤ 25 binary group attributes.
The proof of Corollary 2 is a direct application of the Theorem 2.
A more detailed proof is given inthe SM.
By choosing δ = 1 / 400 the square root of the MSE is at most 1 / 20 = 5% .
Hence, when theestimator γ ∗ is unbiased, Corollary 2 gives an upper bound of 5% in the estimation variance.
Corollary 2 gives a rule for practitioners to design algorithms that reliably determine the minimumgain from personalization.
During a decision-making process, such as deciding to take an examfor a medical diagnosis, the user should take advantage of an accurate estimate of the benefit ofpersonalization and make the best-informed decision.
In general, one would expect that the maximum number of group attributes for a reliable test shouldexceed the maximum number of group attributes for reliable estimation.
Our bounds (5), (8) showotherwise.
This is an artifact of the fact that | γ |∈ [0 , 1] .
Easier estimation for statistics with normsmaller than 1 is simply an artifact of scale and follows from the Le Cam Lemma.
Final Remarks and Limitations
Personalized ML models should provide performance benefits to all individuals who provide personaldata.
We call this gain the benefit of personalization (BoP).
In this work, we have proposed tools toestimate this quantity and determine if it is large enough to warrant the collection of personal data.
Our results suggest that we are limited in our ability to determine if personalization leads to equitablegains in finite-sample settings.
In particular, we show that there is no way to reliably determine if apersonalized model with k ≥ 19 attributes benefits all groups that provide personal data, even whenwe are given a dataset composed of N = 8 × 10 9 samples (i.e., one observation per person in theworld).
Limitations.
The minimax bounds hold for all decision functions and estimators and inherentlyconsider the worst-case distribution for each case.
However, its generality implies that some knowledge of the underlying data distribution may result in a more favorable trade-off between samplesize, estimation (or testing) error, and the number of group attributes.
In particular, in Figure 3, the
MSE in estimating BoP falls below the minimax lower bound, suggesting that the model’s underlyingdistributions for each subgroup is not quite the worst-case scenario.
Regardless, the upper boundof the MSE estimation and the impossibility result always hold in the general case when no priordistribution knowledge is known, as usual for minimax results.
A seeming constraint in our workis that we consider every group to have the same number of data points, but in Sec.
2.1 we arguethat this is the best-case scenario for testing and estimation.
Finally, fair use is a complement togroup fairness metrics, not a substitute.
If the base and personalized model are trained in bad-faithto underperform on a target group, then a positive BoP may mask a loss of accuracy on this group.
Conversely, group fairness does not capture the gain (or loss) in using group attributes.
Recommendations.
At the very least, ML practitioners should use the rules of thumb to controlthe number of group features to ensure reliable estimation and testing.
In addition to not exceeding19 attributes to ensure reliable testing, there should be more samples per group than the number ofsubgroups.
Failing to do so will require additional assumptions on the data to reliably estimate the
BoP or ensure fair use.
When one group has very few data points and estimating the BoP is unreliable,practitioners should reduce the number of groups or use the baseline model only for some groups.
Our bounds can guide on how the number of groups can be reduced to enable reliable estimationof the BoP. We encourage the report of estimated per-group gains in accuracy for all groups.
Thisenables stakeholders to make an informed decision on whether to share their personal information,particularly if this may incur a data collection or privacy cost.
Finally, if the BoP is accuratelyestimated and is negative, groups without a gain in personalization should revert to group-blindmodels.
A per-group estimate of the gain in accuracy can also be used to guide feature selection(Miao and Niu, 2016).