Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: The paper contributes to fairness literature by proposing a new metric, the benefit of personalization (BoP), to measure the benefit of including group attributes in a binary prediction model. The metric measures the worst-case improvement across subgroups, which is attractive in settings where parity is not a reasonable metric, but "do no harm" is an appropriate metric, such as healthcare. The paper considers two settings in which the metric may be used, hypothesis testing and estimation, and provides general characterizations on the reliability of the metric. Specifically, the paper bounds the worst-case error in hypothesis testing and the MSE of the BoP estimator relative to the number of sub-groups and the number of samples per subgroup. Intuitively, they show the BoP metric is more reliable as the number of samples per subgroup increases. These results are then used to characterize general rules of thumb on when BoP estimator is reliable for estimation or testing. Numerical experiments are provided throughout to highlight how well the BoP metric performs empirically in both testing and estimation settings.
strengths_and_weaknesses: This is a well-written paper that provides an interesting metric, BoP, to address a rapid expansion of personalization in machine learning. The paper does a good job of overviewing the setting and the motivation for developing the metric and provides useful characterizations of the metric. By considering worst-case bounds, the paper can construct conservative rules outlining when the metric can be used to make the critical decision of whether or not to introduce group attributes into personalization machine learning models. The paper seems technically sound, and the results are well presented. 

The main drawback of the paper may lie in the significance of the BoP itself, as it only considers the worst-case improvement of the machine learning models, which may be overly conservative. The BoP metric thus can only make the simple and coarse decision of introducing group attributes or not introducing group attributes to a model. In contrast, a potentially more exciting problem could be deciding which subgroups utilize and which subgroups do not utilize the machine learning model with group attributes. This example reflects a more granular version of the problem BoP tries to solve and captures the fair use problem similarly. 
questions: 1. Is it possible that a smaller subset of group attributes produces positive BoP and all group attributes produces a negative BoP? For example, I can imagine a scenario that for 10 group attributes, there may exist a subset of the group attributes that improve classification accuracy of the machine learning model but including all 10 may decrease the classification accuracy of a subgroup. 

2. As a follow up to 1., is it possible to comment on the reliability of BoP in the setting that you perform the hypothesis test on different subsets of the group attributes? Can you just apply existing results in multiple hypothesis testing to do so? This would potentially allow you to identify the smaller subset of group attributes in question 1. 

3. Similar to 2., does the MSE result (Theorem 2) hold uniformly over different subsets of group attributes?

4. Does focusing on accuracy for BoP guarantee improvements in the false negative and false positive rate? If not, can the BoP metric be modified to measure only false negatives or only false positives rather than accuracy? 

5. For Figure 2 Right, do you mean higher values of $\| \epsilon \|$ have higher tolerance for mistakes in the test?  

6. In Corollary 2, is there a reason that the 5% threshold was chosen as the default benchmark? It may be important to clarify the choice as it directly affects the maximum number of binary group attributes that still guarantee reliable estimation of BoP.



limitations: The authors mainly address limitations related to the drawbacks of their worst-case results in settings where stronger assumptions may be made on the underlying data distribution. There isn't a discussion of the negative societal impact of their work, but it is unclear if there is a specific society impact they can discuss. Perhaps further discussion can be made on why accuracy was chosen as the benchmark metric rather than something similar such as the false positive rate or false negative rate.
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: The authors propose a new fairness notion called benefit or personalisation(BoP), in which a personalized classifer (i.e. a classifier that is specific to subgroups) is compared to a classifier that does not use group-membership for prediction. The authors propose a hypothesis testing framework and give a lower bound on the sample complexity for successful testing. The paper then shows tight upper and lower bounds on the MSE estimation of BoP. 
strengths_and_weaknesses: strength:

I like the idea of benefit of personalisation. As a fairness consideration it makes sense and I have not seen it in the literature. 
I agree with the opinion that this should be a consideration taken into account when deploying personalized predictions

I think it was a great idea to make sample sizes concrete and comparing them to the number of people currently alive. I think this made the point very effectively and might be a helpful method when analyzing sample complexities in the context of fairness or human-centric ML in general.
The emphasis on giving a clear message to practitioners that derives from these theorems was appreciated.

The bounds of theorem 2 are tight, making them quite informative. 

several limitations of the paper are addressed

the paper is overall well written. The related work section is sufficiently detailed.

weaknesses:

It is unclear to me what h_0 is. I think this is a crucial thing to discuss. A practitioner might devise a model that (while not explicitly using group information) might be bad on certain subgroups as a comparison. Using such a model as a bad-faith comparison is currently not discouraged or addressed in the paper. I think in practice it might also be unclear how to choose the baseline model. 

One might come up with a model that uses the baseline model for certain groups in which there aren't enough samples. If the models are the same for those groups this might be checked deterministically, not needing any samples for this group. Thus testing could be more efficient in such cases. Groups might also be found via hierarchal clustering to account for group-size. This might solve the issue of having to test a number of groups which is exponential in the number of features. I think these alternative views should be highlighted in the paper. Assuming that each subgroup needs to be viewed independently and they cannot be grouped together might be misleading.

guarantees are worst-case in the distribution; distributions in the real world might be nicer (this concern was raised by the authors themselves)
----

I think this line of work has a lot of potential and there are many parts I like about this paper. Ultimately I think there are still some unanswered questions at this point that I would wish the paper to address in its final version. 
questions: How do you pick the baseline model? How do you make sure that the baseline model is not already biased against certain subgroups?
Can we have personalization only for groups big enough?
limitations: I raised some concerns in the questions part. I think some warning about choosing the baseline model is required.
review_title: null

final_score: rating: 4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.
review_content: summary: The authors propose a new metric, benefit of personalization (BoP), to quantify the minimum gain in accuracy by using group attributes across all subgroups for a prediction model. They also develop a hypothesis testing framework to test fair-use of group attributes. They develop an information-theoretic limit on the error probability for any hypothesis test which is auditing a model for fair-use. They also characterize the statistical limit on the MSE for any estimate of BoP. Finally they conduct experiments to demonstrate the behavior of the probability of error in testing for fair-use
strengths_and_weaknesses: The paper is well written and is easy to follow. The associated theoretical results look good. 
questions: The major comment is on the definition of the benefit of personalization metric. I agree with Definition 1 (Fair Use) where we would expect for each $j$ the accuracy of the personalized model should be *equal* and/or greater than that of the group blind model. However, when defining BoP, taking the min over j, a lot of the information is actually lost.

Especially in the corner case where we benefit every group but one where we get equality, the gamma turns out to be zero since we are taking the minimum. Hence from the recommendation point of view, forcing the test on BoP > epsilon would not let us deploy any of these models where we are benefitting all group lest one (or a few).

In such situations, it is much more preferable to deploy a new personalized model to the other groups while keeping the base model for that group. Being able to quantify such a metric where are not actively harming a group, but making it better for the majority of others should be a good forward step.

Can the authors throw some light on this nuance and actively discuss how to change the metric towards this direction? Also curious how the Information-Theoretic results will change with such a new metric. 


limitations: The major limitation is on the recommendation based on the metric defined in this paper. The details are highlighted in the question above. But following the recommendation would make us miss a big corner case where in practice it is not possible to improve accuracy for each  group but possible for a wide range and keeping it same for others.
review_title: null

Meta-review: metareview: Thank you for submitting your paper to NeurIPS! This paper proposes a new metric, the benefit of personalization (BoP), to verify fair use of predictive models that have varied subgroup-level performance. Reviewers raised a variety of concerns about how the approach should be used in real-world contexts. I believe that these concerns were satisfactorily addressed by the authors and therefore I recommend acceptance (despite the fact that the most negative reviewer failed to engage in the discussion and update their score). There were additional concerns raised by the ethics reviewers that authors did not have a chance to address, but that I fully expect the authors to incorporate in the body of the paper (in addition to the concerns flagged by the reviewers).

Let me summarize the concerns raised in the ethics reviews. First, the choice of the baseline is critical -- poor selection of the baseline could result in very low performance on specific groups to begin with, allowing the improvement criterion to be met on these groups relatively easily, despite potentially large disparities in performance across groups. Second, fairness can be context-dependent, e.g. in clinical trials, improvements in any subgroup is desirable even if some subgroups do not benefit; on the other hand, the proposed definition aligns incentives to improve performance for all subgroups. These trade-offs are crucial and merit discussion. Finally, there was concern regarding whether subgroup accuracy is a suitable stand-in for subgroup preference and/or harm, i.e. can increasing accuracy (according to the target label Y) lead to worse outcomes for a subgroup? See, for example, work by Ustun et al. on risk assessment trained on proxy targets (re-arrest rate for risk of recidivism, or health care costs for risk of adverse health outcomes), where increasing accuracy may not be desirable (e.g. due to sampling bias). I strongly urge the authors to discuss these issues (as they have partly done already) and highlight them in prominent portions of the paper.
confidence: Certain
Decision: Accept