Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.
review_content: summary: The paper proposes a post-processing mechanism to wrap black-box predictors such that their predictions become compatible with an ancillary criterion such as fairness. To this end, the authors propose to use so-called "$\alpha$-trees", an adaptation of decision trees, to learn correction coefficients that are used to change the output probabilities of a black-box model in the desired way. They further introduce adaptations of their algorithm that enable wrappers that satisfy fairness notions such as conditional value at risk, equality of opportunity and statistical parity.
To demonstrate the efficacy of the proposed approach, the paper derives a number of bounds, for instance on the divergence of the wrapped predictor from it's black-box source as well as on the amount of unfairness in the adjusted outputs. In addition, empirical results are provided on commonly used datasets.
strengths_and_weaknesses: ### Strengths
- The paper proposes a method to adjust the outputs of any black-box model to be compatible with a desired fairness goal. Having this ability is quite valuable given that the usage of pre-trained, off-the-shelf models is quite prevalent today which, however, may not be trained with any fairness considerations in mind.
- The paper provides a solid theoretical analysis of the guarantees obtainable from the proposed algorithm along several dimensions, such as the divergence of the wrapped model from its source and the amount of residual unfairness in the wrapped predictions.

### Weaknesses
- The paper has a number of clarity issues:
    - The intuition behind many definitions and results are not clear:
        - Some results (e.g. theorem 1 and 2, lemma 3) propose bounds, but I have no intuition about what they mean. How strict are they? How strict are they relative to the strength of the assumptions made? This makes it hard to understand whether they are meaningful.
        - Some definitions are unclear, see the questions part for more details.
        - A lot of the analysis starts off by introducing some definitions or statements whose meaning only becomes clear later on. It would be very helpful to first provide an intuition, e.g. say what the goal of introducing a certain concept is and what idea it captures, for better understanding the analysis.
    - In the intro, a number of desiderata for the post-processing process are introduced (a-f, and later g). While they mostly make sense, some (esp. c and d) are somewhat vague (see questions) and I am not sure what is meant by e.g. addressing them from a representation point of view and what the difference between the analytical and algorithmic perspective is. It feels like the points are also not addressed in a systematic manner, e.g. in section 4 (b) and (e,f) are mentioned and another point (g) is introduced, in section 5 (c) and (d, f) are mentioned. This scattering, in combination with the unexpressive names of the requirements makes it hard for me to keep track of them and make them overall not very useful. It would be helpful to formulate the requirements more clearly, address them in a more systematic way, and give them more meaningful names.
    - The paper introduces a lot of concepts, esp. in section 3, which subsequently seem to be under-used. For instance, section 3 sets up core concepts using measure spaces, but then this level of abstraction seems to be unnecessary for the remaining paper. It appears like the exposition could be simplified by using more standard and simpler probabilistic notation instead. I did not check all the proofs in the appendix though, maybe this level of abstraction is needed there, but it comes at a high cost in terms of notational complexity which seems not immediately justified.
    - Some concepts are introduced early (e.g. in section 3), but only instantiated a lot later in the paper, which makes it hard to understand them both when they first appear as well as when they are properly defined/used, because you need to go back and re-read their definitions. For example, $\eta_t$ is introduced in section 4, but not really instantiated until section 6 and similarly for mixture measure M in section 3. It would be great to provide at least some intuition behind those concepts when first introducing them and ideally moving their introductions and instantiations closer together.
- The implications of some of the theoretical results are unclear, i.e. what do the bounds mean? See clarity above for more details. This makes it hard to judge how useful the derived bounds are.
- Some assumptions might be unrealistic for practical deployment, most importantly the assumption of having access to a true posterior $\eta^\ast$. See the first question for more details.
questions: - The analysis, in particular also the methodology to achieve fair outputs, relies on $\eta^\ast$, which in section 3, line 85 is defined as the true posterior, approximated from data. What does that mean? Using which approximation scheme? How is it related to some potential other, ground truth posterior? And most importantly, how is the assumption of having access to a true posterior realistic in practice? Without this quantity the fairness constraints in section 6 do not work. There are experiments in section 8, so there must be some way to approximate $\eta^\ast$, but I don't know how it's done (though there seem to be limitations with small datasets).
- The proposed method for achieving fairness is based on splitting the $\alpha$-trees at their root based on the different sensitive feature values. This means that data cannot be shared between the different sub-trees corresponding to different groups. Could this lead to issues in cases where some sub-groups are minorities and thus, insufficient training data for them is available? The minority-setting can be an issue in many scenarios where fairness is of concern.
- Some concepts are not clear:
    - The paper uses the term "twisted" posterior, but it's not fully clear to me what the characteristic property of a twist is and whether it just means "bias" in the conventional way used in the fairness literature, or whether there is some additional meaning to it. Could you clarify that?
    - What is the mixture measure M introduced in section 3 and why do you need it?
    - Regarding requirements (c) and (d), what do you mean by strong algorithmic guarantees (on what?) and what is meant by explainability, e.g. instance-based or for the whole model, about the difference of $\eta_f$ from $\eta_u$ or $\eta_f$'s behavior in general, etc.?
- Minor clarifications:
    - Does theorem 2 imply that the loss is 0 when the conditions hold?
    - It would be good to define the domain of $\alpha$ in section 3. Not all values are permissible and I struggled to understand whether one would generally choose it from e.g. (0, 1), (-1, 1) (with exception of 0), $(-\infty, +\infty)$, etc.
- Potential extension (I do not expect this to be included in this paper): the proposed method can be used to correct twists/biases in black-box predictions. It might be interesting to explore whether the technique can be used to tackle issues other than fairness as well, for example in the context of robustness, to correct for distribution shifts between training and deployment.
limitations: The main practical limitations I can see are the assumption of the availability of a true posterior and the problem having sufficient data for learning $\alpha$-trees for minority groups. These issues are insufficiently addressed in the paper.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This paper proposes a wrapper function in the post-processing to correct the classifier for fairness results. It adopts a previous paper that modifies the improper loss function to make it fair. The proposed boosting algorithm, called $\alpha$-tree, corrects for fairness bias for three different notions: Conditional value at risk, Equality of opportunity, and Statistical Parity.
strengths_and_weaknesses: Disclaimer: I'm not an expert in fairness and the improper loss function and my expertise are in interpretability. I can not evaluate the theoretical section due to my incompetence, but this paper is mostly theoretical so my review will be quite shallow.

# Strengths
- Although the empirical performance is not as good as other methods, this wrapper method seems to be the first one that can adapt to different fairness criteria in a post-hoc way. And the conclusion also admits that.

# Weaknesses
- Questionable claim about interpretability.
questions: I'm wondering why this algorithm is claimed to be interpretable? The only thing related is this method uses a tree-based model to correct for the bias. But an algorithm consisting of multiple trees like boosting or random forests can be equally complex as a DNN and thus should not be claimed interpretable.
limitations: None
review_title: null

final_score: rating: 4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.
review_content: summary: The paper proposes a way to correct black-box models to fulfill certain notions of group fairness via post-processing.
Concretely, the authors suggest to reweigh the posterior distribution of a classifier via a learned decision tree.
To this end, the potential bias of a black-box model is cast as a so-called twist. By considering a well-constructed loss, an untwisting function can be learned by minimizing this loss. The authors instantiate this for decision tree learning and three different notions of group fairness (conditional value at risk, equal opportunity, and statistical parity).
Two variants of their decision tree learning algorithm,  a conservative one, which sacrifices less accuracy to achieve fairness, and an audacious one, which allows larger changes in favor of fairness but potentially costing accuracy, are considered.

This is experimentally evaluated on tabular datasets. The proposed approach outperforms different baselines in terms of fairness notion.
Interestingly, empirically it appears that the audacious and conservative versions don't exhibit the trade-off discussed above, but depending on the setting either one is strictly better (both in terms of fairness metric and accuracy).


strengths_and_weaknesses: 
## Strengths
- Mathematically rigorous derivation.
- Important/interesting use-case: e.g. finetuning a blackbox model to ensure fairness.

## Weaknesses
- Hard to read. (See "Presentation" below.)
- Relation to some related work is unclear. (See "Related" below.)

## Presentation & Evaluation
While I appreciate the mathematical rigor I found the overall work hard to read.
In particular, I think the mathematical presentation almost hides the conceptual approach: i) get blackbox model, ii) train decision tree to debias the posterior. While the choice of loss to train the decision tree is theoretically well-motivated, in the end, it appears quite standard. 
Similarly, it remains unclear what the exact contribution and what an instantiation of a prior approach is, e.g. TopDown from [14]. (Note: I don't think instantiating prior work to reach a particular goal is problematic.)

Further, on the first read, it was unclear to me that section 3 recounts background from [24] rather than a new contribution. At the same time, it appears not to be self-contained as binary task is taken from [24] without much explanation.

I also found the evaluation section hard to read as all results are pressed in Figure 4. The selection of results is also quite small (although I appreciated the full additional results in the supplementary material). Lastly, it seems that the accuracy of the classifier is quite bad (see Questions below).

Minor Presentation issues:
- L240 "of mapping" seems to miss an article.
- "Limitations and conclusion" (L335) does not have a section number.
- While normally I don't mind if papers use modified vspace settings, it seems a little overdone here to the point where pages aver visually very dense. 

## Related Work
The paper does not discuss to which fairness notions it is limited to, i.e. group fairness, and how this relates to other notions of fairness, i.e. individual fairness, and how they are applied in the research community.

questions: - Can you clarify which parts are your key contributions, especially compared to [24] and [14]?
- What does MD (in Figure 4 and Section 8) measure? Is it the accuracy? If so, it seems bad for a binary task.


limitations: The authors provide a Limitation section, which seems to be adequate.
review_title: null

Meta-review: metareview: This paper tackles the important and interesting problem of how to transform black-box models so that their outputs having improved fairness.  The proposal of using an "alpha-tree", an axis-aligned decision tree that re-weights the existing model, seems elegant and does indeed have some useful form of interpretability.  The primary issue---which was essentially raised by all reviewers---was that of the paper's clarity: the presentation often prioritizes rigor over readability.  I think the authors have done a good job in the comments explaining their work, and as no technical issues were raised, I recommend the paper for acceptance.  However, the authors should take steps to improve readability (e.g. my incorporating many of the discussion comments within the text, as space allows).
confidence: Less certain
Decision: Accept