A Generic Framework for Conformal Fairness

Authors: Aditya T. Vadlamani, Anutam Srinivasan, Pranav Maneriker, Ali Payani, srinivasan parthasarathy
Primary area: alignment, fairness, safety, privacy, and societal considerations
Keywords: Fairness, Conformal Prediction, Graph Neural Networks

Abstract:
Conformal Prediction (CP) is a popular method for uncertainty quantification with machine learning models. While conformal prediction provides probabilistic guarantees regarding the coverage of the true label, these guarantees are agnostic to the presence of sensitive attributes within the dataset. In this work, we formalize \textit{Conformal Fairness}, a notion of fairness using conformal predictors, and provide a theoretically well-founded algorithm and associated framework to control for the gaps in coverage between different sensitive groups. Our framework leverages the exchangeability assumption (implicit to CP) rather than the typical IID assumption, allowing us to apply the notion of Conformal Fairness to data types and tasks that are not IID, such as graph data. Experiments were conducted on graph and tabular datasets to demonstrate that the algorithm can control fairness-related gaps in addition to coverage aligned with theoretical expectations.

Decision: accept (poster)
Reviewer scores: 6, 6, 6, 6

Reviews and discussion:
<thread>
creation date:
1730051050409

thread_type:
official_review

<thread_object>
creator: reviewer_3nsu

<content>
summary:
This paper proposes a notion of fairness, named $\textit{conformal fairness}$. In contrast with standard fairness notions (e.g. demographic parity), this notion significantly applies conformal mapping, which the authors clearly defined. The authors developed an algorithm to achieve this type of fairness on graph datas and confirmed with experiments.

strengths:
1. This paper is well-written with a clear structure.

2. The notion of conformal prediction is common, while its application on fairness is not widely studied to the best of my knowledge. The novelty of this notion is well justified.

3. The conformal fairness metrics in Table 1 are intuitive and well-founded.

weaknesses:
1. Although the conformal fairness metrics are well-defined in Table 1, they were not carefully discussed in later results (Theorem 3.4). Specifically, Theorem 3.4 states that "the coverage difference for Predictive Parity is satisfied". In Table 1, the definition of Predictive Parity relies on a conformal prediction $\mathcal{C}$, but the specific mapping/quantification is not discussed in the theorem statement.

2. This is a continuation of the previous point. Suppose there is a conformal prediction in Theorem 3.4, how good is it, i.e. what shall the value of $\alpha$ be, as in Equation 1?

questions:
Please see the weaknesses section.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730531783732

thread_type:
official_review

<thread_object>
creator: reviewer_vvws

<content>
summary:
This paper proposes a framework to check and find the suitable threshold for conformal learning. Further, the framework can be easily extended to some constraints, like fairness constraints. The authors specifically studied achieving fairness properties for graph models. Further, the authors studied how to use their framework to check the fairness property of a model. The authors' algorithm first find the right use inverse quantile to check whether a threshold is good.

strengths:
Conformal prediction is a very interesting area and there are many interesting works in this area. It is a natural question to ask how to achieve fairness for models in this setting. The authors provide a very general framework that can achieve many different definitions of algorithmic fairness. The paper also did many interesting experiments in graph datasets.

weaknesses:
I personally think the author does not elaborate enough on why they chose this algorithm design. The writing of this paper could be improved.

questions:
I think it would be very helpful for the authors to explain more about their design choices. For example, why choose inverse quantile for miscoverage level? Explain a bit more about the intuitions behind lemma 3.1 and 3.2 would really help the readers.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730672134932

thread_type:
official_review

<thread_object>
creator: reviewer_zt1t

<content>
summary:
The paper introduces "Conformal Fairness," a framework that extends Conformal Prediction (CP) to ensure fairness across sensitive groups. The authors develop a theoretically grounded algorithm to control gaps in coverage between these groups, leveraging the exchangeability assumption instead of the typical IID assumption. This allows the framework to be applied to non-IID data types, including graph data. Experiments on both graph and tabular datasets show that the framework can effectively control fairness gaps while maintaining coverage guarantees.

strengths:
- Originality: Introduces "Conformal Fairness," extending Conformal Prediction (CP) to address fairness in uncertainty quantification, especially for non-IID data.
- Quality: Provides strong theoretical backing with rigorous proofs and effective validation through experiments on graph and tabular datasets.
- Clarity: Clearly defined theoretical concepts and a stepwise algorithm description make the methodology accessible to those with relevant background knowledge. Providing additional background information on key concepts would help readers who are less familiar with the topics to follow more easily
- Significance: Tackles an important gap by combining fairness with uncertainty quantification. Adaptable to multiple fairness metrics and data types, making it broadly applicable in ethical AI contexts.

weaknesses:
- The paper lacks a detailed discussion of the fairness-efficiency trade-off. Quantifying acceptable efficiency losses when fairness is improved would make the results more actionable for practitioners balancing both aspects.

- The extension of the exchangeability assumption to real-world data may not always hold. Adding empirical evidence or discussion on when this assumption is valid in practice would make the claims more robust.

-Lack of comparison with all existing fairness-aware methods limits benchmarking. Adding baselines would clarify the framework's effectiveness.

questions:
- How does the framework handle scalability with multiple sensitive attributes, especially computationally?

- Why were existing fairness-aware methods not used as baselines for comparison?

- How practical is the exchangeability assumption for real-world graph data? Any empirical evidence?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730721504043

thread_type:
official_review

<thread_object>
creator: reviewer_st3n

<content>
summary:
The manuscript introduces and studies a general-purpose conformal fairness framework. It consists of an algorithm that can enforce various notions of fairness (such as demographic parity, equalized odds, predictive parity, and several others contained in the ML fairness literature) on the coverage of prediction sets, in the sense of satisfying appropriate conformal guarantees on expected coverage conditional on sensitive labels/attributes. To demonstrate practical performance of the framework given the various fairness notions, experimental evaluation is provided on a variety of supervised learning tasks; besides the standard setting, graph conformal prediction with fairness constraints is also demonstrated in experiments.

strengths:
The paper's main asset is a reasonably broad evaluation of the proposed methods on a variety of datasets where fairness concerns may necessitate using one of the tabulated conformal fairness metrics. Compared to prior works the experimental section appears more extensive, both in terms of metrics and datasets. Moreover, conformal fairness guarantees were displayed on graph data in addition to the standard supervised tasks. The results indicate that explicitly enforcing fairness according to any of the tabulated fairness criteria is in fact often necessary, as evidenced by vanilla methods not satisfying non-enforced fairness coverage constraints. Furthermore, no drastic deterioration in efficiency guarantees is observed, indicating that the price to pay for fairness conditional coverage may often be acceptable.

weaknesses:
--- The main weakness and bottleneck at this point is the writing of the manuscript. In particular, the writing of Section 3, which introduces the objectives, the general algorithm, and the analysis, is currently not acceptable. Indeed, lots of key notions and terms, both on the fairness side and on the conformal side, are not properly and unambiguously introduced and/or are discussed in arbitrary order. 
(Fairness: groups/group collections are never formally defined, nor are requirements on them specified (e.g. non-intersectionality etc); "filters" and "filter functions" are simply thrown into the presentation. No background on the fairness measures, no elaboration on how they are transformed into conformal analogues compared to original notions, except for an incoherent sentence starting with "Essentially achieved..." in line 130. Conformal prediction: auxiliary notions like "interval widths", "label miscoverages" appear right away in the pseudocode, which also uses clunky notation and for which the textual explanation is as confusing as the pseudocode notation.) This is not to say that one cannot fill in many of the details with enough imagination and enough expertise, but at the moment the writing is hardly structured and accessible enough.

--- The theoretical/methodological side does not offer strong novel contributions; for the most part it simply wraps class-conditional and group-conditional conformal methodology into fairness-specific terminology --- and is currently doing it in a somewhat confusing way, given the currently unsatisfactory presentation as stated above.

--- In addition, the literature background review, where it's claimed that there are "very few prior efforts" on fairness and conformal prediction, misses an established line of work on group-conditional fairness guarantees; these works study the enforcement of coverage guarantees (usually of both upper and lower bounds) on rich classes of subpopulations given by possibly arbitrarily overlapping groups.

R. Barber et al: The limits of distribution-free conditional predictive inference [Journal of IMA 2020]
C. Jung et al: Batch Multivalid Conformal Prediction [ICLR 2023]
Z. Deng et al: Happymap: A generalized multi-calibration method [ITCS 2023]
O. Bastani et al: Practical adversarial multivalid conformal prediction [NeurIPS 2022]
I. Gibbs et al: Conformal Prediction with Conditional Guarantees [2023]

questions:
The main desideratum is a substantial overhaul of the paper's writing, especially as it pertains to the methodology Section 3. 

There are other minor concerns, such as the exact conformal methods that the current proposed method is tested against. For instance, APS has been superseded in the literature by RAPS, so that should be included in the comparison. Also, importantly, at least a couple of the many recently appearing class-conditional and local conformal methods should also be included in the comparison for the sake of fairness; indeed, these methods are designed to be able to empirically attain partial versions of conditional coverage without receiving explicit constraints (such as fairness ones) as input, so they should provide a stronger baseline than methods like APS.

----------------------------------------------------------------------------------

UPDATE: The rebuttal, and the post-rebuttal manuscript update, address the immediate concern of improving the presentation of key terms/concepts in Section 3 and beyond. Other improvements such as updates to the experimental section were also made. Given these, I have raised my overall score to 6.
</content>
</thread_object>

</thread>
<thread>
creation date:
1732414530694

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
Overview Response to Reviewers and Area Chairs

comment:
We thank all the reviewers for taking the time to review our paper. We clarify the contributions and summarize the improvements to the presentation and the new results we have added.

### Contributions

Our work is focused on formalizing the notion of Conformal Fairness and presenting an algorithm and associated framework for practitioners to control for gaps in coverage between groups. Our framework differentiates itself from the existing literature in a few different aspects:

1. **_Group Information at Inference:_** Existing works in group conditional prediction are limited by the need for group information at inference time, which can be purposefully withheld. If group information is absent, the CF framework still provides group-conditional (mis)coverages that can be balanced for conformal fairness.

2. **_Balancing miscoverage for all groups and all labels:_** In our framework, fairness is formulated as the pairwise miscoverage disparity between groups. For certain notions of fairness (i.e., Equal Opportunity) we extend the fairness formulation to condition on the label being positive when measuring disparity.
   \
    \
   To balance the miscoverages between groups, we use the inverse quantile to determine the miscoverage for a given threshold. The inverse quantile operation is unique in this context and is not similarly utilized in the existing literature.

3. **_Framework Flexibility:_** Our work differs in the flexibility it provides a practitioner to explore and evaluate fairness on their own conformal prediction methods for common and user-defined fairness metrics. To demonstrate this flexibility, we consider a proxy-variant of Predictive Parity.

### Presentation Updates

Based on the feedback from all the reviewers (in particular Reviewer ST3N's), we have substantially tightened the writing of Section 3 to be more precise. Specific changes include:

- **_(Conformal) Fairness Metrics:_** We elaborate on how the standard fairness definitions are adapted to the conformal formulations. We will include descriptions of each fairness metric in Appendix A in a camera-ready version.
- **_Nomenclature:_** We introduce terms like “interval_width”, “filter_function”, and “label_miscoverage” earlier in the section, before presenting the algorithm, with more mathematical formalism (e.g. “filter_function” is presented as $F_M$) to reduce ambiguity and clunkiness. We will also include a symbols table in the Appendix.\
  \
  To further reduce possible ambiguity, we changed the phrase “closeness threshold” to “closeness criterion” to avoid overloading terminology, since threshold ($\lambda$) is also used when constructing the prediction sets.

Other writing changes we made in response to Reviewers ST3N, zT1T, VVws, and 3NSU are:

- Expanding the motivation/intuition of the theory with concrete examples.
- Clarifying Theorem 3.4 to more precisely state what is being controlled, when considering Predictive Parity.
- In Section 4.3, we include references and discussions for the papers that the reviewers have shared.

### New Results

Our paper is focused on making a fair version of conformal prediction, thus we compare it to standard CP methods. We demonstrate the framework’s effectiveness in improving those methods in terms of fairness. These results are presented in the main body and supplemental material. We also updated all of the figures and tables to include RAPS (as per ST3N).

We further note that existing works, shared by Reviewer ST3N, study group-conditional coverage guarantees they omit class conditionality which is a part of many fairness metrics. We include additional results adapting BatchGCP [1] as a threshold function for Demographic Parity (Figures E12 and E13) and Disparate Impact (Table E6) on the ACSIncome and ACSEducation datasets in Appendix E.6. The results demonstrate that the CF Framework outperforms methods in group-conditional CP in terms of fairness disparity, with a small cost to efficiency. For our other datasets (i.e. Pokec and Credit), the BatchGCP codebase requires additional changes to be accommodated.

For existing works that are at the intersection of fairness and conformal prediction, we note that some are an instantiation of our work [2], are too narrow of an application [3], or are designed for a different task or a single fairness metric [4] to act as a suitable baseline.

We have updated the main paper and parts of the supplementary to reflect all of the reviewer's comments, additional results, and the presentation as discussed above.
\
\
\
Again, we thank the reviewers for their detailed reviews and are happy to answer any further questions that may arise.

[1] C. Jung et al.: Batch Multivalid Conformal Prediction [ICLR 2023]\
[2] Wang et al.: Equal opportunity of coverage in fair regression. [NIPS 2023]\
[3] Lu et al.: Fair conformal predictors for applications in medical imaging. [AAAI 2022]\
[4] Liu et al.: Conformalized fairness via quantile regression. [NIPS 2022]
</content>
</thread_object>

</thread>


Metareview:
metareview:
The paper introduces the concept of Conformal Fairness and proposes a novel algorithm to control coverage gaps between groups in predictive models. This paper makes an important contribution to the fairness issue of conformal learning. More specifically, compared to the existing methods, the proposed algorithm  does not require group information at inference time, which is big advantage. Also, the method is very flexible and can be used with different user-specified fairness metric. I recommend acceptance after reading the paper and the reviewers' feedback.