Paper: Measuring Fairness Using Probable Segmentation for Continuous Sensitive Attributes (ICLR-2024)

Reviewer comment (Novelty):
Very limited related work is presented. For example, are there literature studying other metrics such as equalized odds on continuous sensitive attributes? Are mean prediction disparity and maximum prediction disparity the only approaches for continuous sensitive attributes for demographic parity? Part of the discussion in the introduction should be put in the related work section. There are also existing literature using distance covariance to measure independence between the sensitive attributes and the predictions [1, 2]. They also allow continuous sensitive attributes.

Author response:
A. To the best of our knowledge, there is no paper for continuous sensitive attribute's equalized odds, which means that the continuous sensitive attribute’s fairness has been very limited usage and under-explored. Also, the previous methods for discrete sensitive attributes are not directly applicable. For example, although the fairness of continuous sensitive attributes can be defined and expanded from the references [1, 2], the metric is not computable from the sampled data. This is why we focus on the fairness metric of continuous sensitive attributes.

Reviewer comment (Experiments):
There is no guidance on what alpha value should be chosen for the proposed metric although the alpha value will greatly impact the metric.

Author response:
A. We can choose alpha at a training and inference stage. At first, we experiment the impact of alpha for training. We evaluate the PDP with various alpha for each fixed alpha at the training stage.

|Training \ Inference| alpha=0.1 | alpha=0.2 | alpha=0.3|
|--|--|--|--|
| **alpha=0.0** | 13.11 | 11.74 | 9.96|
| **alpha=0.1** | 9.05 | 8.87 | 7.13 |
| **alpha=0.2** | 9.15 | 7.64 | 6.59 |
| **alpha=0.3** | 8.62 | 7.65 | 7.03 |

As a result, choosing the alpha and training using PDP with this alpha leads to a decrease of various PDP metrics for different alpha, which means that the training process is robust for alpha. For example, training the model with alpha=0.1 leads to a decrease in PDP of alpha=0.2 and 0.3, both. That is, for training, choosing alpha is a simple hyperparameter that we can choose using the performance of validation data.

For inference, we need a more exquisite strategy. In a real scenario, a fairness metric is used to evaluating the model’s ability. When the given model satisfies “epsilon > metric” where epsilon is a hyperparameter, the model is said to satisfy epsilon-fairness. So, when the epsilon is fixed, we can find the minimum alpha that satisfies epsilon > PDP, since the PDP tends to get smaller with increasing alpha as shown in Table 1, 2. Finally, we can analyze the results of this process. When we have minimum alpha_m that satisfies epsilon > PDP with alpha_m, our model is fair at least for the segments that have a larger probability than alpha_m. For this reason, alpha is not a fixed hyperparameter, but a meaningful and versatile parameter.

Reviewer comment (Experiments):
The authors have claimed superiority of the proposed metric over mean prediction disparity and maximum prediction disparity. However, there is no experiment demonstrating such superiority. The evaluation has no other baseline.

Author response:
At first, Fig 1. (a) and (b) illustrate the disadvantages of mean prediction disparity metric and maximum disparity based metric, respectively. The mean disparity based metric underestimates the low probability segment, which is problematic in that the low probability segment usually evokes inequality. On the contrary, the maximum disparity based metric overestimates the low probability segment, which has unstable estimation. PDP aims to balance the two methods by proposing probable segment, which is guaranteed by the stable variance as in thm 2. Figure 2 also describes the detailed results for this. The upper figures of Figure 2 illustrate the estimation error of every segment. In this figure, we can observe that the estimation error is much worse than the probable segmentation, even though some segments include more than one sample. Thus, we can figure out that the estimation error of maximum disparity based metric would be much worse than PDP.

Reviewer comment (Experiments):
How do you choose the appropriate alpha for the proposed metric?

Author response:
A. We can choose alpha at a training and inference stage. At first, we experiment the impact of alpha for training. We evaluate the PDP with various alpha for each fixed alpha at the training stage.

|Training \ Inference| alpha=0.1 | alpha=0.2 | alpha=0.3|
|--|--|--|--|
| **alpha=0.0** | 13.11 | 11.74 | 9.96|
| **alpha=0.1** | 9.05 | 8.87 | 7.13 |
| **alpha=0.2** | 9.15 | 7.64 | 6.59 |
| **alpha=0.3** | 8.62 | 7.65 | 7.03 |

As a result, choosing the alpha and training using PDP with this alpha leads to a decrease of various PDP metrics for different alpha, which means that the training process is robust for alpha. For example, training the model with alpha=0.1 leads to a decrease in PDP of alpha=0.2 and 0.3, both. That is, for training, choosing alpha is a simple hyperparameter that we can choose using the performance of validation data.

For inference, we need a more exquisite strategy. In a real scenario, a fairness metric is used to evaluating the model’s ability. When the given model satisfies “epsilon > metric” where epsilon is a hyperparameter, the model is said to satisfy epsilon-fairness. So, when the epsilon is fixed, we can find the minimum alpha that satisfies epsilon > PDP, since the PDP tends to get smaller with increasing alpha as shown in Table 1, 2. Finally, we can analyze the results of this process. When we have minimum alpha_m that satisfies epsilon > PDP with alpha_m, our model is fair at least for the segments that have a larger probability than alpha_m. For this reason, alpha is not a fixed hyperparameter, but a meaningful and versatile parameter.

Reviewer comment (Experiments):
Can you show in the experiments that the proposed metric is better than mean prediction disparity and maximum prediction disparity?

Author response:
At first, Fig 1. (a) and (b) illustrate the disadvantages of mean prediction disparity metric and maximum disparity based metric, respectively. The mean disparity based metric underestimates the low probability segment, which is problematic in that the low probability segment usually evokes inequality. On the contrary, the maximum disparity based metric overestimates the low probability segment, which has unstable estimation. PDP aims to balance the two methods by proposing probable segment, which is guaranteed by the stable variance as in thm 2. Figure 2 also describes the detailed results for this. The upper figures of Figure 2 illustrate the estimation error of every segment. In this figure, we can observe that the estimation error is much worse than the probable segmentation, even though some segments include more than one sample. Thus, we can figure out that the estimation error of maximum disparity based metric would be much worse than PDP.

Reviewer comment (Evaluation):
What do you think of the distance covariance based metrics from [1] and [2] listed in my Weaknesses when compared to your proposed metric?

Author response:
A. To the best of our knowledge, there is no paper for continuous sensitive attribute's equalized odds, which means that the continuous sensitive attribute’s fairness has been very limited usage and under-explored. Also, the previous methods for discrete sensitive attributes are not directly applicable. For example, although the fairness of continuous sensitive attributes can be defined and expanded from the references [1, 2], the metric is not computable from the sampled data. This is why we focus on the fairness metric of continuous sensitive attributes.

Reviewer comment (Evaluation):
How can the proposed metric be adapted to multiple sensitive attributes? When the number of sensitive attributes grows, it becomes difficult to find the segmentations.

Author response:
A. We agree that there is a possibility that the computation of probable segmentation would be increased with the growth of the number of sensitive attributes. However, when the samples are finite, we can only consider the segments that are available for the given samples. Thus, we can directly adapt our metric to the multiple sensitive attribute case. Detailed computational complexity and an optimal computation method remain as a future work.

Reviewer comment (Experiments):
No validation procedure for choice of alpha proposed

Author response:
**Additional information for alpha in the table**
There are two types of alpha that we can choose, because we trained the model using the loss function L + r*PDP_a1 and tested using PDP_a2. In the original manuscript, we used identical a at the training and inference stage, i.e., a1=a2. So. we experimented when a1!=a2. Even though the a1=0.2, PDP is decreased at a2=0.1, 0.2, and 0.3, which means that the choice of a1 at training stage is not sensitive.

**Additional opinion for the choice of a**
There are two perspective for selection of a.
- The first perspective is related to the robustness of PDP and difference to the DP. Since low a leads to high variance and small max|h_a(k) - k|, there is a trade-off. In here, we can leverage a heuristic for a when the true distribution is unknown. Firstly, we can estimate the E_a:=max|h_a(k) - k| with growing and linear spaced a such as a=(0.1, 0.2, 0.3, … max_a) and a=(0.05, 0.1, 0.15 .., max_a). From this, we can compute E_a[i] - E_a[i+1]. As you suggested, we would choose a where the difference is drastic, because it means that max|h_a(k) - k| is drastically improved and so the bound of Thm 4 is effected.
- The second perspective is the meaning of PDP itself, as we argued at the previous response. PDP_a < e denotes that even though the model is not fair (Y_h is not independent to S), the model is fair at least the segments with the probability a (and we can derive Y_h is independent to quantized S, as Thm 3). In this case, the variance would be internal risk of the data.

Reviewer comment (Experiments):
How do you choose alpha?

Author response:
**Q. Appropriate alpha for the proposed metric**

A. We can choose alpha at a training and inference stage. At first, we experiment the impact of alpha for training. We evaluate the PDP with various alpha for each fixed alpha at the training stage.

|Training \ Inference| alpha=0.1 | alpha=0.2 | alpha=0.3|
|--|--|--|--|
| **alpha=0.0** | 13.11 | 11.74 | 9.96|
| **alpha=0.1** | 9.05 | 8.87 | 7.13 |
| **alpha=0.2** | 9.15 | 7.64 | 6.59 |
| **alpha=0.3** | 8.62 | 7.65 | 7.03 |

As a result, choosing the alpha and training using PDP with this alpha leads to a decrease of various PDP metrics for different alpha, which means that the training process is robust for alpha. For example, training the model with alpha=0.1 leads to a decrease in PDP of alpha=0.2 and 0.3, both. That is, for training, choosing alpha is a simple hyperparameter that we can choose using the performance of validation data.

For inference, we need a more exquisite strategy. In a real scenario, a fairness metric is used to evaluating the model’s ability. When the given model satisfies “epsilon > metric” where epsilon is a hyperparameter, the model is said to satisfy epsilon-fairness. So, when the epsilon is fixed, we can find the minimum alpha that satisfies epsilon > PDP, since the PDP tends to get smaller with increasing alpha as shown in Table 1, 2. Finally, we can analyze the results of this process. When we have minimum alpha_m that satisfies epsilon > PDP with alpha_m, our model is fair at least for the segments that have a larger probability than alpha_m. For this reason, alpha is not a fixed hyperparameter, but a meaningful and versatile parameter.

However, it is still true that the estimation is stable under small alpha. One heuristic solution for this is to control the epsilon with the alpha. As an example, when we set epsilon = 0.1 for alpha = 0.1 as a fairness criterion for our task, using the larger epsilon such as 0.2 for larger alpha as 0.2 would be beneficial concerning the estimation stability.

Reviewer comment (Theory):
Do you need to simultaneously control the difference between M and \tilde{M} for more than one point k?

Author response:
**Q. Do you need to simultaneously control the difference between M and \tilde{M} for more than one point k?**

A. Since PDP is based on the maximum disparity of M, we consider various k on the algorithm to find out the maximum disparity estimation.

In addition, we are thankful for pointing out the related references. We consider 1) Tibshirani, Ryan. "Nonparametric Regression: Nearest Neighbors and Kernels., 2) Hastie, Trevor, et al. The elements of statistical learning: data mining, inference, and prediction. Vol. 2. New York: springer, 2009., and 3) Wasserman, Larry. All of nonparametric statistics. Springer Science & Business Media, 2006. as the reference and reflect these on our final version.

Reviewer comment (Theory):
What are the conditions that the RHS in Theorem 4 goes to 0 as alpha -> 0 and at what rate does it converge?

Author response:
**Q. Detailed discussion on Thm 4**

A1. We additionally explore the specific condition that the RHS of Thm 4 goes to 0 as alpha -> 0. Suppose that the probability density function has a lower bound L_p. Then, |h_alpha(k) - k| <= alpha/L_p holds because there is a lower bound for the probability density function. Finally, we can find out that if alpha -> 0, the RHS goes to 0. In short, if there exists a lower bound L_p of a probability density function, the RHS of Thm 4 goes to 0 as alpha -> 0.

A2. Here, we also want to respond to your other concern related to the Thm 4. As you noted, it is correct that the bound of Thm 4 is derived from the true distribution. However, we think it is still meaningful since our contribution is that proving the difference between DP and PDP is bounded and reveals the detailed bound from the true distribution, even if it is impossible to estimate the bound from the sampled data directly.