Paper: Empirical Likelihood for Fair Classification (ICLR-2024)

Reviewer comment (Experiments):
The simulation results in Table 2 do not seem to be a fair comparison. I would suggest adding more granularity for Zafar et al. with 0<c<0.1.

Author response:
Response to Weakness 1：We have included results for Zafar et al. (2017) with $0<c<0.1$ in our response to Question 1.

Response to Question 1:  In the following table, we present the outcomes of Zafar et al. (2017) with $0<c<0.1$. Notably, none of these results exhibit superior performance in both accuracy and fairness metrics when compared with our method. Our method achieves a higher (or equal) accuracy and a higher fairness than Zafar et al. (2017): $\alpha=0.5$ vs. $c=0.035$, $\alpha=0.1$ vs. $c=0.095$, $\alpha=0.2$ vs. $c=0.095$. 

| c     | ACC  | F1    | p%      | DP    | EO    |
|-------|-------|-------|--------|--------|-------|
| 0.015 | 0.832 | 0.833 | 89.941 | -0.016 | 0.109 |
| 0.035 | 0.835 | 0.836 | 86.465 | 0.038  | 0.149 |
| 0.055 | 0.842 | 0.842 | 83.077 | 0.050  | 0.188 |
| 0.075 | 0.843 | 0.843 | 81.346 | 0.065  | 0.207 |
| 0.095 | 0.842 | 0.840 | 78.548 | 0.094  | 0.240 |

Reviewer comment (Evaluation):
In Table 3, increasing alpha from 0.05 to 0.5 does not seem to affect the performance much (both in training and testing).

Author response:
Response to Weakness 2: The impact of increasing $\alpha$ has been discussed in our response to Question 2.

Response to Question 2: The initial glance may suggest only marginal fairness enhancement when increasing $\alpha$ from 0.05 to 0.5. However, when assessing the balance between accuracy loss and fairness improvement, we have a more detailed perspective. We calculate the average AC, DP, and EO reductions across the eight states. On average, AC decreases 0.0022, DP decreases 0.0150, and EO decreases 0.0185 for the training data. The ratios of fairness improvement to accuracy loss are more significant, with values of 0.0150/0.0022=6.78 for DP and 0.0185/0.0022=8.34 for EO. For the test data, AC decreases 0.0019, DP decreases 0.0156, and EO decreases 0.0132 on average, with ratios 0.0156/0.0019=8.28 for DP, 0.0132/0.0019=7.01 for EO.
 
In addition, we increase $\alpha$ to 0.9 as shown in the following table. We compare the results between $\alpha=0.05$ and $\alpha=0.9$. On average, AC decreases 0.0028, DP decreases 0.0207, and EO decreases 0.0224 for the training data. The ratios are 0.0207/0.0028=7.28 for DP and 0.0224/0.0028=7.86 for EO. For the test data, AC decreases 0.0027, DP decreases 0.0214, and EO decreases 0.0194 on average, with ratios 0.0214/0.0027=7.79 for DP, 0.0194/0.0027=7.07 for EO. These findings underscore the effective role of $\alpha$ in balancing the accuracy and fairness, where fairness gains are substantial comparing to the minor accuracy loss.
 
 |         |  |  $\alpha$ |   =     |     0.9   |            ||
|-------|-------------|-------------------|--------|--------|------------|------------|
|    |       ID       | (training)     |        | OOD     |   (test)  |        |
| State | ACC          | DP                | EO     | ACC     | DP         | EO     |
| CA                | 0.7239      | 0.1967            | 0.2259 | 0.7282 | 0.2020     | 0.2111 |
| TX                | 0.7330      | 0.2195            | 0.2605 | 0.7337 | 0.1874     | 0.1956 |
| DE                | 0.7290      | 0.2186            | 0.2882 | 0.7375 | 0.2595     | 0.2538 |
| NV                | 0.7152      | 0.1874            | 0.1948 | 0.7312 | 0.1611     | 0.1518 |
| KY                | 0.7451      | 0.1413            | 0.1464 | 0.7181 | 0.0759     | 0.1225 |
| FL                | 0.7376      | 0.2083            | 0.2402 | 0.7295 | 0.1546     | 0.1669 |
| MO                | 0.7455      | 0.1399            | 0.2025 | 0.7220 | 0.0917     | 0.1319 |
| NE                | 0.7297      | -0.0411           | 0.0317 | 0.7109 | 0.0354     | 0.0972 |

Reviewer comment (Experiments):
Only one baseline is compared in the experiments. I would suggest the authors to test against other baseline approaches optimizing for DP.

Author response:
Response to Weakness 4：Our experiments have been expanded by comparing with a method that directly optimizes DP (Weerts et al., 2023). Please refer to our global response for the new experimental results.

    Weerts, H.; Dud´ık, M.; Edgar, R.; Jalali, A.; Lutz, R.; Madaio, M. Fairlearn: Assessing and improving fairness of ai systems. Journal of Machine Learning Research, 24(257):1–8, 2023.

Reviewer comment (Experiments):
In Table 3, increasing alpha from 0.05 to 0.5 does not seem to affect the performance much (both in training and testing). Is it still the case when alpha is increased to e.g. 0.9?

Author response:
Response to Weakness 2: The impact of increasing $\alpha$ has been discussed in our response to Question 2.

Response to Question 2: The initial glance may suggest only marginal fairness enhancement when increasing $\alpha$ from 0.05 to 0.5. However, when assessing the balance between accuracy loss and fairness improvement, we have a more detailed perspective. We calculate the average AC, DP, and EO reductions across the eight states. On average, AC decreases 0.0022, DP decreases 0.0150, and EO decreases 0.0185 for the training data. The ratios of fairness improvement to accuracy loss are more significant, with values of 0.0150/0.0022=6.78 for DP and 0.0185/0.0022=8.34 for EO. For the test data, AC decreases 0.0019, DP decreases 0.0156, and EO decreases 0.0132 on average, with ratios 0.0156/0.0019=8.28 for DP, 0.0132/0.0019=7.01 for EO.
 
In addition, we increase $\alpha$ to 0.9 as shown in the following table. We compare the results between $\alpha=0.05$ and $\alpha=0.9$. On average, AC decreases 0.0028, DP decreases 0.0207, and EO decreases 0.0224 for the training data. The ratios are 0.0207/0.0028=7.28 for DP and 0.0224/0.0028=7.86 for EO. For the test data, AC decreases 0.0027, DP decreases 0.0214, and EO decreases 0.0194 on average, with ratios 0.0214/0.0027=7.79 for DP, 0.0194/0.0027=7.07 for EO. These findings underscore the effective role of $\alpha$ in balancing the accuracy and fairness, where fairness gains are substantial comparing to the minor accuracy loss.
 
 |         |  |  $\alpha$ |   =     |     0.9   |            ||
|-------|-------------|-------------------|--------|--------|------------|------------|
|    |       ID       | (training)     |        | OOD     |   (test)  |        |
| State | ACC          | DP                | EO     | ACC     | DP         | EO     |
| CA                | 0.7239      | 0.1967            | 0.2259 | 0.7282 | 0.2020     | 0.2111 |
| TX                | 0.7330      | 0.2195            | 0.2605 | 0.7337 | 0.1874     | 0.1956 |
| DE                | 0.7290      | 0.2186            | 0.2882 | 0.7375 | 0.2595     | 0.2538 |
| NV                | 0.7152      | 0.1874            | 0.1948 | 0.7312 | 0.1611     | 0.1518 |
| KY                | 0.7451      | 0.1413            | 0.1464 | 0.7181 | 0.0759     | 0.1225 |
| FL                | 0.7376      | 0.2083            | 0.2402 | 0.7295 | 0.1546     | 0.1669 |
| MO                | 0.7455      | 0.1399            | 0.2025 | 0.7220 | 0.0917     | 0.1319 |
| NE                | 0.7297      | -0.0411           | 0.0317 | 0.7109 | 0.0354     | 0.0972 |

Reviewer comment (Theory):
On the bottom of page 3, you state that a system of $m$ constraints and $d$ parameters with $m\geq d$ can only have the solution $\theta=0$. If I understand correctly $m$ is the number of sensitive attributes and $d$ is the number of features. Does it make sense that $m\geq d$?

Author response:
Response to Question 1: $m$ is the number of sensitive attributes and $d$ is the number of non-sensitive features. There are some cases where $m\geq d$.  To illustrate this situation, consider the Ricci 20 dataset (Le et al., 2022). This dataset encompasses five features "race", "position", "written", "oral", and the derived feature "combine" representing a composite score based on "written" and "oral", and one label "promoted". If we only consider the four original features and study the discrimination of race and position on the promotion, we have two sensitive features "race" and "position" and two non-sensitive features "written" and "oral". Then $m=d=2$.

    Le Quy, T., Roy, A., Iosifidis, V., Zhang, W., & Ntoutsi, E. (2022). A survey on datasets for fairness-aware machine learning. WIREs Data Mining and Knowledge Discovery, 12(3), e1452.

Reviewer comment (Theory):
In equation (8), is $\lambda$ a decision variable of the minimization problem?

Author response:
Response to Question 2: Yes. In equation (8), $\lambda$ is a decision variable of the minimization problem.

Reviewer comment (Theory):
Am I correct in my analysis that (8) is nonconvex?

Author response:
Response to Question 3: The optimization problem in equation (8) is nonconvex. We can obtain a local minimum when dealing with nonconvex problems. Our empirical experiments confirm that this approach yields satisfactory performance. Achieving a global minimum in nonconvex problems is indeed a demanding task, and it falls outside the scope of our focus in this paper.

Reviewer comment (Presentation):
I think Figure 1 is very hard to understand. Should we compare Figure 1 with Figure 2 and observe that the ID results are more representative of the OOD resutls than in Figure 2?

Author response:
Response to Question 4: The in-distribution (ID) results represent the training outcomes, and the out-of-distribution (OOD) results indicate the performance across 49 other states during the testing. Figure 2 (adapted from Figure 3 of Ding et al. (2021)) shows the accuracy of the training data (ID) is much higher than the accuracy among all the test data (OOD), which underscores a marked separation between the in-distribution and out-of-distribution results. In contrast, our results in Figure 1, depict OOD outcomes distributed around the ID results in a random pattern without a marked separation between ID and OOD. The key point in comparison between Figure 1 and Figure 2 is that Figure 2 demonstrates a marked separation between in-distribution (ID) and out-of-distribution (OOD) results, while Figure 1 not. We can say that the ID results are more representative of the OOD results in Figure 1 than in Figure 2.

Reviewer comment (Experiments):
While the comparison to Zafar et al. (2017) is very thorough, no other fair learning methods are considered in the empirical evaluation. In particular, comparing group fairness metrics such as DP and EO against baselines that directly optimize those notions could serve to more strongly justify the proposed approach using covariance-based fairness constraints.

Author response:
Response to Weakness 1：Our experiments have been expanded by comparing with a method that directly optimizes DP (Weerts et al., 2023). Please refer to our global response for the new experimental results.

    Weerts, H.; Dud´ık, M.; Edgar, R.; Jalali, A.; Lutz, R.; Madaio, M. Fairlearn: Assessing and improving fairness of ai systems. Journal of Machine Learning Research, 24(257):1–8, 2023.

Reviewer comment (Theory):
I was a bit confused about the discussion at the end about applying EL to other fairness criteria. If indeed extending to non-linear measures is straightforward, it is unclear why the current method only considers covariance as a fairness proxy.

Author response:
Response to Weakness 2：We can extend EL to other fairness criteria using similar techniques as presented in our paper. However, the extension involves carefully finding the influence functions and deriving the estimating equations of the specific fairness criteria, which serves as the foundation of the logarithm of the EL ratio converging to the $\chi^2$ distribution. Our current paper represents a preliminary exploration of applying EL to the fairness metrics. We leave the extension of EL to other fairness criteria as a future project.

Reviewer comment (Theory):
The motivation for the covariance-based fairness constraints was not very compelling. The authors suggest that determining the threshold for accuracy-fairness tradeoff becomes easier with the proposed approach. However, it is still unclear what the appropriate confidence region is for a given group fairness notion (whether a soft constraint to bound the violation or as a hard constraint).

Author response:
Response to Weakness 3：The tradeoff indicator $\alpha$ in our framework has a bounded range from 0 to 1, while covariance is unbounded. In this sense, it is easier to determine the threshold by our method. The choice of the confidence region depends the preference of the user's accuracy and fairness. Our constraint is formed by imposing $0$ included in the confidence region of the covariance at a user-determined significant level. The inherent mechanism of the confidence region allows the violation of the covariance as $0$, controlled by $\alpha$. In this sense, our constraint can be seen as a soft constraint.

Reviewer comment (Experiments):
How does the proposed method compare against Zafar et al. (2017) under distribution shifts?

Author response:
Response to Question 1: Our method performs better than the method in Zafar et al. (2017) under distribution shifts. We conducted experiments using datasets characterized by explicit distribution shifts (Ding et al., 2021), and the results are shown below. In each row of the following table, our method performs better in both accuracy and fairness than Zafar et al. (2017).

|       |  Zafar|  et al.| (2017) |        | EL    | based  |fairness|        |
|-------|-------|--------|--------|--------|-------|--------|--------|--------|
| State | $c$   | ACC    | DP     | EO     | $\alpha$ | ACC | DP     | EO     |
| CA    | 0.007 | 0.7284 | 0.2036 | 0.2125 | 0.50  | 0.7284 | 0.2034 | 0.2123 |
| TX    | 0.020 | 0.7348 | 0.2009 | 0.2110 | 0.01  | 0.7348 | 0.1983 | 0.2106 |
| TX    | 0.006 | 0.7340 | 0.1909 | 0.1993 | 0.50  | 0.7340 | 0.1905 | 0.1988 |
| TX    | 0.003 | 0.7338 | 0.1893 | 0.1978 | 0.70  | 0.7338 | 0.1892 | 0.1975 |
| NV    | 0.020 | 0.7319 | 0.1686 | 0.1598 | 0.50  | 0.7319 | 0.1657 | 0.1542 |
| NV    | 0.015 | 0.7317 | 0.1665 | 0.1595 | 0.50  | 0.7319 | 0.1657 | 0.1542 |
| NV    | 0.009 | 0.7314 | 0.1642 | 0.1549 | 0.70  | 0.7314 | 0.1634 | 0.1535 |
| FL    | 0.008 | 0.7304 | 0.1605 | 0.1774 | 0.30  | 0.7305 | 0.1605 | 0.1771 |

The following table shows the results of Zafar et al. (2017) whe $c=0.01$ under distribution shifts. The fairness metrics differ significantly between the training data and test data, which indicates the method is not robust to distribution shifts.

|       |   ID        |results  |(training)|OOD    |results |(test)  |
|-------|-------------|---------|---------|--------|--------|--------|
| State | ACC         | DP      | EO      | ACC    | DP     | EO     |
| CA    | 0.7247      | -0.2286 | -0.1996 | 0.7286 | 0.2053 | 0.2130 |
| TX    | 0.7341      | -0.2640 | -0.2193 | 0.7343 | 0.1936 | 0.2038 |
| DE    | 0.7288      | -0.2882 | -0.2191 | 0.7378 | 0.2620 | 0.2620 |
| NV    | 0.7161      | -0.1969 | -0.1893 | 0.7314 | 0.1645 | 0.1564 |
| KY    | 0.7458      | -0.1495 | -0.1443 | 0.7197 | 0.0823 | 0.1273 |
| FL    | 0.7396      | -0.2451 | -0.2123 | 0.7307 | 0.1619 | 0.1795 |
| MO    | 0.7459      | -0.2043 | -0.1434 | 0.7239 | 0.1084 | 0.1427 |
| NE    | 0.7306      | -0.0336 | 0.0182  | 0.7123 | 0.0426 | 0.1056 |

    Ding, F., Hardt, M., Miller, J., & Schmidt, L. (2021). Retiring adult: New datasets for fair machine learning. In Advances in Neural Information Processing Systems. Curran Associates, Inc.

Reviewer comment (Novelty):
Is the discussion about EL for general fairness measures new or known results? If it is the former, there needs to be more details and proofs.

Author response:
Response to Question 2: The extension of EL for general fairness measures can be considered as new results, as explained in our response to Weakness 2. We leave the extension as a future project.