Paper: Fair Adversarial Training: on the Adversarial Attack and Defense of Fairness (ICLR-2024)

Reviewer comment (Experiments):
The experimental setup is not clear.

Author response:
Sorry for the confusion. Detailed experimental setup can be found in Section A of the appendix.

Reviewer comment (Reproducibility):
It would be better if the authors provide the code.

Author response:
We 'll provide the source code in final version.

Reviewer comment (Evaluation):
The explanation of Figure 1 is unclear. The paper claims that the Balck TPR and White TNR, with 100% accuracy, will cause destructive and social injustice against the disadvantaged group. However, it may be correct. Only discussing Black TPR and White TNR is not sufficient and not convincing. The problem for causing injustice is the model has a high Black TPR while having a low Black TNR. (or Low White TPR and High White TNR).

Author response:
Sorry for the confusion. As shown in Fig. 1, under the fairness attack, the black TPR increases to 1, while the black TNR decreases to 0. This means all the black people are predicted as "high risk". On the contrary, the white TPR decreases to 0, while the white TNR increases to 1. This means all the white people are predicted as "low risk". Consequently, such disparity leads to crucial injustice problems.

Reviewer comment (Writing):
The presentation could be clearer (sentence after Equation 5, “adversarial sample ny xi”, what does ny represent?).

Author response:
"ny" should be "by" in the content. We are sorry for the typos and we 'll fix them in final version.

Reviewer comment (Novelty):
The purpose of a fairness attack remains unclear. It is not explained what specific objective a fairness attacker would have in attacking the fairness of an ML model. Consequently, the emphasis on increasing model fairness against such attackers is questionable. Addressing fairness issues stemming from data or algorithmic biases appears to be more important.

Author response:
We agree with the reviewer that static fairness, namely fairness issues stemming from data or algorithmic biases is an important problem, and therefore our fair adversarial framework takes into account both static fairness and fairness robustness, as shown in Eq. 5. Similar to that of the accuracy attack, in terms of pracitcal applicability, it is peculiar to intentionally depreciate a model's accuracy or fairness, unless out of some egregious concern, as dicussed in Fig.1 of the main paper. Instead, the fairness attack can be considered as means of measuring of generalizability of model in terms of fairness on some unknown data under the worst-case scenario. As discussed in Section 6.2 of the main paper, under a successful accuracy attack, the accuracy of the model decreases to 0, while the DI or EOd does not show a monotonic increase as the perturbation level of the accuracy attack increases. This shows that the accuracy attack alone does not provide adequate information in terms of the generalizability regarding the fairness of the model under dynamic secnarios. Therefore, as with the accuracy attack, we propose the fairness attack to quantify the generalizability of a model in terms of fairness regarding the worst-case scenario.

Thank you for the follow-up. Regarding the generability perspective of adversarial training, we may consider the objective of a typical adversarial training:
$$
\min \_\theta \mathbb{E}\_{(x, y) \sim \mathcal{D}}\left[\max \_{\delta \in B(x, \varepsilon)} \mathcal{L}\_{c e}(\theta, x+\delta, y)\right],
$$
where $\mathcal{D}$ is the training distribution and $B(x, \varepsilon):=\\{x+\delta \in \mathbb{R}^m \mid \|\delta\|_p \leq \varepsilon\\}$. Such mini-max formulation corresponds directly to the worst-case performance of the classifier under varying data distributions, i.e., the generability of the classifier under worst-case scenarios. Consequently, the adversarial attacks (namely the accuracy attacks in this case) can be thought of as means to obtain the worst-case samples [1,2]. Using the accuracy attack corresponds to the worst-case in generability in accuracy, while the fairness attack considers the worst-case in generability in fairness measures.

Moreover, regarding the practical threats, fairness attacks can be carried out for malicious intentions or anarchistic reasons against the disadvantaged groups [3].

[1] Bai, Tao, et al. "Recent advances in adversarial training for adversarial robustness." arXiv preprint arXiv:2102.01356 (2021).

[2] Wang, Yisen, et al. "On the convergence and robustness of adversarial training." arXiv preprint arXiv:2112.08304 (2021).

[3] Chhabra, Anshuman, et al. "Robust fair clustering: A novel fairness attack and defense framework." arXiv preprint arXiv:2210.01953 (2022).

Reviewer comment (Theory):
The concept of an adversarial attack concerning a group fairness measure seem unconventional. While existing adversarial attacks utilize point-wise objectives, the proposed fairness attack employs a group-wise objective such as DI or EOd. When this proposed attack is applied at a single point, it can influence all subsequent attacks from other points. This leads to a notational issue in defining fairness adversarial samples: $L_{DI}$ and $L(x, a, y)$ are used interchangeably. It might be more suitable to carry out the attack using individual fairness measures.

Author response:
We would like to clarify that both the fairness attack and the accuracy attack employ collective adversarial objectives, rather than point-wise objectives. Regarding the poisoning attacks, the adversarial objectives take into account the statistical properties of the classifier at a collective level, such as accuracy or DI/EOd subject to the specified targets, rather than the point-wise objectives as in the evasion attacks.

Moreover, similar to that of the accuracy attack, while the adversarial objective is targeted against the statistical properties of the model, i.e., DI and EOd of the model, the fairness attack on different data points are independent. This is further discussed in Eq. 2 and Eq. 3 of the main paper, which is a instance-wise reformulation of Eq. 1. While the estimation of DI and EOd relies on group information, regarding gradient-based fairness attack, the adversarial objective $L_{\text{DI}}$ only takes into account the soft predicton of $x_i$, and information from all the other points can be treated as constants, as they do not affect the update of the fairness adversarial samples by $x_i$. **Therefore, the fairness attack applied to one single instance does not influence the subsequential attacks for the others, and the adversarial attack is performed based on purely the gradient information of the trained model w.r.t. the very instance.**

Reviewer comment (Writing):
The presentation of fairness robustness lacks clarity in its definition. Only expressions - related to fairness robustness, indicate improved fairness robustness - are presented. The absence of a clear definition weakens the authors' assertions.

Author response:
The definition of fairness robustness is presented in the introduction of main paper: "Here we similarly define fairness robustness as a model’s ability to resist adversarial perturbation by an fairness attack and remain same predictions on clean and fairness adversarial samples.", i.e., $\mathbb{P}\_{x \sim \mathcal{D}\_{\mathcal{X}}}[\exists \|\epsilon^{\text{Fair}}\| \le \epsilon\_0$ s.t. $f(x+\epsilon^{\text{Fair}}) \neq f(x)]$ where $\epsilon^{\text{Fair}}$ is specified by the fairness attack. This reflects the consistency of the classifier to maintain the same predictions both with and without the adversarial perturbation under the fairness attack. Similarly, in the introduction, we define accuracy robustness as "a model's ability to resist adversarial perturbation by an accuracy attack and remain same predictions on clean and accuracy adversarial samples.", i.e., $\mathbb{P}\_{x \sim \mathcal{D}\_{\mathcal{X}}}[\exists \|\epsilon^{\text{Acc}}\| \le \epsilon\_0$ s.t. $f(x+\epsilon^{\text{Acc}}) \neq f(x)]$ where $\epsilon^{\text{Acc}}$ is specified by the accuracy attack. This reflects the consistency of the classifier to maintain the same predictions both with and without the adversarial perturbation under the accuracy attack.

Reviewer comment (Theory):
The main assertion here is that the robustness secured by accuracy adversarial samples can guarantee fairness robustness, and vice versa. However, there appears to be a significant issue. When a fairness attack is executed with an objective like DI, its evaluation relies on cross-entropy. Then, in terms of accuracy robustness, fairness attacks essentially amount to weaker attacks on cross-entropy compared to PGD attacks with a cross-entropy objective. In light of this, the proposed claims about alignment of robustnesses become somewhat trivial, potentially diminishing the authors' contributions.

Author response:
Regarding the effect of adversarial attack, both the magnitude and the direction of the attack matter. As discussed in Section 4 of the main paper, The fairness attack and the accuracy attack acts upon different directions regarding different samples. Under the same step size and the same perturbation level up to the same iterations, the fairness attack and the accuracy attack act equivalently upon samples in the 'Same Direction' category. For samples in the 'Opposite Direction' category, the fairness attack and the accuracy attack act towards the opposite directions. We cannot assert that "weaker attacks on cross-entropy" leads to trivial alignment between fairness robustness and accuracy robustness, owing to the misalignment in the direction of the two attacks. Therefore, we show in Theorem 1 in the revised paper that the fairness robustness, measured by the change in cross-entropy loss before and after the fairness attack, regarding $x_{\text{FN},1}$ is upper-bounded by the accuracy robustness of $x_{\text{FN},0}$, i.e., the fairness robustness guarantee by accuracy robustness. Moreover, our discussion in Theorem 3 in the revised paper also points to the non-trivialness of the problem: if the fairness robustness amounts to weaker robustness than the accuracy robustness, we cannot have the accuracy robustness guarantee by the fairness robustness.