Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The authors first describe an attack to poison fair classifiers, and then introduce a defense (filtering based) to mitigate poisoning attacks on fair classifiers.
strength_and_weaknesses: ## Strengths:
1. The paper is well rounded and thorough - the authors describe the setting and the problem well, introduce an attack, and then describe a defense to this attack.
2. The results, at least in the setting described in the paper (linear classifier, adult census dataset) improve upon existing techniques. 

## Weaknesses:
1. Significance - I'm not convinced this work is significant enough to warrant publication at a top tier conference. While fair classification is of great interest, and data poisoning attacks at well, a defense against data poisoning for fair classification networks seems a bit too niche/lacking general interest.

2. The attack seems like a straightforward extension of the technique in [1] with a straightforward addition of penalty to encourage fairness in the inner optimization problem for poisoning. Granted, the proposed defense does seem novel.

3. The setting for this paper seems weak. The poisoning attacks allow for techniques like label flips, and it seems like a simple linear network is used for the victim model (avoiding any questions of white-box vs. black-box attacks). This is in contrast to recent, more sophisticated poisoning attacks that are in "clean-label" setting, and/or against deep networks.


### Random tidbits:
- I believe the equation in between eqns 2, 3 should have an $\leq$, and not $=$ in the last part. 


[1] Muñoz-González, Luis, et al. "Towards poisoning of deep learning algorithms with back-gradient optimization." Proceedings of the 10th ACM workshop on artificial intelligence and security. 2017.

clarity,_quality,_novelty_and_reproducibility: The work was clear enough to read, and the defense is novel as far as I know.
summary_of_the_review: While I think the paper is well presented, and thorough, the combination of the attack scenario being somewhat niche, and the attack constraints being rudimentary, I don't think the work is of sufficient interest at this point to warrant publication at a top conference. 

I am, however, willing to change my score in response to new information/reasoning from the authors.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper studies data poisoning attacks against fair classification algorithms, and proposes a new poisoning attack, which is a variant of the existing min-max attack. In addition, a robust fair classification defense is proposed, which is an extension of the Sever defense into fair classification problems. Empirical results show that the proposed attack is more effective than the existing min-max attack against different baseline defenses and the proposed defense is also robust against the newly proposed attack.  
strength_and_weaknesses: Strength:
1. vulnerability of fair classification under data poisoning is interesting. 
2. the proposed attack and defense performs better than the existing attacks and defenses.

Weakness:
1. the major concern is in the significance of this work: the F-attack is a variant of the existing min-max attack, where the major difference is how the model weight is updated after the poisoning point is selected. The proposed defense is also a variant of the Sever defense, with finer analysis of the training data at particular subgroups. Therefore, the technical novelty of the paper is limited.
2. the considered attacks are quite weak and may not even be a real threat. The fraction of poisoning points is >10%, which is considered quite a high ratio in practical applications. However, even with such a high poisoning ratio, the attack effectiveness is limited. Taking the poisoned model's test accuracy as an example, we expect a strong poisoning attack to induce much higher test error in comparison to the poisoning ratio $\epsilon$, while results in this paper induce test error smaller than $\epsilon$. Therefore, one may conclude that current fair classifications are not under severe threat from poisoning attacks. 
3. As for the evaluation, one relevant baseline is missing (Jo et al., 2022). The min-max attack may also be improved by leveraging target models, which is done in Koh et al., (2021) and also demonstrated similarly in Suya et al., (2021).  

Jo et al., "Breaking Fair Binary Classification with Optimal Flipping Attacks", ISIT 2022. 
Suya et al., "Model-Targeted Poisoning Attacks with Provable Convergence", ICML 2021. 
clarity,_quality,_novelty_and_reproducibility: The paper is mostly well-written and is easy to follow. However, some of the statements in the paper are inaccurate. For example, upper bound (ii) may not be tight even when $\epsilon$ is small, as it is related to the property of the training loss and also the search space of the poisoning points. In extreme cases, the poisoned loss can be much higher than the average clean loss and hence the upper bound can be very loose. In Eq. (3), the authors should provide more details on why we choose to solve the "min-max" form when the loss function is convex. Is it of similar reason as in Steinhardt et al., (2017)? Specifically, the (easier to solve) "min-max" form is a universal upper bound to the original "max-min" form and also has an asymptotic convergence when the loss function is convex. As for the originality, the core techniques for the attacks and defenses are mostly from Steinhardt et al., (2017) and  Diakonikolas et al., (2019). Therefore, I am worried that originality is somewhat limited.   
summary_of_the_review: The idea of studying fair classification under poisoning attacks is interesting. However, the core techniques in this paper are mostly based on existing poisoning attacks and defenses for traditional classification and hence, have limited technical novelty. In addition, the proposed attacks also have limited effectiveness empirically, which further weakens the results. Considering all these factors, a weak reject is recommended.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper studied data poisoning attacks against fair classification algorithms. The objective of poisoning a fair learner is approximated by a minimax optimization problem. When the loss function is convex, one can switch the min-max operator and use gradient descent to construct the optimal poisoning dataset. On the basic fair learner and a few robust variations such as k-NN based defense and Sever-based defense algorithms, the authors demonstrate that their poisoning algorithm can reduce the accuracy and the fairness metric of the victim learners. Motivated by that, the paper designed a defense mechanism called RFC, which employs similar ideas as prior works, but adapted to the fair classification setting. The paper analyzed the relationship between the poisoning scores of clean and poisoned samples. Extensive experiments on abundant datasets show that the proposed defense can enhance the resilience of fair learner against poisoning attacks.
strength_and_weaknesses: Strength:

(1). The paper performed extensive experiments and detailed analysis on the experimental results, which clearly conveyed two critical points: i) fair classification learners, although equipped with existing defense strategies such as k-NN and Sever, are not robust enough against data poisoning attacks. ii) the RFC defense mechanism proposed in this paper indeed helps mitigate the effect of data poisoning attacks on fair learners and outperforms existing defense methods.

(2). The paper is easy to read. I can quickly understand the topic and the key research points after reading the problem statements. The presentation and paper structure is also nice and clear.

Weaknesses:

(1). The paper is technically weak from a scientific point of view. While the paper performed extensive experiments and also designed interesting attack and defense algorithms, I find it hard to justify that their approaches contain very novel techniques compared to prior works. Their attack algorithm is pretty straightforward and looks like adapted from existing research outcomes. The minimax formulation also applies to traditional classification victim learners. The only difference here is that in this paper, the min operator is restricted to models that satisfy the fairness constraint. Therefore, I don't see why the attack algorithm is technically novel. Similarly, as the authors pointed out, the defense here also borrows some ideas from prior works. Overall, I am a bit concerned about not having enough novelty and technical contribution for this paper.

(2). The theoretical results do not appear appealing to me because it does not state directly to the point that how much improvement in the accuracy or fairness is guaranteed by the defense. Right now, the results are somewhat weak because it's about the poisoning score. In contrast, the more interesting theoretical problem is how much accuracy is preserved when using the RFC defense compared to without using RFC. I am wondering if the theoretical part of this paper can be improved in that way.

(3). In the objective approximation equation (i.e., the equation between (2) and (3)), the last equality does not make sense to me. Instead of an equality, I believe that should be an inequality. I am wondering if that's a typo or did I miss anything?

(4). The scope of this paper is limited to classification only, Furthermore, it's restricted to convex loss functions. As a result, the results in this paper are not directly applicable to DNN-based classification learners. I believe the paper can easily generalize their results, e.g., by designing different poisoning algorithms for DNN-based learners. The minimax formulation no longer works but from a practical perspective, I don't really think the minimax formulation is super critical to this paper. There are a variety of poisoning attacks against DNN, and these attacks can be easily adapted to the fair classification setting. Please correct me if I am wrong.
clarity,_quality,_novelty_and_reproducibility: I am a bit concerned about the novelty and technical contribution of the work.
summary_of_the_review: I worked in related areas.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: The authors studying poisoning attacks which are effective specifically against fair classification methods. The hope would be that there would be some special insights that arise that arise as a result of combining poisoning and fairness, but it seems relatively limited. Specifically, it seems that a straightforward modification of a prior method of Munoz-Gonzalez et al., to handle fairness constraints. Study is also restricted to relatively simple settings. On the positive side, the paper was relatively thorough. On the whole though, the paper wasn't considered strong enough to warrant acceptance. 
justification_for_why_not_higher_score: Little special insight obtained by studying the intersection of poisoning and fairness.
Decision: Reject