Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The paper studies the fairness attack and defense problem. The authors propose a black-box attack against fair clustering algorithms that works by perturbing a small percentage of samples’ protected group memberships. They also proposed a defense algorithm, named Consensus Fair Clustering (CFC), that utilizes consensus clustering along with fairness constraints to output robust and fair clusters. Empirically, existing fair clustering algorithms are highly susceptible to adversarial influence, while the proposed CFC algorithm is highly effective and robust as it resists the proposed fairness attack well.
strength_and_weaknesses: Strength: 
- The proposed problem of robust fair clustering is interesting and novel.
- The proposed black-box attack is natural in robust clustering literature.
- The empirical results that existing fair clustering algorithms are highly susceptible to adversarial influence are convincing.

Weaknesses:
- The proposed approach CFC does not have any provable guarantees, both in fairness and in accuracy. I expect to see at least some performance analysis for CFC.
clarity,_quality,_novelty_and_reproducibility: Clarity: The writing is good and clear. The empirical results can verify the proposed conclusions. A comment is that the authors should provide an ethnic discussion since the topic is about fairness.
Quality: The proposed black-box attack is interesting, but lacks some theoretical analysis. For instance, what is the worst influence of such a black-box attack on some known fair clustering algorithms? 
Novelty: The considered problem is novel.
summary_of_the_review: The paper studies the fairness attack and defense problem and proposes a novel black-box attack. The empirical results that existing fair clustering algorithms are highly susceptible to adversarial influence are convincing. The major concern is the lack of theoretical analysis on both the influence of black-box attacks and the proposed CFC approach.

Overall, I tend to accept the paper. I think the paper can be further improved by providing some theoretical analysis.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: The paper studies adversarial attacks on fair clustering. The authors demonstrate simple adversarial attacks on existing fair clustering approaches and propose an algorithm that is robust to the attack.
strength_and_weaknesses: ------------------------
Strengths:
------------------------
-- The paper is mainly well written (except section 3.1) and easy to follow.

-- The idea of adversarial attacks on fair clustering is natural and worth exploring. 

------------------------
Weaknesses:
------------------------
-- The paper would be strengthened if the authors provide a clear motivating example.

-- The size of $G_A$ in all the experiments is rather large. In all the definitions, it is assumed that this proportion is small but all of the experiments focus on 15% which is rather large. How do the results change for proportions much smaller than 15%?

------------------------
Minor Comments:
------------------------
-- What is the axis in Figure 2? It would be nice to have labels for graphs.

-- Can you elaborate on why the fairness violation can decrease after accounting for attacks? This intuitively does not make sense as the approach which only optimizes for the fairness violation should achieve a smaller loss.

clarity,_quality,_novelty_and_reproducibility: The paper is mainly clear (except for section 3.1). The ideas are somewhat novel as similar attacks have been studied for the clustering objective (with no fairness considerations). I do not see any link to the code but most of the experimental details are described somewhat clearly. 
summary_of_the_review: Overall, I think this is interesting work, though I am not an expert in the area.

-------------------------
Post Rebuttal:
-------------------------
I thank the authors for their response. After reading all the reviews and responses, I have increased my score from borderline accept to accept.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper tackles **unsupervised fair clustering**. The authors first propose **a black-box adversarial attack** on state-of-the-art fair clustering algorithms, effective on a toy dataset they propose as well as real datasets (MNIST, USPS, Office-31). In their threat model, an adversary can modify *the protected attribute* of a subset of the dataset. Its goal is to minimize a fairness utility function while trying not to degrade the performances. Success of the attack is evaluated on the remaining untouched datapoints.
In a second time, the authors propose **a defense against that attack**, an unsupervised fair clustering algorithm which is robust to the attack.
strength_and_weaknesses: ### Strengths

- Being *black-box* and exploring a wide range of ratio of poisoned data, the threat model seems highly relevant;
- Brings neural network in the field of fair machine learning, which seems to still be understudied (as showed by the state-of-the-art algorithms used to compare to in the paper);
- The paper is well written and easy to read.

### Weaknesses

- Some comparisons would have benefited being more thorough:
    - Figure 2 and 4 would have benefited showing results on CFC;
    - Table 2 and 4 would have benefited comparing with random attacks;
    - Figure 6 experiments would have benefited from a confidence interval;
    - Comparing different fair algorithms that could be used to generate the $J$ parameter in equation $(4)$.
- Even though an in-depth exploration of CFC is proposed in appendix, an ablation study would have been relevant, especially on the different losses being used.
- Studying the robustness of CFC against other known attacks on clustering (relaxing some of the problem constraints if necessary) would have been highly appreciated.
- Is the ratio of poisoned samples reasonable? 15% seems like a lot, it would be interesting to see how CFC compare with a low number of adversarial samples (1%?)
clarity,_quality,_novelty_and_reproducibility: ### Clarity

The paper is well written and quite clear, except for a few details:

- Page 8, right before equation $(3)$, the “cluster $c_k$” are mentioned but I still did not understand where they came from.
- Page 8, between equation $(3)$ and $(4)$, the meaning of $P(g)$ seems unclear to me. Why would $P$ take $g$ as an argument when the $P$ has allegedly been constructed with the probabilities $p_k^x$ which do not seem to depend on $g$.
- Page 13, in Balance definition, the symbol $r_k$ seem confusing. Balance has been generalized to more than two groups in Suman K. Bera’s *Fair Algorithms for Clustering* and a similar notation explicitly showing the group considered such as $r_k^g$ would have been preferred.

### Quality

One of the claim in the paper seems confusing. At page 7, the authors write: “*Since attacked samples are a tiny portion of the whole training data, the probability of these being selected into the subset is also small, which decreases their negative impact.*” But since all of the samples are used in a basic partition (since $\cup_{i=1}^r X_i = X$), the reason for the robustness of CFC must be attributed to another mechanism. This question is later investigated in appendix G but further work would be required (comparing several values of $r$?)

### Novelty

This work is undoubtedly novel as the threat model and downstream task do not seem to appear in other works.

### Reproducibility

Reproducilbility cannot be evaluated as such without the source code but the given implementation details are highly appreciated. Solutions exist to ensure confidentiality or uploaded in supplementary material.
summary_of_the_review: The novelty and relevance of this work make it interesting for the scientific community in both Robust Machine Learning and Fairness-related Machine Learning research. Although I pointed a few (alleged) weaknesses, the paper should have tackled, they nonetheless exhibited a relevant attack and experimentally demonstrated a defense to that attack. Further work on this topic will be of great value for the community.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The paper studies robust fair clustering and motivates the issue of robustness under attack. The authors conduct numerical experiments to show that state-of-the-art models are highly susceptible to the attack they design. The authors then propose a Consensus Fair Clustering and numerically show its robustness.
strength_and_weaknesses: Strength: 
1. The issue of robustness under attack is well-motivated. I believe this could be an important property for any robust fair clustering. 
2. The designed attack is quite novel.

My major concern is the following:
The proposed Consensus Fair Clustering appears to respond well to the designed attack. Is there any guarantee that it remains robust under ANY attack that the adversary would choose?
clarity,_quality,_novelty_and_reproducibility: The issue of robustness under attack is well-motivated. The designed attack is quite novel.
summary_of_the_review: The issue of robustness under attack is well-motivated, and I believe this could be an important property for any robust fair clustering. 
2. The designed attack is quite novel. The designed attack is quite novel.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: The authors study the classic fair classic problem. They first propose a novel black-box attack and show that a very simple strategy can destroy the guarantees of previous algorithm. Then they present a new technique and show experimentally that it provides better guarantees that the previous result.

The paper is well-written and it study a classic problem from a novel perspective so it would be a nice addition to the ICLR program.
justification_for_why_not_higher_score: The results is nice and interesting but not strong enough to be a spotlight
justification_for_why_not_lower_score: The contribution is novel and study a well-know problem from a different and interesting perspective.
Decision: Accept: poster