Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The paper focuses on the problem of accessing sensitive group attributes in the validation set for hyperparameter tuning. To solve this issue, the paper proposes Antigone, a hyper-parameter tuning framework for fairness techniques that do not access sensitive group attributes in training data. The proposed framework is inspired by a prior work (Lamy et al., 2019), which has been proposed for fair training under noisy sensitive attributes. This algorithm is evaluated on two datasets, including CelebA and Waterbirds, with some baselines.
strength_and_weaknesses: [Strength]

S1: The paper tackles the practical problem of accessing sensitive attributes in the validation set. 

S2: The paper utilizes an idea from fair training under noisy sensitive attributes, and such inspiration gives an interesting approach. 


[Weakness]

W1: The significance of the proposed algorithm seems a bit limited, as it only targets fairness techniques that do not access sensitive attributes in training data. Since many more algorithms have been proposed for fair training with sensitive attributes, it would be helpful if the paper can clarify 1) the significance of this work and 2) how the proposed algorithm can be extended in other scenarios.

W2: Several design choices in the proposed framework Antigone are not fully justified. For example, why using the previous idea of handling noisy group attributes is the most appropriate way to solve the target problem? Why the current approach is better than other approaches like training a weak-labeler on groups?

W3: More importantly, the experimental results are insufficient. 
- The baselines are not enough. One of the key advantages the paper argues is that Antigone can be used for any algorithm of not accessing sensitive attributes on training data. However, the paper only shows its effectiveness on JTT. Thus, it is hard to believe that Antigone will effective in other algorithms like LfF and DRO.
- Also, all experimental results are reported without error range, which makes the observations less reliable. For example, in several rows of Table 1, the improvements in Antigone compared to GEORGE are a bit marginal, so it is questionable how it changes after multiple runs. It would be much better if the paper shows the results with error ranges to clarify the performance gain.

Minor Typo: First paragraph in Section 2.1) target labels (X) => target labels (Y)

clarity,_quality,_novelty_and_reproducibility: The paper is clearly written, but several clarifications are needed as discussed in the previous section.
summary_of_the_review: Although the paper tries to solve a practical problem, there seems some room for improvement in 1) clarification on the importance of the work, 2) justification of the design choices, and 3) enhancement of the experimental results. Overall, I think that this paper is on the borderline.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper propose a fair model training method (called Antigone) without sensitive attribute access. The idea is to train a biased classifier where all its correctly-labeled examples are considered to be the majority sensitive group, and all its incorrectly-labeled examples form the minority group. The measure of success for such a biased classifier is the Euclidean distance between the means of the two groups (EDM) where a classifier with a larger EDM is best. As a justification, an mutually contaminated noise model is used where maximizing EDM results is shown to minimize DP and EO gaps. Experiments show that Antigone outperforms the competing method GEORGE.
strength_and_weaknesses: Strengths
- Solves the realistic and challenging problem of fair training without sensitive attributes.
- The EDM maximization strategy is supported by theoretical results.
- Antigone outperforms GEORGE on real datsets.

Weaknesses
- The key assumption that there can be a bias classifier where all its correct (incorrect) predictions form the majority (minority) group seems a bit unrealistic. This means that minority group examples are extremely difficult to classify while majority group examples are much easier. If we consider the motivating examples mentioned in the Introduction like algorithmic hiring, is hiring female and male applicants that different in difficulty? I would argue that making predictions on the two sensitive groups is quite similar in difficulty, so it is not clear if the paper's assumption holds. Perhaps the assumption holds for applications where there is severe class imbalance, but in fairness problems, the gap can be more subtle than obvious. The paper does not attempt to demonstrate the key assumption empirically either. There are two real datasets CelebA and Waterbirds, and it is not clear how imbalanced their sensitive groups are. 
- While the theoretical justification of EDM using the MC noise model is useful, it is not clear if the MC noise model itself really holds on real datasets. One way to verify is to add a synthetic dataset that clearly follows the MC noise model first, and then show that the Antigone results are similar on real dataset. It is often the case that real datasets have all sorts of noise that makes it difficult to understand why the proposed method is working well. 
- It seems like EDM is only justified because the MC noise model is quite simplistic. Intuitively, is maximizing the Euclidean distance really all we need to do to tune the bias model? Should we not care about say the variance of the two distributions? If not, why? 
- Can Antigone be extended to support more than two sensitive groups? The MC noise model seems to be limited to the binary case.
- Only experimenting on CelebA and Waterbirds seems a bit thin. What about other fairness benchmarks like AdultCensus, COMPAS, and the recent ACSIncome? More importantly, it would be informative to know how each dataset is useful in showing the advantages of Antigone. Also, it would be nice to have datasets with varying imbalances where if the imbalance is small, Antigone actually does not perform well, which is expected.

clarity,_quality,_novelty_and_reproducibility: - The reasoning of why EDM needs to be maximized is still not clear.
- The experiments are rather thin and do not fully support the claims in the paper.
summary_of_the_review: This paper solves the challenging and important problem of fair training without sensitive attributes. There is theoretical support on why the biased classifier should maximizd EDM. However, the key assumption that training the desired biased classifier is possible is a bit questionable. In addition, the MC noise model seems quite simplistic and should be verified using the datasets. Finally, the experiments are a bit thin using only two real datasets and not fully analyzing how Antigone behaves.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The paper presents a method for performing hyperparameter tuning without access to a sensitive attribute. Instead of obtaining ground-truth sensitive attribute labels on the validation set, the proposed method uses "pseudo" attribute labels, which are simply whether an observation is correctly predicted or not. They use these labels for performing hyperparameter tuning on JTT and George for two datasets.

strength_and_weaknesses: ## Major comments

* It seems there is a clear baseline that is not evaluated here: simply use some of the (fully-labeled) training data as a held-out validation set. In that case, ground-truth attribute labels would be available. It is not clear why the paper doesn't compare to this?

* It would be helpful if the authors formally state the problem they are trying to solve (P3 simply says "we seek to train models that optimize fairness under a constraint on average accuracy").

* Similarly, it seems like the works on multicalibration/multiaccuracy are trying to solve a similar problem ("fair" classification without access to any sensitive labels); it seems particularly the fair boosting approach in (Kim et al, "Multiaccuracy: Black-box post-processing for fairness in classification") as a postprocessing method with a strong backbone model is relevant to compare to as well.

* The notation in the paper was quite confusing for me to read. There are many different sub- and superscripts, and D and X both denote (different) datasets. It also seems like \mathcal{A} is used to refer to the set of potential sensive labels (2.1) and a function class (in definition of \theta^*)? The intution of using the max-EDM model is also not clear to me; why do the authors state "we would like our noisy labeler to be biased"? It seems that *we would like to obtain accuracy pseudo-labels for the sensitive attribute* (see next comment), which is not the same thing. It is also not even clear what "bias" means in this context.

* It seems the key empirical/theoretical result for the entire paper is to show that the pseudo- attribute labels are good estimates of the true attribute labels on the validation set. The only evidence we have of this is Table 1, which strangely gives the precision (not the accuracy?) of these attribute labels. It would be useful to (1) provide the *accuracy* of these labels, (2) perhaps give some examples of data points in each (pseudo) class to see whether these match our expectation since the attribute in each dataset are easily interpretable from images, (3) perform some simulated analyses to demonstrate that under ideal conditions the pseudo-labels correctly recover the true attribute labels.

## Minor comments

* Please provide (a) the size of the data subsets in Table 1 (how many "blond men" in the validation set of CelebA?) and (b) Clopper-Pearson confidence intervals on the accuracies shown in the tables, or some other measure of statistical uncertainty of these point estimates. As is, the authors use some very fuzzy language ("JTT+Antigone is very close" and "[b]oth substantially improve upon").

* It is not clear why the method is called Antigone. This name is also not mentioned in the abstract.

* Table 3 is way too small.


## Typos etc.

There are many typos in the paper. I list a few below.

P2 "incorrectly classifier examples"

After Proposition 1: "From ... that" -> "From ... we can conclude that"

Lemma 1 "inputs data" -> input data

P7 "tune it's hyperparameters" -> tune its hyperparameters

* 3.2 why is GEORGE "a competing approach"? Competing against what? Please clarify.
clarity,_quality,_novelty_and_reproducibility: I think the clarity could be considerably improved by cleaning up the notation (see comments above). The overall quality is also difficult to gauge (see comments above regarding baselines, and giving better estimates of the statistical uncertainty). The proposed method is novel, to my knowledge.
summary_of_the_review: Overall, the paper presents an interesting, if somewhat ad hoc seeming, approach. However, I think the exposition of the method is fairly confusing, and more importantly, I have some concerns about the empirical evaluation -- they are missing any estimates of statistical uncertainty/variability, and the empirical results are mixed and missing some clear baselines (see other comments).
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: In this paper, the authors propose an algorithm for mitigating unintended bias without requiring access to the sensitive attribute. The intuition is similar to some recent works, which assume that the errors of a model provide a noisy proxy for the sensitive attribute (e.g. minorities would have a disproportionately larger error). There have been some recent works along this direction but it has been observed that their performance was sensitive to the choice of hyper-parameters. In this paper, the authors introduce a method for hyper-parameter selection without requiring access to the sensitive attribute in both training and validation data and demonstrate experimentally (using CelebA and WaterBirds) that it performs better than previous methods.
strength_and_weaknesses: **Strengths**

Overall, the paper is clearly written and the proposed algorithm Antigone is well-motivated. The experimental results show a significant improvement compared to previous methods. In addition, Antigone can be applied to any prior method for hyper-parameter selection, such as JTT (Liu et al., 2021). I think the method is neat and the motivation behind the EDM criterion using the mutually contaminated (MC) noise model is elegant.


**Weaknesses**

The primary weakness is that the method only applies to binary sensitive attributes. Often, sensitive attributes are non-binary, such as based on age, race, or nationality. 

Second, the authors argue that their method would also work with other techniques, such as LfF (Nam et al., 2020) and CVaR DRO (Hashimoto et al., 2018a). However, they do not report any experiments to support this claim. While I don’t expect a different conclusion, it would be useful to include one or two additional methods.

Third, the hyper-parameters the authors experiment with are quite limited. They only consider three pairs of learning rate and weight decay combination and the choice of those parameters seem odd to me. A learning rate of 1e-4 is very small for a ResNet50 architecture while a weight decay of 1e-1 seems extremely large! Have the authors tried other combinations but decided not to include them because the model failed to train or did they simply stick to the choices used in JTT? 

Finally, the authors experiment with two vision datasets only. Is there a reason the authors did not try additional datasets, such as Adult Income or Credit Score? It would be interesting to see how the method performs in such cases. My concern is that this line of work assumes that the error of the model is a proxy for the sensitive attribute so it would be useful to see how well this assumption holds in other settings.

**Typos**

There is a typo in Section 4: “on validation data based the EDM criterion” → “on validation data based on the EDM criterion”.

clarity,_quality,_novelty_and_reproducibility: The paper is clearly written and I find the proposed algorithm elegant and novel. The authors include a list of the hyper-parameters they use and provide a code for reproducibility.
summary_of_the_review: I think the paper is novel and the experimental results show a significant improvement. The paper is clearly-written and well-motivated, particularly for such a challenging topic.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: The paper presents a new and effective way to tune the hyperparameters of fairness procedures without requiring access to sensitive attributes.

The reviewers appreciated the proposed method that provides less dependence on sensitive attributes which may not be available in practice or expensive to obtain. The reviewers also appreciated the promising empirical results.

However, several reviewers pointed out that the paper does not validate or motivate the assumptions and reasoning behind why the method should work. Such would be required to complete the paper, and as such, was the key reason for rejection.

justification_for_why_not_higher_score: Not validating the assumptions or motivating why the method should work.
Decision: Reject