Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper focuses on the problem of debiasing representations learned by encoder-decoder models when demographic information is missing. The authors propose Unsupervised Locality-based Proxy Label assignment (ULPL), which assigns a binary “proxy” label to each example based on whether the model incurs a larger or smaller than average loss on that example. Once examples have been assigned proxy labels, it is possible to apply existing debiasing methods. In order to be robust to noisy outliers, the paper also proposes smoothing the proxy label for each example via a majority vote among that example's neighbors in the encoded space. In experiments on 5 datasets (with various protected attributes in focus for each dataset) they show that ULPL leads to larger areas under the performance fairness curve than ARL and a baseline that does not employ fairness interventions; ULPL is claimed to lead to similar performance as some supervised methods. Analysis showing that proxy labels generated by ULPL are generally correlated with protected labels is provided, as well as some analysis of kNN smoothing and an experiment where groups are identities by cross-products of protected attributes.

strength_and_weaknesses: Strengths:
- The crux of the approach in this paper is to try to equalize training loss among all examples. The proposed method, ULPL, simply groups the examples into examples that experience lower than average loss, and those that experience higher.  Once labeled, other debiasing methods are used to equalize the loss. This seems reasonable as a way to improve fairness. The argument made in the paper is that examples with lower loss are likely to be of the same (under-represented) demographic group. While there is some empirical evidence that supports this claim in previous work, I find it weaker and somewhat inconsequential: if training loss is more evenly divided across examples (and training loss is a good proxy for performance) then fairness should improve across all groups.
- This paper is clearly written and easy to follow.
- Experiments are performed with 5 datasets. Significant analysis is also presented.

Weaknesses:
- The proposed label smoothing is not consistently effective. Table 3 shows that in 50% of experimental conditions, no label smoothing is better than label smoothing.
- Label smoothing seems to be impossible to tune in realistic settings (in terms of the number of neighbors k). A standard approach of tuning k would be selecting k that yields the best results on the validation set. Since the metric being optimized is AUC-PFC, demographic group information is required to compute the metric. But the motivation of this paper includes the assumption that no demographic information is available.
- The results are difficult to interpret. Table 1, which includes the main results, reports AUC-PFC, which characterizes the tradeoff between performance and fairness. I assume that the proposed method helps to improve fairness, but does it also improve performance? A more detailed characterization of how ULPL affects performance and fairness would help. Moreover, the difference between the proposed approach and other competing methods is often a few thousandths of a unit of area, which seems small, but as a reader it is difficult to be certain. 


clarity,_quality,_novelty_and_reproducibility: Clarity: while the paper is easy to follow, after reading I’m unsure of the following:
- For each dataset, are the protected attributes in question always binary? How many values can each take on? This could help shed more light on the results.
- For ULPL, is the loss per instance taken from the final model or summed over all training iterations?
- What is the precise way to apply this? Train a model, then label the examples, assign new labels with knn, then debias? Or does this whole process happen at each iteration?
- During neighborhood smoothing, are points relabeled with their smoothed label or do they retain their unsmoothed labeled? I imagine that an example’s smoothed label can be different from its unsmoothed label, leading it to vote differently in smoothing decisions for other examples. 
- In the sensitivity to k-NN hyperparamters section, what is the variable p?
- In the same section: what is class-specific vs. ins-pecific smoothing?

Quality:
- ULPL seems like reasonable approach to debiasing without group information
- The experiments seem to be technically sound, however the results do not seem very significant.
- The writing is clear, but some of the details about the method and experimental methodology have been left out.

Novelty:
- The primary novelty is the idea to group examples into those that experience more/less than average loss (ULPL) as a proxy for protected groups when none are provided. As previously mentioned, while I don’t believe that examples with more loss will always (or even most of the time) correspond to specific protected groups, using existing debiasing methods to more evenly distribute the loss seems like a reasonable way to encourage parity across groups, when groups are unknown.
- The idea to use k-NN smoothing is not particularly novel and moreover, as previously mentioned, it seems problematic from a hyperparameter-tuning perspective. Moreover, it only helps half of the time. 
- The problem of debiasing without group information is interesting, relevant, and has not received much attention.

Reproducibility:
- A link to the code used to run the experiments is provided, so I believe the results could be replicated. However, I have not tried to replicate them myself.

Other comments:
- Please bold the best results in the table to make them easier to read.
- Consider adding a citation to work on dataset cartography, which also groups points based on how difficult they are to predict correctly

summary_of_the_review: The paper is written clearly but some important details are missing. One of the methods proposed in this paper (label smoothing) does not consistently provide benefit and seems impossible to tune. The primary method proposed (ULPL) seems reasonable but doesn’t lead to significant gains.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The paper proposes a new framework to achieve fair outcomes in settings where access to sensitive attributes is not available. The method is called Unsupervised Locality-base Proxy Label assignment (ULPL) and is based on assigning proxy labels according to model predictions for poor vs well-modeled instances. The authors perform various experiments on datasets from ML and NLP domains.
strength_and_weaknesses: **Strengths:**
1. The paper studies an important subject. Achieving fair outcomes in settings where access to protected attributes is limited is important and interesting.
2. Authors perform various experiments on datasets both from NLP and ML domains to showcase various aspects of the paper.
3. The method is easy and straightforward.
4. Paper is well-written and easy to follow.


**Weaknesses:**
1. It seems like the method is data dependent. Maybe results from section 5.1 on TrustPilot dataset is exactly due to this reason? I think further analysis or discussion need around this issue.
2. Related to my above comment, while the paper does a good job in terms of empirical analysis, theoretical discussion is lacking. This theoretical discussion maybe address my above comment (1). That this approach can be applicable regardless of specific data properties.
3. The performance of the approach is not impressive considering that comparison with the only unsupervised approach ARL is not done in a fair manner. 
4. How much do you think the base supervised approach that ULPL is implemented over is having effect on the results. Although authors discuss this briefly in "Different ULPL methods:" section. More in depth discussion might be useful to improve the paper.
5. The approach is dependable on some parameters such as k which might make it inefficient. Is there a good way to make this more flexible?

**Additional Questions that needs clarification:**

The framework based on indicating poorly vs well modeled instances seems more aligned with equalized odds or equality of opportunity notions of fairness. Do authors know why their approach works for statistical parity which is different in nature than what is proposed here? 

clarity,_quality,_novelty_and_reproducibility: **Clarity:**

The paper was written clearly, and it is easy to follow. 

**Quality:**

The paper is of high quality in terms of empirical experiments, but not so much theoretically. 

**Novelty:**

While the problem on its own is not very novel, the approach proposed can be considered to some extend novel. 

**Reproducibility:**

Authors provided an anonymous source code which will be really useful for reproducibility. Discussions in the Appendix also is comprehensive which can make it easier to reproduce the exact results in the paper.
summary_of_the_review: Overall, the paper is a well-written and motivated paper; however, some deeper theoretical discussions can improve the paper drastically. The approach has also some flaws in terms of dependability to data, parameters, as well as some performance related concerns which I discussed in detail under weaknesses.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: Bias creeps into ML models trained from human behavioural data, which can usually be mitigated against but (typically) with labeled data for that purpose. This paper proposes a meta-algorithm for debiasing representations called Unsupervised Locality-based Proxy Label assignment. It is evaluated over five datasets, from NLP to structured data, and this evaluation shows that the provided technique recovers proxy labels that correlate with ‘unknown demographic data’. Performance is competitive with supervised methods. 
strength_and_weaknesses: Stengths

- Bias remains a crucial (and ongoing) challenge across many subdomains of ML. It is commendable that this work generalized beyond one of these subdomains in its analysis. Moreover, the focus on unlabeled demographics is more ecologically valid and therefore useful.
- Good, relevant, and (mostly) recent literature is effectively compared and contrasted in a well-structured manner.
- The datasets and baselines for empirical evaluation are well chosen
- The questions asked in each of the sub analyses of Sec 5 are well-chosen and mostly well executed

Weaknesses

- There are multiple group-wise measures of fairness beyond the one adopted in Sec 3.1. A minor point, but it would be preferable to show some rationale for the method chosen, and some indication of the alternatives.
- Certain assumptions seem to be glossed over in Sec 3.2, such as how (or whether) over- or under-represented groups can be binarized according to z_i’. This may of course be a useful heuristic empirically, but whether it has anything to do with actual groups or is merely correlated with them for other, more spurious reasons should be better explained (e.g.., in sec 5.2). 
- Empirical results are fairly similar across rows, for multiple datasets and attributes, in Table 1. That this is the case for the unsupervised method is actually a positive outcome, but claims of ‘consistently outperforming’ other approaches seems to require some tempering, and some statistical tests of significance should nevertheless be run.

clarity,_quality,_novelty_and_reproducibility: - The paper is written clearly, and mostly with sufficient depth although some assumptions are glossed over. As the authors note, other approaches exist to unsupervised approaches in this space, but the simple approach proposed here is technically novel. The baselines are run with the fairlib library. The Appendix is fairly complete

summary_of_the_review: 
- The positives of moderate novelty, fairly good empirical approaches (sub analyses, datasets, baselines), and the importance of this space outweigh negatives of glossed-over rationales and assumptions, and perhaps some overstatement of performance.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The paper proposes an unsupervised method for bias-mitigation. The presented approach leverages proxy labels derived from comparing the instance-wise loss values to a threshold. These proxy labels are consolidated/smoothed through nearest neighbors in latent space (through embedding). The labels indicate poorly and well represented groups, which can be used in combination with arbitrary supervised methods for obtaining fairer classification models. The approach is extensively evaluated and is competitive with group-label supervised methods for bias mitigation.
strength_and_weaknesses: Strengths:

1. The paper is well written. I enjoyed reading this paper and found it well-structured and good to follow.
2. The paper addresses an important and timely problem. Bias is common in machine learning and methods to deal with it are relevant in practice. Furthermore, bias is often introduced implicitly, i.e., without group labels known at model training time. Therefore, the proposed method may be a valuable step towards fairer automated decisions.
3. The experimental evaluation is extensive and rigorous and overall convincing. It includes several data modalities (tabular and text), fairness measures, datasets and baselines. Furthermore, important sanity checks such as the validity of the nearest-neighbor approximation for smoothing are in place. I briefly looked at the experimental code which appears to be in a good state. Intersectional discrimination of multiple attributes is also considered.
4. The approach is simple and can be implemented in a few lines of code. Also, it only has one hyperparameter (the number of neighbors used for smoothing). This makes the method ready-to-use for practical applications.

Weaknesses:

1. There is no comparison between proxy labels and ground truth groups (e.g., given by race, gender, etc.). However, it is frequently claimed that the proxy labels correlate with demographic groups (for instance, right in the abstract, it says: “[...] that correlate with unknown demographic data”). The claim is repeated in 5.1, however, the performance gap is compared to the proxy labels only and not the ground truth group labels. This could be achieved by providing the statistics, e.g., confusion matrices, with ground truth group labels.
2. The theoretical grounding of the work is weak, which raises the question whether there exist other simpler and principled solutions. While the paper provides some intuition for why the proposed method works, it lacks theoretical evidence from point to point. For instance, it is not clear why U should be generally replaced with the Loss in Section 3.2 and how the group-based formulation is transformed to an individual one. I see that the experimental results are promising, but the derivation could be backed with more evidence or theoretical results.
3. There are no theoretical guarantees that the proposed method will lead to fairer models and that the proxy objective corresponds to the actual one (or bounds on the error). This also raises the question if the fairness could possibly be decreased. This could be a problem in high-stakes scenarios. In such settings, a black-box method like the one presented is unlikely to be deployed.

Minor comments:
1. I would have appreciated further evidence or intuition for some claims made in the paper. Here are some examples: 
(1) Because TrustPilot is already fair, the ground truth labels cannot be aligned with the TPR gaps (Section 5.1) 
(2) Why is GD-CLA more robust to conflated classes (final paragraph of 4.2) 
(3) The discussion of upstream and downstream debiasing (Section 2, “Unsupervised bias mitigation”). As far as I understand, upstream debiasing is also unsupervised? Thus, a comparison with one of these methods could be insightful in terms of future work, since these methods could be applicable to the same set of problems.
2. Related work: Please use \citep (end of first paragraph, Point (1) in second paragraph).

Questions:
1. How exactly was the ROC-RFC computed? Were the scores normalized? For instance, looking at Figure 5b), the area under the curve is well above 0.8 x 0.8=0.64 for both methods. Even if you normalize the curve (taking the left and right-most point as references), the areas are well above 0.5. However, in Table 1, values reported for Bios-G are all below 0.5. How can this be explained?
2. ROC-PFC values for Trust Pilot, Adult, Compas are extremely low (~0.1). This indicates that performance is hurt drastically even with slowly introduced fairness. Why is this the case? Is the AUC-PFC a still a representative measure in this case? Maybe this is connected to the first question.
3. Table 3 shows the best values for the nearest neighbors smoothing which are different across datasets. How would you choose a suitable value for k in practice, without having access to ground truth fairness measures?




clarity,_quality,_novelty_and_reproducibility: Clarity: Overall, the work is well presented. The method and experimental setup are well described. 
Novelty: Some works follow a similar high-level sprit (e.g, Hashimoto et al.) but the follow-up ARL method is a competitor in the experiments and an improvement over this method is shown. Therefore, I would consider this work as sufficiently novel, but there might be some works which I have missed.
Reproducibility. The authors provide the full source code which appears to be in a good shape. Therefore, I am confident that the results are reproducible.

summary_of_the_review: The topic of bias mitigation is relevant and the empirical results in this paper are interesting. While the empirical evidence suggests that the method improves fairness on real datasets, there are no theoretical guarantees or considerations, making it hard to deploy this method in high-stakes applications. However, I found the experiments well-designed and executed and the results very promising, which is why I am currently leaning towards acceptance.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: The paper presents a framework to mitigate bias without the need to access sensitive attributes. Overall, all the reviewers are positive about this paper and major concerns are resolved during the rebuttal. 

Strengths:
- The setting is practical as in real-world scenarios, sensitive attributes are often not available or are prohibited to keep due to privacy concerns. Achieving fair outcomes under this condition is a challenging and important topic. 
- The experiments are comprehensive and conducted studies in multiple domains. 
- Overall, the paper is well-written and easy to follow. 

Weaknesses: 
- The theoretical justification of the proposed approach is relatively weak. 
- There are some over-claiming and handwaving assumptions. A careful revision is needed before publishing. 


Missing reference: 
The setting of evaluating NLP fairness without giving sensitive attributes is discussed in the following paper.
LOGAN: Local Group Bias Detection by Clustering, EMNLP 2020

justification_for_why_not_higher_score: Although the authors provide additional discussion in the rebuttal period, the theoretical justification of the paper is relatively weak as pointed out by multiple reviewers. 

justification_for_why_not_lower_score: All reviewers are positive about this paper, and there is no major concern.
Decision: Accept: poster