Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The paper presents a study on the effect of training dml techniques on imbalanced data and show the negative impact of learned representations on the downstream tasks. The fairness is analyzed through 3 properties of the representation space; a) inter-class alignment, b) intra-class alignment, c) uniformity showing that the bias in the upstream task (dml) is propagated to the downstream classification tasks even when the data for training the classifier (downstream task) is balanced. To address this, an objective (PARADE) that de-correlates the discriminative features from the sensitive attributes are learned during training. The experiments are conducted on 4 benchmark datasets.
main_review: Strengths: 
------------
1) The first study that shows the effect of dml training on imbalanced datasets and how they are propagated to the downstream tasks
2) An exhaustive set of experiments showing this bias due to learning from imbalanced data
3) The paper is well written and organized
4) Reproducible as the code is available

Weaknesses:
----------------
1) Limited scope: The study proposed in this paper is limited to deep metric learning tasks and hence may not generalize to other tasks. The PARADE objective that employs DIVA technique to de-correlate the sensitive attribute from the discriminative features is also taylored to dml tasks. Hence all the analysis done in this paper would be limited to dml applications only. In addition, I have seen very few works that use dml for downstream tasks like classification because the main aim of dml is to learn a good metric (embedding space) where the features can be compared and generally used in scenarios where classification may not work well (large number of classes with few images in each class). I would like to know whether the analysis used here can be widely used for other tasks?
 
2) The method depends on user specified predefined attributes which could limit the applicability of the proposed approach. Also it can only be applied to datasets where these type of attributes are available. 
summary_of_the_review: The paper studies one of the sources of bias (data imbalance) in machine learning applications. Given the nature of current datasets and the bias with the current machine learning techniques, this is an important topic to explore. Although the proposed technique is currently applied/evaluated under restricted settings, this is one of the first work to do so for deep metric learning.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper proposes three measures that evaluate the fairness of learned representations in multiple aspects. The authors empirically demonstrate that the existing metric learning approaches become less fair (i.e., shows larger performance gaps between attribute-based subgroups) when there is a class imbalance in the training data. Specifically, they propose a protocol that evaluates the fairness by manually adjusting the number of samples per class in the training set (finDML). In addition, they propose PARADE that de-correlates image embedding from auxiliary attibutes embedding via adversarial separation to improve fairness of the learned image embedding. In the experiments, the method shows overall improved fairness on five datasets.
main_review: - First of all, the paper addresses an important problem, i.e., the fairness of the learned representation in deep metric learning. They provide an interesting observation that the existing deep metric learning methods become less fair when trained with class imbalance, and the learned bias propagates to the downstream tasks even when the downstream task datasets are balanced.

- They carefully designed the fairness measures in three different aspects, which are reasonable and help comprehensive understanding of the model behavior. In addition, they provide extensive experiments on five datasets with several DML methods that shows the decreased fairness for imbalanced dataset and motivates this work.

- I think the method itself serves as a good baseline but is technically not very novel. They employed the existing adversarial separation method and the idea to de-correlate two features are not new. Considering that there are two evaluation measures, accuracy and fairness, it would be interesting to study how to alleviate their trade-off.

- PARADE reduces the performace gaps between subgroups with different attributes on some datasets, but it does not always improve fairness depending on the dataset (Table 2).

- From the current explanation, it is unclear to me why "class" imbalance leads to the larger performance gaps between subgroups of different "attributes", which is also related to the proposed evaluation protocol.

- Since the proposed method can be appiled to any formulation of loss \mathcal{L}, it would be interesting to see if the method generalzes and improves fairness for different choices of loss formulations.

- As discussed in the limitations section, there is a trade-off between utility and the fairness. It is good that comparison of the actual accuracy is provided for some datasets in the supplmentary document. It shows that fairness was improved at the cost of overall low accuracy. It would be also interesting to discuss how to overcome the trade-off.

- Additional analysis of the effect of hyper-parameters (alpha and rho) would be necessary. It would be useful to discuss the strategy how to determine the values given a desired trade-off of fairness and utility.

- The method uses ground truth attribute labels to train a model. One simple baseline would be train a model with attribute-balanced sampling, i.e., to construct minibatch through balanced sampling over attributes. How does the method work on top of this baseline?
summary_of_the_review: Overall, the paper addresses an important problem, i.e., fairness in deep metric learning. Although the method does not always improve fairness compared to the baseline and the technical novelty seems weak, I think this work could motivate research on fairness for deep metric learning.

---
After authors' feedback:  
I appreciate the extensive experiments and detailed answers. Most of my concerns are addressed.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper provides the finDML (fairness in non-balanced DML) benchmark in the scope of deep metric learning to characterize representation fairness and provide the method for reducing subgroup gaps in deep metric learning methods.

main_review: Strengths:
1. This is a well-written paper and it is easy to read.
2. This paper studies a new problem and provides a new perspective for the research in this field.
3. The experimental results are sufficient.

Weakness:  

The proposed method looks simple and not novel enough (since the proposed method is largely based on previous work).
summary_of_the_review: This paper is well-written and the proposed problem is intrersting. The supplementary materials provided by the author are very sufficient, and the comparative experiment is very detailed. The innovation of the method is general, but it systematically studies a new problem, so it is recommended to accept it.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The paper investigates the fairness problem in the deep metric learning task, which is underexplored by the research community. The authors propose finDML to benchmark previous methods on multiple imbalanced datasets with three newly proposed metrics. The experimental results show that all previous deep metric learning methods have the fairness issue, i.e., larger performance gaps across different subgroups. The authors further propose PARADE to mitigate the biases, which has been shown to be effective by the experiments.
main_review: Strengths:
1. The paper is well-written and easy to follow.
2. The paper studies an underexplored problem and proposes evaluation metrics to benchmark existing methods, which can inspire other researchers for future works.

Weaknesses:
1. The novelty of PARADE seems to be limited since the authors directly “use the adversarial separation (de-correlation) method from (Milbich et al., 2020)” (Subsec. “Partial De-correlation” in Sec. 4). Could the authors further demonstrate the novelty compared with (Milbich et al., 2020)? Otherwise, I do not think the authors can claim PARADE as “a novel objective” as one of the contributions (last paragraph of Sec. 1).
2. The authors show that the biases can be propagated to the downstream classification task and a naive method, training classifiers on the balanced dataset, cannot simply address the fairness issue. Since there are many bias mitigation methods ([1]) proposed for image classification, can authors also benchmark those bias mitigation methods for downstream classification? I understand that the authors want to pursue “fairness in representations” by proposing PARADE to address the fairness problem in the upstream embedding. However, I believe that the authors need to add some non-trivial bias mitigation methods to show that the bias cannot be fully addressed solely in the downstream stage, which can further demonstrate the necessity of achieving fairness in the upstream task, i.e., learning fairer representation in deep metric learning.

[1] Z. Wang et al., “Towards Fairness in Visual Recognition: Effective Strategies for Bias Mitigation,” in CVPR, 2020

summary_of_the_review: Since the paper has weaknesses in terms of the novelty of PARADE and missing experiments of bias mitigation in the downstream classification task, I recommend “marginally below the acceptance threshold” to this paper. I would like to increase my rating if the authors address my concerns.

============

After Authors' Response:

I appreciate the authors’ feedback, which addressed my concerns. I will increase my rating to “6: marginally above the acceptance threshold.”
review_title: null

Meta-review: This paper investigates an important problem, i.e., the fairness of the learned representation in deep metric learning, which is relatively under-explored by the research community. Observing that the existing metric learning approaches become less fair when trained on an imbalanced dataset, the authors propose finDML to benchmark previous methods on multiple imbalanced datasets with three newly proposed metrics. 
Further, a PARADE module is adapted into this problem to tackle the fairness issue. 

The paper is meticulously written of good structure, and well motivated by experimental findings. The authors have a deep and thorough discussion with reviewers, through which the mixed preliminary ratings became all positive, with most concerns well addressed. AC found no ground for rejection and thus recommended acceptance. Authors shall integrate all response material into the next revision.
Decision: Accept (Poster)