Year: 2021

Abstract:
Deep metric learning (DML) enables learning with less supervision through its emphasis on the similarity structure of representations. There has been much work on improving generalization of DML in settings like zero-shot retrieval, but little is known about its implications for fairness. In this paper, we are the first to evaluate state-of-the-art DML methods trained on imbalanced data, and to show the negative impact these representations have on minority subgroup performance when used for downstream tasks. In this work, we first define fairness in DML through an analysis of three properties of the representation space – interclass alignment, intra-class alignment, and uniformity – and propose finDML, the fairness in non-balanced DML benchmark to characterize representation fairness. Utilizing finDML, we find bias in DML representations to propagate to common downstream classification tasks. Surprisingly, this bias is propagated even when training data in the downstream task is re-balanced. To address this problem, we present Partial Attribute De-correlation (PARADE) to de-correlate feature representations from sensitive attributes and reduce performance gaps between subgroups in both embedding space and downstream metrics.

heading: 1 INTRODUCTION
text: Deep metric learning (DML) extends standard metric learning to deep neural networks, where the goal is to learn metric spaces such that embedded data sample distance is connected to actual semantic similarities (Globerson & Roweis, 2006; Weinberger et al., 2006; Hoffer & Ailon, 2018; Wang et al., 2014). The explicit optimization of similarity makes deep metric spaces well suited for usage in unseen classes, such as zero-shot image or video retrieval or facial re-identification (Milbich et al., 2021; Roth et al., 2020c; Musgrave et al., 2020; Hoffer & Ailon, 2018; Wang et al., 2014; Schroff et al., 2015; Wu et al., 2018; Roth et al., 2020c; Brattoli et al., 2020; Hu et al., 2014; Deng et al., 2019; Liu et al., 2017). However, while DML is effective in establishing notions of similarity, work describing potential fairness issues is limited to individual fairness in standard metric learning (Ilvento, 2020), disregarding embedding models.
Indeed, the impacts and metrics of fairness are well studied in machine learning (ML) generally, and representation learning specifically (Dwork et al., 2012; Mehrabi et al., 2019; Locatello et al., 2019b). This is especially true on high-risk tasks such as facial recognition and judicial decision-making (Chouldechova, 2017; Berk, 2017), where there are known risks to minoritized subgroups (Samadi et al., 2018). Yet, relatively little work has been done in the domain of DML (Rosenberg et al., 2021). It is crucial to address this knowledge gap – if DML embeddings are used to create upstream embeddings that facilitate downstream transfer tasks, biases may propagate unknowingly.
To tackle this issue, this work first proposes a benchmark to characterize fairness in non-balanced DML - finDML. finDML introduces three subgroup fairness definitions based on feature space performance metrics – recall@k, alignment and group uniformity. These metrics measure clustering ability and generalization performance via feature space uniformity. Thus, we select the metrics for our definitions to enforce independence between inclusion in a particular cluster or class, and a protected attribute (given the ground-truth label). We leverage existing datasets with fairness limitations (CelebA (Liu et al., 2015) and LFW (Huang et al., 2007)) and induce imbalance in training data of standard DML benchmarks, CARS196 (Krause et al., 2013) and CUB200 (Wah et al., 2011), in order to create an effective benchmark for fairness analysis in DML.
Making use of finDML, we then perform an evaluation of 11 state-of-the-art (SOTA) DML methods representing frequently used losses and sampling strategies, including: ranking-based losses (Wang et al., 2014; Hoffer & Ailon, 2018), proxy-based (Kim et al., 2020) losses, semi-hard sampling (Schroff et al., 2015) and distance-weighted sampling (Wu et al., 2018). Our experiments suggest that imbalanced data during upstream embedding impacts the fairness of all benchmarks methods in both upstream embeddings (subgroup gaps up to 21%) as well as downstream classifications (subgroup gaps up to 45.9%). This imbalance is significant even when downstream classifiers are given access to balanced training data, indicating that data cannot naively be used to de-bias downstream classifiers from imbalanced embeddings.
Finally, inspired by prior work in DML on multi-feature learning (Milbich et al., 2020), we introduce PARtial Attribute DE-correlation (PARADE). PARADE addresses imbalance by de-correlating two learned embeddings: one learnt to represent similarity in class labels, and one learnt to represent similarity in the values of a sensitive attribute, which is discarded at test-time. This creates a model in which the ultimate target class embeddings have been de-correlated from the sensitive attributes of the input. We note that as opposed to previous work on variational latent spaces, PARADE de-correlates a learned similarity metric. We find that PARADE reduces gaps of SOTA DML methods by up to 2% downstream in finDML.
In total, our contributions can be summarized as follows:
1. We define finDML; introducing three definitions of fairness in DML to capture multifaceted minoritized subgroup performance in upstream embeddings through focus on feature representation characteristics across subgroups, and five datasets for benchmarking. 2. We analyze SOTA DML methods using finDML, and find that common DML approaches are significantly impacted by imbalanced data. We show empirically that learned embedding bias cannot be overcome by naive inclusion of balanced data in downstream classifiers. 3. We present PARADE, a novel a novel adaptation of previous zero-shot generalization techniques to enhance fairness guarantees through de-correlation of class discriminative features with sensitive attributes.

heading: 2 BACKGROUND
text: Deep Metric Learning DML extends standard metric learning by fusing feature extraction and learning a parametrized metric space into one end-to-end learnable setup. In this setting, a large convolutional network ψ provides the mapping to a feature space Ψ, while a small network f , usually a single linear layer, generates the final mapping to the metric or embedding space Φ. The overall mapping from the image space X is thus given by ϕ = f ◦ ψ. Generally, the embedding space is projected on the unit hypersphere SD−1 through normalization (Weisstein, 2002; Wu et al., 2018; Roth et al., 2020c; Wang & Isola, 2020) to limit the volume of the representation space with increasing embedding dimensionality. The embedding network ϕ is then trained to provide a metric space Φ that operates well under some predefined, usually non-parametric metric such as the Euclidean or cosine distance defined over Φ.
Typical objectives used to learn such metric spaces range from contrastive ranking-based training using tuples of data, such as pairwise (Hadsell et al., 2006), triplet- (Schroff et al., 2015; Wu et al., 2018) or higher-order tuple-based training (Sohn, 2016; Wang et al., 2020a), procedures to bring down the effective complexity of the tuple space (Schroff et al., 2015; Harwood et al., 2017; Wu et al., 2018) or the introduction of learnable tuple constituents (Movshovitz-Attias et al., 2017; Qian et al., 2019; Kim et al., 2020).
More recent work (Milbich et al., 2020; Roth et al., 2020c; Jacob et al., 2019) extends standard DML training through incorporation of objectives going beyond just sole class label discrimination: e.g., through the introduction of artificial samples (Lin et al., 2018; Duan et al., 2018), regularization of higher-order moments (Jacob et al., 2019), curriculum learning (Zheng et al., 2019; Harwood et al., 2017; Roth et al., 2020a), knowledge distillation (Roth et al., 2020b) or the inclusion of additional features (DiVA) to produce diverse and de-correlated representations (Milbich et al., 2020).
DML Evaluation Standard performance measures reflect the goal of DML: namely, optimizing an embedding space Φ for best transfer to new test classes via learning semantic similarities. As
immediate applications are commonly found in zero-shot clustering or image retrieval, respective retrieval and clustering metrics are predominantly utilized for evaluation. Recall@k (Jegou et al., 2011) or mean average precision measured on recall (Roth et al., 2020c; Musgrave et al., 2020) typically estimate retrieval performance. Normalized mutual information (NMI) on clustered embeddings (Manning et al., 2010) is used as a proxy for clustering quality (see Supplemental for detailed definitions). We leverage these performance metrics to inform finDML and our experiments.
Fairness in Classification Formalizing fairness in ML continues to be an open problem (Mehrabi et al., 2019; Chen et al., 2018a; Chouldechova, 2017; Berk, 2017; Locatello et al., 2019b; Chouldechova & Roth, 2018; Dwork et al., 2012; Hardt et al., 2016; Zafar et al., 2017). In classification, definitions for fairness such as demographic parity, equalized odds, and equality of opportunity, rely on model outputs across the random variables of protected attribute and ground-truth label (Dwork et al., 2012; Hardt et al., 2016).
Fairness in Representations A more relevant family of fairness definitions for DML would be those explored in fairness for general representation learning (Edwards & Storkey, 2015; Beutel et al., 2017; Louizos et al., 2015; Madras et al., 2018). Here, the goal is to learn a fair mapping from an original domain to a latent domain so that classifiers trained on these representations are more likely to be agnostic to the sensitive attribute in unknown downstream tasks. This assumption distinguishes our setting from previous fairness work in which the downstream tasks are known at train time (Madras et al., 2018; Edwards & Storkey, 2015; Moyer et al., 2018; Song et al., 2019; Jaiswal et al., 2019). DML differs from this form of representation learning as it aims to learn a mapping capturing semantic similarity, as opposed to latent space representation.
Earlier works in fair representation learning intended to obfuscate any information about sensitive attributes to approximately satisfy demographic parity (Zemel et al., 2013) while a wealth of more recent works focus on using adversarial methods or feature disentanglement in latent spaces of VAEs (Locatello et al., 2019a; Kingma & Welling, 2013; Gretton et al., 2006; Louizos et al., 2015; Amini et al., 2019; Alemi et al., 2018; Burgess et al., 2018; Chen et al., 2018b; Kim & Mnih, 2018; Esmaeili et al., 2019; Song et al., 2019; Gitiaux & Rangwala, 2021; Rodríguez-Gálvez et al., 2020; Sarhan et al., 2020; Paul & Burlina, 2021; Chakraborty et al., 2020). In this setting, the literature has focused on optimizing on approximations of the mutual information between representations and sensitive attributes: maximum mean discrepancy (Gretton et al., 2006) for deterministic or variational (Li et al., 2014; Louizos et al., 2015) autoencoders (VAEs); cross-entropy of an adversarial network that predicts sensitive attributes from the representations (Edwards & Storkey, 2015; Xie et al., 2017; Beutel et al., 2017; Zhang et al., 2018; Madras et al., 2018; Adel et al., 2019; Zhao &
Gordon, 2019; Xu et al., 2018); balanced error rate on both target loss and adversary loss (Zhao et al., 2019); Weak-Conditional InfoNCE for conditional contrastive learning (Tsai et al., 2021).
PARADE shares aspects of these previous methods in its choice of de-correlation or disentanglement. However, PARADE de-correlates the learned similarity metric as opposed to the latent space. In addition, with DML-specific criteria, PARADE learns similarities over the sensitive attribute while not directly removing all information about the sensitive attribute, as the sensitive attribute and target class embeddings share a base network.
3 EXTENDING FAIRNESS TO DML - finDML BENCHMARK
To characterize fairness with finDML, this section introduces the key constituents – definitions to characterize fairness in embedding spaces and respective benchmark datasets.

heading: 3.1 PRELIMINARIES
text: Our embedding space fairness definitions rely on embedding space metrics adapted from (Wang & Isola, 2020) and (Roth et al., 2020c), namely alignment and uniformity. Both metrics we use to characterize embeddings for our definitions in the next section (intra- as well as inter-class alignment and uniformity) have been successfully linked to generalization performance in contrastive selfsupervised and metric learning models (Wang & Isola, 2020; Roth et al., 2020c; Sinha et al., 2020). Alignment succinctly captures the similarity structure learned by the representation space with respect to the target labels through measuring distances between pairs of samples. On the other hand, notions of uniformity can differ. Uniformity of the sample distribution over the hypersphere has been studied through the radial basis function (RBF) over pairs of samples. Alternatively, uniformity of the feature space has been studied through the KL-divergence DKL between the discrete uniform distribution UD and the sorted singular value distribution Sϕ(X) of the representation space ϕ on dataset X .
UKL(X) = DKL ( UD,Sϕ(X) ) (1)
Here, lower scores indicate more significant directions of variance in learned representations. Both introduced notions of uniformity represent important aspects of the embedding space, but the computational overhead in computing RBF over all pairs of samples in large datasets makes it impractical for our uses and is less interpretable than UKL. Therefore, we leave the uniformity metric utilized in finDML general, but utilize UKL for our experiments.

heading: 3.2 DEFINING FAIRNESS
text: Building on the aforementioned performance metrics, we introduce three definitions for fairness in the embedding spaces of DML models. As the recall@k and alignment metrics inform inclusion in an embedded cluster (or class), we follow fair classification literature in the motivation for our first fairness definition: inclusion in a class should be independent of a protected attribute given the ground-truth label. Thus, we examine the probability of encountering a data instance of the same class in a data point’s k-nearest neighbors to form the first definition. The second definition relies on equal expectation of alignment across sensitive attribute values. Departing from classification literature, our third definition encapsulates fairness in a task-agnostic sense (as DML is often applied in such settings): fairness across the “goodness" of the learned features via a uniformity metric.
Let X denote the input data, and A a protected attribute variable. Denote Xa the partition of X with attribute a ∈ A. To recap common DML terminology, a positive pair of samples is defined as (x1, x2) ∈ X ×X s.t. the class label of x1 and x2 are identical. A negative pair of samples is defined as (x1, x2) ∈ X ×X such that the class label of x1 and x2 differ. Let Pa denote the set of all positive pairs s.t. at least one of x1 or x2 has attribute a ∈ A, and analogously for Na and negative pairs. Definition 1 (K-Close Fairness). Define NNk : Φ ⊂ SD−1 → P(X) as a function that receives a point ϕ(x) ∈ Φ and returns a set in the powerset of X , P(X), containing points in X that map to the k nearest neighbors of ϕ(x) in Φ. Thus, ϕ is k-close fair with respect to attribute A if:
Pr x∈Xa (∃x̃ ∈ NNk(ϕ(x))s.t.Y (x̃) = Y (x)) = Pr x∈Xb (∃x̃ ∈ NNk(ϕ(x))s.t.Y (x̃) = Y (x)) ∀a, b ∈ A (2) Note: the criteria weakens as k increases, similar to recall@k.
Definition 2 (Alignment). ϕ is fair, according to alignment with respect to attribute A, if: E(x1,x2)∈Pa [||ϕ(x1)− ϕ(x2)|| 2] = E(x1,x2)∈Pb [||ϕ(x1)− ϕ(x2)|| 2] (3)
E(x1,x2)∈Na [||ϕ(x1)− ϕ(x2)|| 2] = E(x1,x2)∈Nb [||ϕ(x1)− ϕ(x2)|| 2] ∀a, b ∈ A (4) i.e. the expectation of the alignment is equal across domain of A. Definition 3 (Uniformity Across Groups). ϕ is fair, according to uniformity, and with respect to attribute A, if the expectation of the uniformity is equal across domain of A:
U(ϕ(Xa)) = U(ϕ(Xb)) ∀a, b ∈ A (5) where U(·) denotes some measure of uniformity over a set V ∈ SD−1.
3.3 CONSTRUCTED finDML BENCHMARK DATASETS
finDML encompasses existing DML benchmark datasets, CUB200 and CARS196, and facial recognition datasets, CelebA and LFW (Wah et al., 2011; Krause et al., 2013; Liu et al., 2015; Huang et al., 2007). For fairness analysis, we investigate bird color in CUB2001, Race in LFW and Skintone in CelebA (Kumar et al., 2009). A detailed description of dataset and attribute labeling is included in the Supplemental. To create additional fairness benchmarks, we induce class imbalance in CUB200 and CARS196, as both datasets are naturally balanced w.r.t. class.
Manually Introduced Class Imbalance We introduce imbalance by reducing the number of training data samples of 50 randomly selected classes by 90% (Imbalanced). We run an experiment with the original datasets as a balanced control (Balanced) for comparison. In the imbalanced setting, we adjust (increase) the number of training samples of the majoritized groups to match the number of datapoints in the balanced control experiments. We average metrics over 10 sets of 50 randomly selected classes for imbalanced experiments. We use the standard ratio of 50 − 50 for train-test split of these datasets, but split over number of data points per class, as opposed to splitting over the classes themselves. The manually imbalanced datasets are used to benchmark standard DML methods, validate our framework, and analyze downstream effects.
Although dataset imbalance does not constitute the sole source of bias in machine learning applications, unfairness as a result of imbalance is the most well-understood in the literature (Chen et al., 2018a). Additionally, we do not assume for our naturally imbalanced datasets, particularly the facial datasets, that attribute imbalance is the only source of bias we observe.

heading: 4 PARTIAL ATTRIBUTE DE-CORRELATION (PARADE)
text: In this section, we present Partial Attribute De-correlation, or PARADE, in which we incorporate adversarial separation (Milbich et al., 2020) during training to de-correlate separate embeddings. We enumerate several significant changes: 1) only target embedding released at test-time; 2) triplet formation and loss term w.r.t. sensitive attribute; 3) de-correlation with sensitive attribute as opposed to de-correlation to reduce redundancy in concatenated feature space. These two representations branch off from the deep metric embedding model at the last layer. The two representations encode the similarity metrics learned over the sensitive attribute and target class, respectively. The sensitive attribute embedding layer is discarded at test time. The resulting network expresses a similarity metric with respect to the target class, de-correlated from the sensitive attribute (Figure 1). Therefore, PARADE figuratively optimizes the first two fairness definitions proposed in Section 3.2 via an objective that maximizes independence between the sensitive attribute and target class.
Objective Term Per Embedding To achieve efficient training and de-correlation of the target class and the sensitive attribute embedding layers, we simultaneously train both layers that branch from the penultimate layer of the model and de-correlate at each iteration. Because PARADE must learn one embedding w.r.t. target class (ϕtarg) and one embedding w.r.t. the sensitive attribute (ϕSA), we introduce separate objectives for each embedding:
Ltarg = 1
N ∑ t∼Ttarg L(t) LSA = 1 N ∑ t∼TSA L(t)
1While bird color in CUB200 does not represent a real-world fairness setting, CUB200 is widely used as a DML benchmark. Thus, a fairness angle allows fairness analysis of previous methods benchmarked on CUB200.
where N is the number of training triplet samples, and L represents a generic loss function, such as triplet loss (Hoffer & Ailon, 2018). We use t ∼ Ttarg to illustrate sampling over triplets of the form (xa, xp, xn) where xa and xp are of the same target class and xa and xn are of differing target classes. Similarly, t ∼ TSA indicates sampling over triplets of the form (xa, xp, xn) where xa and xp are of the same sensitive attribute subgroup and xa and xn are of differing sensitive attribute subgroups. See Figure 2 for a t-SNE visualization of the distinct embeddings of PARADE.
Partial De-correlation In order to minimize the correlation between ϕtarg and ϕSA, we use the adversarial separation (de-correlation) method from (Milbich et al., 2020), which minimizes the mutual information between a pair of embeddings. The task of mutual information minimization is accomplished through learning an MLP to maximize the pair’s correlation, c, and consequently performing a gradient reversal R, which inverts the gradients during backpropagation. The MLP, ξ, is trained to maximize c(ϕtargi , ϕ SA i ) = ∥R(ϕ targ i ) ⊙ ξ(R(ϕSAi ))∥22, s.t. ⊙ denotes element-wise multiplication. Combining the loss terms results in total loss:
LPARADE = Ltarg + αSALSA − ρ · c(ϕtarg, ϕSA)
where αSA weights the sensitive attribute loss and ρ weights the degree of de-correlation. ρmodulates the de-correlation term to allow ψ to retain some attribute information (i.e. partial de-correlation). Thus, the deployed model ϕtarg = ftarg ◦ ψ can retain information about the sensitive attribute in its feature representations, as αSALSA appears in the loss function back-propagated through the full model ψ. The extent to which the sensitive attribute affects the output features is controlled by αSA; we suggest optimizing αSA ∈ (0, 1) and ρ through maximization of worst-group performance (Lahoti et al., 2020) (See Supplemental C.5 for further analysis of PARADE hyperparameters).

heading: 5 EXPERIMENTS
text: Baseline DML Methods For all datasets, we use a ResNet-50 (He et al., 2016) architecture with best performing parameters on a validation set (for further implementation details, see Supplemental). To investigate a sweeping set of frequently used DML methods, we benchmark across a diverse, representative set of 11 techniques, including: three standard ranking-based losses (margin, triplet, n-pair, and contrastive) three batch mining strategies (random, semi-hard and distance-weighted sampling) and three common loss functions (multisimilarity loss, ArcFace loss, for handling facial datasets, and proxy-based loss, ProxyNCA) (Hoffer & Ailon, 2018; Hadsell et al., 2006; Wu et al., 2018; Sohn, 2016; Hadsell et al., 2006; Kim et al., 2020; Wang et al., 2020a; Deng et al., 2019; Wu et al., 2018; Schroff et al., 2015). See Supplementary for more details.
Fairness Evaluation In the embedding space, we analyze fairness via performance gaps between minoritized groups and majoritized groups, or worst-group performance gaps (Lahoti et al., 2020). For fairness of the feature representations, we compute gaps in three metrics: recall@k and NMI for intra- and inter-class distance (Section 3.2), and the uniformity measure UKL corresponding to Definition 3 (defined in Section 3.1).
Training and Evaluation on Downstream Classifiers To link fairness performance in the embedding space to downstream classification (in which more extensive prior work has been completed), we train downstream classifiers and evaluate classification bias. After training the DML model with the aforementioned criteria, the network is fixed. The output embeddings from the image training datasets, in addition to the class labels, are used to train four downstream classification models: logistic regression (LR), support vector machine (SVM), K-Means (KM), and random forest (RF) (Pedregosa et al., 2011). In the manually imbalanced upstream setting, we train downstream classifiers on the original balanced image datasets to ascertain if bias incurred in the embedding can propagate downstream even if the downstream classifier is trained with real balanced data.
We execute class imbalanced experiments for CARS196 and CUB200 and vary the level of imbalance between minoritized and majoritized classes in the upstream training set.
PARADE Configuration We test Partial Attribute De-correlation, PARADE, by training models in the listed settings: manually color imbalanced dataset for CUB200, CelebA and LFW. The attribute used to train the sensitive attribute embedding for each dataset, and the attribute used for fairness evaluation. We compare PARADE with margin loss and distance-weighted sampling (Wu et al., 2018) to standard margin loss and distance-weighted sampling.

heading: 6 RESULTS
text: 6.1 SOTA DML METHODS HAVE LARGE FAIRNESS GAPS IN finDML BENCHMARK
Our experiments indicate that current DML methods encounter crucial fairness limitations in the presence of imbalanced training data. Table 1 (along with a corresponding table for CARS196 in the Supplemental) demonstrate that gaps in the manually class imbalanced setting are greater than the balanced control setting. In four combinations of loss functions and sampling strategies, we do not observe a scenario in which the class imbalanced setting achieves a smaller gap than the control in the embedding space, nor the downstream classification. This is particularly significant due to the nature of sampling strategies studied (Wu et al., 2018; Schroff et al., 2015), which batch samples to force the model to correct “hard" examples. The results validate finDML as a benchmark and framework for fairness through the lens of well-studied fairness characterization in classification.
Interestingly, Table 1 displays non-negligible gaps in downstream performance metrics recall and precision even in the balanced control case. This could represent stenography of underlying structures in the data, such as car color or bird size. More likely, however, these gaps are due to use of macroaveraging in recall and precision calculations. Nonetheless, the manually class imbalanced settings consistently produce larger gaps.

heading: 6.2 PROPAGATION OF BIAS TO DOWNSTREAM TASKS
text: The tabular results emphasize a significant result: naive re-balancing with real data downstream cannot overcome bias incurred in the upstream embedding in any setting studied. Indeed, Table 1 exhibits propagation of bias from upstream embeddings (trained on imbalanced data) to downstream tasks (trained on fixed upstream embeddings with a re-balanced dataset). To provide additional context for the result, we direct to increasing use of DML models as components of larger classification models. This trend is arising in literature such as supervised contrastive learning, and recent developments in pre-training and lifting DML models for classification (Khosla et al., 2020). This necessitates tackling bias in the representation space of DML as opposed to patches downstream, and emphasizes the importance of defining fairness in this setting as done in our work.
Impact of imbalance degree on lack of fairness Figure 3 shows that gaps in downstream classification mimic those upstream, even as we vary the level of imbalance introduced when training the upstream embedding. Here, the random forest classifier sees greater gaps in downstream metrics
than the control, even when manual imbalance is set at 40 − 60 upstream, and the downstream training dataset is balanced. For results with additional downstream classifiers, see Supplemental. This experiment demonstrates that the propagation of bias to downstream will occur even with lower levels of imbalance, and does not appear to depend on the downstream classifier chosen.

heading: 6.3 REDUCED SUBGROUP GAPS THROUGH PARTIAL DE-CORRELATION WITH SENSITIVE ATTRIBUTE
text: Table 2a shows results for performance gaps between relevant subgroups in both facial recognition datasets. PARADE shows strong results for CUB200 bird color dataset, primarily reducing gaps downstream and accordingly to recall@1 (Definition 1). PARADE can reliably reduce gaps for both the representation space and downstream classifiers on LFW. Interestingly, we observe that the majoritized subgroup (“White") had worst performance of all “Race" subgroups (see Supplemental), contrary to previous results (Samadi et al., 2018).3 As such, we measure gaps between the worstperforming subgroup and others.
2Due to the great number of singleton classes in LFW, recall@1 is discarded as a metric. 3Note: minoritized subgroups can still encounter notable bias across other axes more difficult to mea-
sure (Radford & Espenshade, 2014).
For CelebA, we find for standard methods the minoritized subgroups to generally perform worst. PARADE excels at gap reduction upstream but encounters larger subgroup gaps downstream compared to standard methods. While PARADE does reduce downstream gaps between light skintones (I, II, and III), and the two lighter dark skintones (IV, V), gaps increase between lighter skintones and the darkest skintone (VI) (see Supplemental). Because skintone VI constitutes < 1% of the CelebA dataset, PARADE is likely not able to learn similarity between faces over attributes besides skintone. And PARADE is prevented from learning similarities based on skintone due to de-correlation. In such settings, PARADE could be combined with oversampling minoritized subgroups to ensure better performance.
In general, the results show promising benefits of PARADE to adequately address and improve on the challenge of subgroup gaps for DML models used in facial recognition; and in the standard DML dataset CUB200, for recall@1 upstream (Definition 1) and across metrics downstream (Table 2b).

heading: 7 DISCUSSION
text: In this work, we introduce the finDML benchmark, a framework for fairness in deep metric learning (§3.2). We demonstrate the fairness limitations of established DML techniques, and the surprising propagation of embedding space bias to downstream classifiers. Importantly, we find that this bias cannot be addressed at the level of downstream classifiers but instead needs to be addressed at the DML stage. We investigate the limit of this propagation in manually introduced imbalance, and finally show that PARADE can reduce subgroup gaps in several settings.
Limitations PARADE suffers from pitfalls similar to other “fairness with awareness" methods: PARADE uses information only on pre-defined sensitive attributes and therefore can be unfair w.r.t. other sensitive attributes. PARADE does have an advantage in addressing the combinatorial number of attributes considered in multi-attribute fairness through DML, which will scale sub-combinatorially in time/space complexity. We also note that subgroup gaps are not sufficient to capturing societal understandings of fairness, and there is no consensus as to how to remedy such gaps (Chouldechova & Roth, 2018; Dwork et al., 2012; Hardt et al., 2016; Zemel et al., 2013; Zafar et al., 2017). Additionally, while PARADE intentionally optimizes Definitions 1 and 2, we provide no explicit guarantee and optimization of uniformity, Definition 3, remains an open problem. Finally, PARADE does incur slight decrease in overall performance, similar to other methods (Wick et al., 2019) (see Supplemental for per-subgroup performance and additional fairness-utility trade-off analysis for PARADE).
Code of Ethics Statement The work presented here deals with fairness in deep metric learning. A portion of our studies in the paper focus on CARS196 and CUB200-2011 datasets, which have
consistently been used in benchmarking novel DML frameworks (Krause et al., 2013; Wah et al., 2011). The fairness analysis considered for CUB200-2011 deals with bird color, which does not, to our knowledge, correspond with any societal problems relating to fairness. Nonetheless, as CUB2002011 is used in a litany of papers for SOTA performance comparison, finDML includes CUB200 so that DML methods can be analyzed w.r.t. fairness on a dataset used in their original paper.
We do include facial recognition datasets and tasks and analyze fairness with respect to facial attributes. Facial recognition does raise ethical concerns in practice. We note that our paper attempts to address primary social concerns in facial identity recognition. We do not encourage the task of facial attribute recognition, and solely use labeled attributes that correspond to known axes of bias for fairness analysis (e.g. Race and Skintone). As PARADE has solely been tested in two widely used public facial recognition datasets, we cannot guarantee fairness nor privacy in practical settings with private facial datasets.
Reproducibility Statement Additional experimental results discussed in the main paper and others are contained in Supplemental C. Implementation details including attribute information, generation of attributes, training parameters, metric calculation and gap computation are listed in Supplemental D. In our zipped supplemental materials file, we include all code used to generate the experiments. Individual scripts for generation of each experiment are contained in the “experiments" directory of the code file.

heading: SUPPLEMENTAL MATERIAL

heading: A ADDITIONAL BACKGROUND

heading: A.1 DEEP METRIC LEARNING DEFINITIONS
text: Here, we iterate through some common DML criteria and batch mining strategies more formally than in the main paper. Throughout this section, let X denote the input data, ϕ(X) the embedded data, Y the class label, and let Y (x) denote the value of the ground truth class label for data instance x. Denote the set of all positive pairs with respect to class label Y as P = {(x1, x2) ∈ X × X : Y (x1) = Y (x2), x1 ̸= x2}. Denote the set of all negative pairs with respect to class label Y as N = {(x1, x2) ∈ X ×X : Y (x1) ̸= Y (x2)}. We use the notation (xa, xp, xn) ∈ X ×X ×X to denote a triplet with an anchor sample xa, positive sample xp where Y (xa) = Y (xp), and negative sample xn where Y (xa) ̸= Y (xn).
Batch Sampling and Mining The batch sampling procedure in deep metric learning methods differ from that of generic deep classifiers in that canonical loss functions require tuples or pairs of samples in order to utilise ranking objectives as training surrogates to learn an appropriate similarity metric. To ensure that tuples with positive and negative examples can be extracted from the batch, the Samples-Per-Class-n (SPC-n) heuristic (see e.g. Roth et al. (2020c)) is generally used, where commonly n = 2, 4, 8. Given a batch size b, the SPC-n technique randomly selects b/n classes from which n training samples are then drawn randomly to be included in each batch B. After feeding the batch through the network, tuples are mined from the batch to use in the loss function. We refer to mining in this paper either as batch mining or overload the both as batch sampling terminology. The naive solution to tuple mining is random mining, in which all possible tuples of the form (xa, xp, xn) are considered and b are randomly chosen from the batch. However, this method lacks the capacity to utilize valuable information about the current embedding space, and is prone to significant redundancy in the training signal Schroff et al. (2015); Wu et al. (2018). Definition 4 (Random Mining). Hu et al. (2014) For each xa ∈ B, we randomly draw a positive example from {xp ∈ B : Y (xp) = Y (xa), xp ̸= xa} and a negative example from {xn ∈ B : Y (xn) ̸= Y (xa)} to form the triplet (xa, xp, xn).
Intuitively, this could be mitigated by hard mining heuristics searching for negative samples that are closer to the anchor sample in the embedding space than positive samples, thereby always ensuring a significant training signal. Unfortunately, such approaches are prone to heavy overfitting, training instability and large gradient variance, thereby commonly resulting in less-than-optimal solutions (see e.g. Schroff et al. (2015); Harwood et al. (2017); Wu et al. (2018)). Recent approaches thus establish more lenient heuristics, such as through the introduction of slack parameters to the hard mining objective (e.g. semi-hard mining Schroff et al. (2015) or softhard mining Roth & Brattoli (2019)). Definition 5 (Semi-hard Mining). For each xa ∈ B, we randomly draw a positive example from {xp ∈ B : Y (xp) = Y (xa), xp ̸= xa}, and a negative example from the set
{xn ∈ B : Y (xn) ̸= Y (xa), ∥ϕ(xa)− ϕ(xn)∥22∥ϕ(xa)− ϕ(xp) + γ∥22} where γ ∈ R is a slack parameter, to form the triplet (xa, xp, xn).
While other adaptive means (e.g. Harwood et al. (2017); Roth et al. (2020a)) have shown strong performance improvements, modern predefined heuristics such as distance-weighted tuple mining Wu et al. (2018) offer a better cost-to-performance tradeoff Roth et al. (2020a). Here, the heuristic leverages the fact that embeddings are commonly normalized to have unit L2 norm for regularization purposes Wu et al. (2018). This ensures a distribution over a unit hypersphere, in which explicit pairwise distributions can be established Weisstein (2002); Wu et al. (2018). By inverting this distribution, distance-weighted mining can thus encourage a much more diverse coverage of tuple difficulties, improving generalization performance and reducing gradient variance Wu et al. (2018).
Definition 6 (Distance-weighted). For embedding spaces normalized to the (D − 1)-dimensional hypersphere SD−1, we haveWeisstein (2002); Wu et al. (2018) the following pairwise sampling distribution q(•, •):
q (d (ϕ(xi), ϕ(xj))) ∝ d (ϕ(xi), ϕ(xj))D−2 [ 1− 1
4 d (ϕ(xi), ϕ(xj))
]D−3 2
for embedding pairs (ϕ(xi), ϕ(xj)) ∈ SD−1 and Euclidean distance d(•, •). For each xa ∈ B, we randomly draw a positive example from {xp ∈ B : Y (xp) = Y (xa), xp ̸= xa}, and sample a negative example based on an inverse distance distribution w.r.t. q:
P (xn|xa) ∝ min(λ, q−1(d(ϕ(xi), ϕ(xj))))
where λ ∈ R defines a clipping parameter to avoid potentially erroneous training samples.
Examined Objectives The primary goal of DML loss functions is to provide a training surrogate that implicitly optimizes for desired metric space quantities by narrowing down the expected distance between positive pairs of samples and expanding on the expected distance between negative pairs of samples in the embedding space. Most commonly employed pair Hadsell et al. (2006) and tripledbased Schroff et al. (2015); Hoffer & Ailon (2018) ranking losses penalize close negative pairs and disparate positive pairs up to a predefined margin to avoid overclustering. Using P(x) to denote all positive pairs containing x
P(x) = {(x1, x2) ∈ P : x1 = x}
and N(x) to denote all negative pairs containing x
N(x) = {(x1, x2) ∈ N : x1 = x}
we define
Definition 7 (Contrastive). Hadsell et al. (2006) Given a batch B, and pairs of samples S over B × B, the contrastive objective is defined as:
Lcontr = 1
b ∑ (xi,xj)∈S IY (xi)=Y (xj)d (ϕ(xi), ϕ(xj)) + IY (xi) ̸=Y (xj) [γ − d (ϕ(xi), ϕ(xj))]+
with margin γ.
Definition 8 (Triplet). Hoffer & Ailon (2018) The triplet loss extends the contrastive objective with sample triplets and can be defined as:
Ltripl = 1
b
∑ (xa,xp,xn)∈T
Y (xa)=Y (xp) ̸=Y (xn)
[d (ϕ(xa), ϕ(xp))− d (ϕ(xa), ϕ(xn)) + γ]+
with margin γ.
Margin loss extends the triplet objective through the inclusion of a learnable boundary β between positive and negative pairs Wu et al. (2018). In our experiments, we utilise β = 1.2. These criteria are widely used (see e.g. Roth et al. (2020c); Musgrave et al. (2020)) and require mining to make use of the batch information.
Definition 9 (Margin). Wu et al. (2018) The margin objective integrates the learnable distance boundary β between positive and negative pairs of samples for a relative ordering of pairs with respect to β as
Lmargin = ∑
(xi,xj)∈S
γ + IY (xi)=Y (xj) (d (ϕ(xi), ϕ(xj))− β)− IY (xi) ̸=Y (xj) (d (ϕ(xi), ϕ(xj))− β)
Going beyond pairs and triplets, one can also consider the case of more general n-tuples, which was investigated e.g. in the N-Pair objective Sohn (2016) and the Multisimilarity loss Wang et al. (2020a).
Definition 10 (N-Pair). Sohn (2016) N-Pair loss is a simple augmentation of the triplet framework in which all negatives in the batch B are incorporated in the objective function as:
Lnpair = 1
b
∑ (xa,xp)∈B
Y (xa)=Y (xp),a ̸=p
log 1 + ∑ xn∈B
Y (xa )̸=Y (xn)
exp ( ϕ(xa) ∗,Tϕ(xn)− ϕ(xa)∗,Tϕ(xp)∗ )+
ν b · ∑ i∈B ∥ϕ(xi)∗∥22 (6)
where ν denotes an embedding regularization parameter due to slow convergence for normalized embeddings stated in Sohn (2016)
Definition 11 (Multisimilarity). Wang et al. (2020a) Multisimilarity loss fits into the ranking loss category, but in addition to evaluation of cosine similarity between positive-anchor pairs and negativeanchor pairs, the objective evaluates positive-positive and negative-negative pairs with respect to the anchor:
s∗c(xi, xj) =  sc (ϕ(xi), ϕ(xj)) sc (ϕ(xi), ϕ(xj)) > minxj∈P(xi) sc (ϕ(xi), ϕ(xj))− ϵsc (ϕ(xi), ϕ(xj)) sc (ϕ(xi), ϕ(xj)) < maxxk∈N(xi) sc (ϕ(xi), ϕ(xk)) + ϵ0 otherwise Lmultisim = 1
b ∑ xi∈B 1 α log 1 + ∑ xj∈P(xi) exp (−α (s∗c (ϕ(xi), ϕ(xj))− λ)) + 1
β log 1 + ∑ k∈N(xi) exp (β (s∗c (ϕ(xi), ϕ(xk))− λ))  (7) where cosine similarity sc(x, y) = xT y for two normalized vectors x, y ∈ X .
Notably, the Multisimilarity loss employs a masking process as a stand-in for the lack of batchmining heuristic. While this proves to be similarly successfull in addressing the tuple sampling complexity issue, this can also be addressed through the usage of proxy-samples. These are dummy variables that represent various contextual properties (such as mean class representations) to serve as standing for actual samples, which is found e.g. in the ArcFace Deng et al. (2019) or ProxyNCA loss Movshovitz-Attias et al. (2017). Definition 12 (Proxy-NCA). Kim et al. (2020) ProxyNCA learns class proxies, or class centers, which each represent a class in the set of unique classes Y . Then, each anchor from the batch is sampled and a positive or negative proxy ψc ∈ Rd per class c ∈ Y is introduced in lieu of a positive or negative sample, respectively, giving:
Lproxy = − 1
b ∑ xi∈B log
( exp ( −d ( ϕ(xi), ψY (xi) )∑ c∈Y\{Y (xi)} exp (−d (ϕ(xi), ψc) ) Definition 13 (Arcface). Deng et al. (2019) Arcface combines proxy and angular loss methods (e.g. in Wang et al. (2017)) to enforce an angular margin between the embeddings ϕ and a proxy (or approximate center) W ∈ Rc×d for each class, giving the following:
Larc = − 1
b ∑ xi∈B log exp
( s · cos ( WTY (xi)ϕ(xi) + γ = 0.5 )) exp ( s · cos ( WTY (xi)ϕ(xi) + γ = 0.5 )) + ∑
xj∈B Y (xi) ̸=Y (xj)
exp ( s · cos ( WTY (xj)ϕ(xi) )) where the angular component is encoded in additive angular margin penalty γ, and s is a scaling parameter, which denotes the radius of the effective utilized hypersphere S.
Standard Performance Metrics Performance metrics in deep metric learning aim to capture the quality of the similarity metric learned by the deep embedding model. Therefore, standard performance metrics in DML reflect the closeness between samples of the same class, the separability
of samples of different classes, the clustering quality of embedding, and the uniformity over the hypersphere embedding space, which has been linked to zero-shot generalization capability Wang & Isola (2020), as discussed in Section 2. In our experiments, we utilize recall@1 Jegou et al. (2011), normalized mutual information Manning et al. (2010) between cluster labels assigned by the well-known K-Means Lloyd (1982) algorithm and ground-truth class labels, and UKL to measure the closeness between samples of same class, cluster quality of the embedding (and hence, the separability of distinct classes) and uniformity, respectively. Here, we define these metrics formally, but we note that there exist multitudinous performance metrics for DML that we do not define here or use explicitly for our results, including f1 score, mean average precision (mAP), and recall@k for k > 1 Jegou et al. (2011). Definition 14 (Recall@k). Jegou et al. (2011) Given k ∈ {1, . . . , |X|}, denote NNk as defined in Definition 1. Then, Recall@k is measured as:
Recall@k = 1 |X| ∑ x∈X { 1 ∃x̃ ∈ NNk(x) : Y (x̃) = Y (x) 0 else
Definition 15 (Normalized Mutual Information Score on Clusters). Manning et al. (2010) Let C a clustering algorithm, such as K-Means Lloyd (1982) with the number of clusters set to |Y |, such that C(x) indicates the cluster label for data point x ∈ X . The normalized mutual information score between the target labels Y and the cluster labels C is measured as:
NMI = 2 · I(Y (X);C(X))
H(Y (X)) +H(C(X))
where for random variables X,Y , I(·, ·) denotes the mutual information function: I(X;Y ) = H(Y )−H(Y |X)
and H(·) denotes the entropy function: H(X) = − ∑ x∈X Pr(x) log(Pr(x))
The performance metric UKL, used to measure feature uniformity for our empirical evaluations, is defined in Section 3.1.

heading: A.2 CLASSIFICATION FAIRNESS DEFINITIONS
text: Fairness definitions and criteria in classification are briefly mentioned in Section 2 of the main paper. Here, we provide explicit formulas for the most common fairness definitions, including demographic parity, equalized odds, and equality of opportunity Hardt et al. (2016), and provide some additional context on fairness definition evolution. Definition 16 (Demographic Parity). The predictor Ŷ satisfies demographic parity with respect to attribute A and class Y if the predictor is independent of A:
Pr[Ŷ = 1|A = a] = Pr[Ŷ = 1|A = b] ∀a, b ∈ A
Specifically, demographic parity has largely been used over the years as a simple and intuitive definition of fairness, in which a classifier is said to satisfy demographic parity if the sensitive attribute is independent of the output of the classifier. While demographic parity provides a simple fairness definition, the measure cannot capture fairness in classification tasks where the ground-truth label is inherently related to a certain attribute value Li et al. (2017).
Definition 17 (Equalized Odds). The predictor Ŷ satisfies demographic parity with respect to attribute A and class Y if the predictor is independent of A conditional on Y :
Pr[Ŷ = 1|A = a, Y = y] = Pr[Ŷ = 1|A = b, Y = y] ∀a, b ∈ A,∀y ∈ {0, 1} from Hardt et al. (2016).
Definition 18 (Equality of Opportunity). The predictor Ŷ satisfies demographic parity with respect to attribute A and class Y if the predictor is independent of A conditional on positively labelled Y :
Pr[Ŷ = 1|A = a, Y = 1] = Pr[Ŷ = 1|A = b, Y = 1] ∀a, b ∈ A from Hardt et al. (2016).
This lead to the introduction of other fairness definitions that capture such nuances, the most wellknown of which are probably equalized odds and equality of opportunity Hardt et al. (2016). However, fairness metrics overall have been criticized due to the choice of protected attribute over which to measure, and the inability of these metrics to capture bias with respect to certain attributes which are not known at test-time. We discuss this to a limited extent in Section 7.

heading: B DATASET SUMMARY STATISTICS

heading: C ADDITIONAL RESULTS
text: C.1 CARS196
Additional results for all loss and batch mining strategies for the manually class imbalanced experiments and balanced controls for CARS196 are located in Tables 6 and 7. K-Means was also
tested as a downstream classifier but showed poor performance. The impact of varying imbalance in the manually class imbalanced CARS196 experiments with all tested downstream classifiers is displayed in Table 8. Additional results for benchmarking of further fairness improvement methods in downstream classification of "imbalanced" embeddings (aside from naive use of balanced datasets) are shown in Table 8.
C.2 CUB200
Additional results for all loss and batch mining strategies for the manually class imbalanced experiments and balanced controls for CUB200 are located in Tables 9 and 10. K-Means was also tested as a downstream classifier but showed poor performance. The impact of varying imbalance in the manually class imbalanced experiments in the upstream embedding, and all tested downstream classifiers is displayed in Table 9. Additional results for benchmarking of further fairness improvement methods

heading: Overall

heading: Gap

heading: Overall

heading: Gap
text: in downstream classification of "imbalanced" embeddings (aside from naive use of balanced datasets) are shown in Table 11. Benchmarking of fairness improvement methods in downstream classification for bird color are shown in Table 12. Per-subgroup and overall results for CUB200 color experiments with standard margin-distance and PARADE are displayed in Table 13.

heading: C.3 CELEBA
text: Additional results for all loss and batch mining strategies for the CelebA dataset are located in Tables 14 and 15. Additional PARADE results for subgroup gaps excluding Fitzpatrick Skintone VI (as mentioned in Section 6.3) are located in Table 16.

heading: C.4 LFW
text: Additional results for all loss and batch mining strategies for the LFW dataset are located in Tables 18 and 19. Per-subgroup results for LFW to demonstrate worst-group performance for the “White" subgroup (as mentioned in Section 6.3) are located in Table 20. Benchmarking of fairness improvement methods in downstream classification for bird color are shown in Table 21.

heading: Overall

heading: Gap

heading: Overall

heading: Gap

heading: C.5 EXPLORATION OF FAIRNESS - UTILITY TRADEOFF AND VARYING HYPERPARAMETERS IN
text: We vary αSA and ρ in the PARADE objective to explore the relationship between the overall performance, subgroup gap, and worst-group performance in PARADE. As stated in the main paper, we optimize αSA and ρ via worst-group performance. Results of this analysis are displayed in Figure 12. We use our exploration to expound on how to optimize for αSA and ρ. As seen in Figure 12, a clear trend that inversely relates overall performance, and fairness as measured by subgroup gap and worst-group performance is seen for the uniformity metric, UKL over the grid of αSA and ρ values (Note that higher values of UKL correspond to worse performance). Recall@1 and NMI demonstrate noisier relationships between overall performance and fairness; and several αSA, ρ choices appear to select an optimal tradeoff. In Figure 12, for Recall@1, we observe that at the location αSA = 0.1, ρ = 500. in the optimization grid, PARADE reaches peak overall performance and fairness (measured by low subgroup gap and high performance for the worst-performing subgroup) simultaneously. Thus, we could conclude that this choice of αSA and ρ represents an optimal tradeoff for utility and fairness
in PARADE as measured by Recall@1. By the other displayed metrics, we see that αSA = 0.1, ρ = 500. demonstrates a reasonable utility-fairness tradeoff. Therefore, the choice of αSA = 0.1, ρ = 500. would be optimal for PARADE in CUB200 bird color setting. Note that the choice of where to operate within this trade-off should depend on the application that is being targeted. For
example, here we use Recall@1 to determine the optimal choice of hyperparameters and validate with the other two considered metrics. However, for LFW, which has a high population of singleton classes (see Figure 7), NMI would be a better metric to use for selecting optimal point.
D IMPLEMENTATION DETAILS

heading: D.1 DATASET ATTRIBUTE INFORMATION
text: Dataset manipulation for the CARS196 and CUB200 manually class imbalanced experimentsis explained in Section 3.3.
For CUB200 bird color experiments, we utilized the labeled bird color attributes from Wah et al. (2011). Each image can have multiple “primary color" labels. Therefore, we take the mode over
all bird colors labeled for each image in order to determine a single bird color associated with the image. For CelebA, we calculate the Fitzpatrick skintone based on the image pixel information for each image. The calculation is described in Section D.2. For LFW, we construct the “Race" attribute from labels of “White", “Black", “Asian," and “Indian" as labelled by Kumar et al. (2009). For each of these attributes, the labelling provided by Kumar et al. (2009) has a float value, which we map to binary values: the image is considered to have the attribute if the value is greater than 0, and the image is considered to not have the attribute if the value is less than 0. Naturally, the labelling is not necessarily correct for each image, as the confidence about the “Race" labelling can be quite low for some images. We remove all images without at least one of these attributes, though we note that these attributes do not encompass all races. Therefore, our analysis may not be relevant for other races not labelled by Kumar et al. (2009).

heading: D.2 FITZPATRICK SKINTONE CALCULATION
text: We follow the methods from Cheng et al. (2021) for calculation of Fitzpatrick Skintone based on image pixel information. However, we calculate these values for CelebA, as opposed to CelebAHQ. As CelebA-HQ incorporates higher resolution images, but has fewer images, our process of Fitzpatrick Skintone calculation on CelebA is slightly modified to account for lower resolution, and differing image size.
In Cheng et al. (2021), two sample skin patches are selected from each image of CelebA-HQ to determine the skintone. We select three sample skin patches, as we are forced to reduce the dimensions of the patches to account for the smaller image size of CelebA. Additionally, we leverage facial landmark attributes provided by CelebA Liu et al. (2015) in order to choose our sample patches. Specifically, given the (x, y) landmarks for the left eye, right eye, and nose for each image, we choose to sample square patches of size 20× 20 (all 3 color channels are selected) with the following center points:
(xleft eye, ynose)
(xright eye, ynose)
(xnose, ynose)
The first two center points are intended to capture the likely location of the left and right cheeks, respectively, as these are likely located below each eye and adjacent to the nose. The last center point is the nose. We note that this protected attribute generation is not perfect. In some cases, such label generation can accidentally use aspects of the background, if, the individual’s face position in the image is not facing forward. Also, extreme lighting can lead to misclassification of skintone. Nonetheless, we believe the procedure provides a good approximation of Fitzpatrick skintone category, but do not recommend these attribute labels for use outside of fairness analysis.
The selected sample patches are converted to CIELab-space to retrieve the L (luminance) and b (yellow) values. We then calculate the Mean Individual Typology Angle (ITA) value:
ITA = arctan ( L− 50 b ) × 180 ◦ π
Based on the Mean ITA calculation, we classify each image into one of the 6 Fitzpatrick skintone categories, as listed in Table 23. To calculate subgroup gaps, we calculate gaps between the mean value over the 3 lightest Fitzpatrick skintones and the mean value over the 3 darkest Fitzpatrick skintones.

heading: D.3 TRAINING PARAMETERS
text: For CUB200 and CARS196, we did not perform hyperparameter search but followed reported hyperparameters from Roth et al. (2020c) for best performance with an ImageNet Deng et al. (2009) pretrained ResNet50 He et al. (2016) and frozen batch normalization layers. As detailed in Roth et al. (2020c), we train for 150 epochs with embedding dimension 128, learning rate 0.00001 with no scheduler, and weight decay 0.0004. We train with a batch size of 128, with the Adam optimizer Kingma & Ba (2015) over five seeds inclusive for the balance control datasets, and for CUB200 color experiments; and seeds 0 − 9 for the manually class imbalanced experiments. For training transforms, we normalize each image using color channel means (0.485, 0.456, 0.406) and standard deviations (0.229, 0.224, 0.225), randomly crop the image and re-size to 224 × 224 and horizontally flip with probability 0.5. For testing transforms, we normalize each image with the aforementioned color channel means and standard deviations, resize to 256× 256, and center crop to 224× 224. For CelebA and LFW, we performed hyperparameter search over the following hyperparameters: architectures: ResNet50 He et al. (2016), and SE-Net50 (both with and without frozen batch normalization layers); number of training epochs; learning rates; last linear layer learning rate (differ from other layer learning rates); learning rate schedulers; embedding dimensions: 64, 128, 256; pre-training; image augmentations. We evaluated hyperparameter sets on a validation set we cut from the typical training set (20% of training set), and chose the set of hyperparameters with best recall@k score for CelebA and best NMI score for LFW. NMI is used for LFW due to the high number of singleton classes present in the dataset (recall@1 is meaningless for singleton classes).
For CelebA, we train on the ResNet50 He et al. (2016) architecture with frozen batch normalization layers, for 125 epochs with learning rate 0.00001, and no scheduler, weight decay 0.0004, Adam Kingma & Ba (2015) optimizer, and batch size of 128. For training transforms, we normalize each image using color channel means (0.5, 0.5, 0.5) and standard deviations (0.5, 0.5, 0.5), resize to 256× 256, center crop to 224× 224 and horizontally flip with probability 0.5. For testing transforms, we normalize each image with the aforementioned color channel means and standard deviations, resize to 256× 256, and center crop to 224× 224. We average over runs with seeds 0− 2, inclusive. For LFW, we train on the ResNet50 He et al. (2016) architecture with frozen batch normalization layers, for 125 epochs with initial learning rate 0.00001 for all model parameters except the last linear layer, which has initial learning rate 0.0001, and a multi-step learning rate scheduler which reduces the learning rate by a factor of 0.3 at epochs 50 and 100, weight decay 0.0004, Adam Kingma & Ba
(2015) optimizer, and batch size of 64. For training transforms, we normalize each image using color channel means (0.5, 0.5, 0.5) and standard deviations (0.5, 0.5, 0.5), resize to 256× 256, center crop to 224× 224 and horizontally flip with probability 0.5. For testing transforms, we normalize each image with the aforementioned color channel means and standard deviations, resize to 256× 256, and center crop to 224× 224. We average over runs with seeds 0− 2, inclusive. For each dataset we chose a set of loss and batch mining strategies that have historically been used for the relevant task, encompassing a broad range of methods, and / or achieved good performance. However, for n-pair loss and sampling, good performance was not achieved for the facial datasets despite use in the past for facial recognition Sohn (2016). For manually class imbalanced experiments with CARS196 and CUB200 and the associated balanced controls, we used: margin loss / distanceweighted sampling, margin loss / semi-hard sampling, triplet loss / distance-weighted sampling, triplet loss / semi-hard sampling, contrastive loss / distance-weighted sampling, multisimilarity loss, and proxy-NCA loss. For the color experiments with CUB200, we used: margin loss / distance-weighted sampling. For CelebA and LFW, we used: margin loss / distance-weighted sampling, arcface loss, and n-pair loss and sampling. For all testing and evaluation experiments with PARADE, we used margin loss and distance-weighted sampling, but PARADE can be used with any loss and mining strategy.
DML-specific parameters Here we list the hyperparameters that we use for each evaluated loss function and batch mining strategy, if applicable. Refer to A for explicit formulas associated with the parameters here. We set γ = 0.2 in semi-hard mining. For distance-weighted mining, we set λ = 0.5 and clip the maximum distance to 1.4. In the triplet objective, we use γ = 0.2 for triplet loss. For margin loss, the learning rate of the boundary β is set to 0.0005, with initial value 1.2 and triplet margin γ = 0.2. For N-Pair uses embedding regularization parameter ν = 0.005. In Multisimilarity loss, we use α = 2, β = 40, λ = 0.5 and ϵ = 0.1. Finally, for ArcFace, additive angular margin penalty is set to γ = 0.5, while scaling parameter s = 16 and class centers are optimized with learning rate 0.0005.
The two PARADE parameters, αSA and ρ, as described in Section 4, were optimized via worst-group performance over a grid search. For CUB200, we set αSA = 0.3, ρ = 1500. For CelebA, we set αSA = 0.1, ρ = 1000. For LFW, we set αSA = 0.3, ρ = 100.

heading: D.4 FAIRNESS EVALUATION
text: For each dataset, we calculate subgroup gaps between the majoritized and minoritized subgroup (CARS196, CUB200 class, CelebA) or between the worst-performing subgroup and other subgroups (LFW). In CUB200 color experiments, due to the large number of subgroups, we calculate the gap between the top 6 performing subgroups and the bottom 6 performing subgroups (there are 12 total subgroups).
Upstream In the upstream embedding tasks, in which we denote ϕ as the embedding function for the learned model, and use A(x) to denote the value of the attribute A for data point x, we calculate recall@1 for data samples in X with associated class label Y and attribute a ∈ A as:
Recall@k = 1 |{x ∈ X : A(x) = a}| ∑
{x∈X:A(x)=a}
{ 1 ∃x̃ ∈ NNk(x) : Y (x̃) = Y (x) 0 else
Note here that the nearest neighbors function is computed with respect to all x ∈ X , not exclusively x ∈ X with attribute a ∈ A, but the input to the nearest neighbors function is exclusively {x ∈ X : A(x) = a}. To calculate NMI, let C be the output of a clustering algorithm C on the entire dataset X , i.e. C = C(X) and let C|S denote the output of clustering algorithm C restricted to some subset S ⊂ X . The important note here is that the clustering algorithm is run over the entire dataset, but C|S expresses the cluster labels only for S ⊂ X . Then, we measure NMI for data samples in X with associated class label Y and attribute a ∈ A as:
NMI = I(Y ({x ∈ X : A(x) = a}); C|{x∈X:A(x)=a})
H(Y ({x ∈ X : A(x) = a})) +H(C|{x∈X:A(x)=a})
We calculate UKL for data samples in X with attribute a ∈ A as: UKL(X) = DKL ( UD,Sϕ({x∈X:A(x)=a}) )
where Sϕ({x∈X:A(x)=a}) denotes the singular values over ϕ({x ∈ X : A(x) = a}).
Downstream In the downstream tasks, for data samples in X with class label Y , and predictor Ŷ , let Y (x) express the value of the ground-truth label for data sample x and let Ŷ (x) express the value of the predicted label. Then, we denote TP (y)a the number of true positives with attribute a ∈ A:
TP (y)a = {x ∈ X : A(x) = a, Y (x) = y, Ŷ (x) = y}
FP (y) a the number of false positives with attribute a ∈ A
FP (y)a = {x ∈ X : A(x) = a, Y (x) = y, Ŷ (x) ̸= y}
and FN (y)a the number of false negatives with attribute a ∈ A:
FN (y)a = {x ∈ X : A(x) = a, Y (x) ̸= y, Ŷ (x) = y}
for y ∈ Y . We calculate macro-averaged recall for data samples in X with associated class label Y and attribute a ∈ A as:
Recall = 1 |Y | ∑ y∈Y
TP (y) a
TP (y) a + FN (y) a
where |Y | is the number of possible class labels, i.e. the size of the set of all possible values of Y . We calculate macro-averaged precision for data samples in X with associated class label Y and attribute a ∈ A as:
Precision = 1 |Y | ∑ y∈Y
TP (y) a
TP (y) a + FP (y) a
We calculate accuracy for data samples in X with associated class label Y and attribute a ∈ A as:
Accuracy = |{x ∈ X : A(x) = a, Y (x) = Ŷ (x)}|
|{x ∈ X : A(x) = a}|
The subgroup gaps are then considered to be the difference between the metric value for the majoritized subgroup and the metric value for the minoritized subgroup (CARS196, CUB200, CelebA); or between the metric value for each subgroup with better performance than the worst-performing subgroup and the metric value for the worst-performing subgroup (LFW). As stated in Section D.4, for CUB200 bird color experiments, the subgroup gaps were calculated between the top performing 50% of subgroups and bottom performing 50 of subgroups.

Review summaries:
This paper proposes three measures that evaluate the fairness of learned representations in multiple aspects. The authors empirically demonstrate that the existing metric learning approaches become less fair (i.e., shows larger performance gaps between attribute-based subgroups) when there is a class imbalance in the training data. Specifically, they propose a protocol that evaluates the fairness by manually adjusting the number of samples per class in the training set (finDML). In addition, they propose PARADE that de-correlates image embedding from auxiliary attibutes embedding via adversarial separation to improve fairness of the learned image embedding. In the experiments, the method shows overall improved fairness on five datasets.

The paper presents a study on the effect of training dml techniques on imbalanced data and show the negative impact of learned representations on the downstream tasks. The fairness is analyzed through 3 properties of the representation space; a) inter-class alignment, b) intra-class alignment, c) uniformity showing that the bias in the upstream task (dml) is propagated to the downstream classification tasks even when the data for training the classifier (downstream task) is balanced. To address this, an objective (PARADE) that de-correlates the discriminative features from the sensitive attributes are learned during training. The experiments are conducted on 4 benchmark datasets.

The paper investigates the fairness problem in the deep metric learning task, which is underexplored by the research community. The authors propose finDML to benchmark previous methods on multiple imbalanced datasets with three newly proposed metrics. The experimental results show that all previous deep metric learning methods have the fairness issue, i.e., larger performance gaps across different subgroups. The authors further propose PARADE to mitigate the biases, which has been shown to be effective by the experiments.