An Investigation of Representation and Allocation Harms in Contrastive Learning
Authors: Subha Maity
Mayank Agarwal
Mikhail Yurochkin
Yuekai Sun
Venue: ICLR 2024 poster (2024)
Keywords: Underrepresentation
Self-supervised learning
Contrastive learning
Primary area: societal considerations including fairness, safety, privacy
TL;DR: We demonstrate that contrastive learning tends to collapse representations of minority groups with certain majority groups.

Abstract: The effect of underrepresentation on the performance of minority groups is known to be a serious problem in supervised learning settings; however, it has been underexplored so far in the context of self-supervised learning (SSL). In this paper, we demonstrate that contrastive learning (CL), a popular variant of SSL, tends to collapse representations of minority groups with certain majority groups. We refer to this phenomenon as representation harm and demonstrate it on image and text datasets using the corresponding popular CL methods. Furthermore, our causal mediation analysis of allocation harm on a downstream classification task reveals that representation harm is partly responsible for it, thus emphasizing the importance of studying and mitigating representation harm. Finally, we provide a theoretical explanation for representation harm using a stochastic block model that leads to a representational neural collapse in a contrastive learning setting.

Published as a conference paper at ICLR 2024
AN INVESTIGATION OF REPRESENTATION AND ALLO- CATION HARMS IN CONTRASTIVE LEARNING
Subha Maity*
Mayank Agarwal IBM Research
Department of Statistics University of Michigan Ann Arbor, MI sma weriitt@ulii
MIT-IBM Watson Lab Cambridge, MA mayank agarwa iimm
edu edu
Mikhail Yurochkin
Yuekai Sun
IBM Research
Department of Statistics University of Michigan Ann Arbor, MI yueka @umi edu
MIT-IBM Watson Lab Cambridge, MA
mikhail schhill.yrodiie yurochking com
ABSTRACT
The effect of underrepresentation on the performance of minority groups is known be a serious problem in supervised learning settings; however, it has been underexplored so far in the context of self-supervised learning (SSL). In this paper, we demonstrate that contrastive learning (CL), a popular variant of SSL, tends to collapse representations of minority groups with certain majority groups. We refer to this phenomenon as representation harm and demonstrate on image and text datasets using the corresponding popular CL methods. Furthermore, causal mediation analysis allocation harm on downstream classification task reveals that representation harm partly responsible for it, thus emphasizing the importance of studying and mitigating representation harm. Finally, we provide a theoretical explanation for representation harm using stochastic block model that leads to representational neural collapse in contrastive learning setting.
INTRODUCTION
Most prior studies of algorithmic bias focus on allocation harms: objectionable demographic disparities in the allo- cation of resources (Angwin al., 2016; O' O'Neil, 2017; Dastin, 2018). They are widely studied because they are easy to measure (simply measure disparities resource allocation), and they be tied to common performance metrics in supervised learning (e.g. misclassification rates), For example, consider credit default risk assessment model that classifies potential borrowers as high-risk and low-risk. Any disparities in the misclassification rate be- tween borrowers from different demographic groups lead disparities in lending rates. This focus on allocation harms is reflected in the plethora of methods for mitigating such harms, including methods based on resampling and reweighting (He et al., 2008; Ando & Huang, 2017: Byrd & Lipton, 2019), enforcing invariance constraints (Agarwal et al., 2018), re-calibrating the logits (Tian et al., 2020).
automobile
ship
horse
truck
Figure 1: tSNE visualization of CL representations for balanced (left) and aut omobi le under-represented (right) CIFAR 10 data.
In contrast to the voluminous literature on allocative harms, representation harms have received less attention. This is because representation harms are harder to measure and their effects are
*Corresponding author. Accompanying codes can be found in http://gihaaccmmmmmiyyniccchh representation-harm.

---

Published as a conference paper at ICLR 2024
more diffuse and longer-term (O'Neil, 2017). For example, Google Photos mistakenly tags black people as gorillas (Barr, 2015). There are no resources that are being misallocated here; rather, the harm is inflicted by perpetuating historical biases against black people. Despite recent advances in representation learning (Bengio et al., 2013), there is a lack of research on algorithmic biases in this area, especially studies that look into representation harms they may cause. In this paper, we seek to fill this gap in the literature by investigating the effects of group underrepresentaaion in contrastive learning (CL) algorithms (Chen et al., 2020; Chen & He, 2021; He et al., 2020). CL is a popular instance of self-supervise learning (SSL), an approach for learning representations that can effectively leverage unlabeled samples improve (downstream) model performances (Wang et al., 2021; Babu et al., 2021). However, these datasets often exhibit inherent imbalances (De-Arteaga et al., 2019) and it is important to know whether training CL model on such datasets may detrimentally affect the quality of representations and cause downstream allocative and representation harms.
Here is a preview of the results of our controlled study of underrepresennation with the CIFARIO dataset. In Figure I we plot 2D t-SNE embeddings (Van der Maaten & Hinton, 2008) of CL representations of vehicles in the dataset. We see that underrepresentttionn of aut omobi ile images in CL training causes their representation to cluster with those of trucks. This is instance of stereotyping (Abbasi et al., 2019) because samples from an underrepresented group are lumped together with those of similar majority group; thus erasing the unique characteristies of the underrepresented. This is a form of representation harm (Crawford, 2017), which can lead to downstream allocation harms (e.g. missclassifications between aut omobi les and trucks). In this paper, we show that this mechanism of representation to downstream allocation harms is general. This complements prior works, which show that underrepresenttation leads allocation harms without identifying the underlying mechanism. The rest of the paper is organized as follows.
1. In Section 2 we empirically show that underrepresentaaion leads to representation harms, as the CL representations of underrepresented groups collapse to semantically similar groups.
In Section we develop simple model of CL on graphs that exhibits a collapse of (the learned representations of) underrepresented groups to (those of) semantically similar groups. This suggests that the representation harms we measured are intrinsic to CL representations.
3. In Section we show via a (causal) mediation analysis that some of the allocation harms from a linear head/probe (trained on top of the CL representations) can be attributed to the representation harms in the CL representations. Thus, in order to mitigate the allocation harms from ML models built on top of CL representations, we must mitigate the harms in the learned representations.
1.1 RELATED WORKS
The paradigm of self-supervised learning (SSL) allows the use of large-scale unsupervised datasets to train useful representations and has drawn significant attention in modern machine learning (ML) research (Misra & Maaten, 2020; Jing & Tian, 2020). It has found applications in a broad spectrum of areas, such computer vision (Chen et al., 2020; He et al., 2020), natural language processing (Fang et al., 2020; Liu et al., 2021a; Hsu et al., 2021), and many more. In this paper, we investigate the effect of underrepresentattion in contrastive learning (CL. (Chen et al., 2020; Chen & He, 2021; He et al., 2020), which is a popular variant for SSI,, that uses similar samples to learn representations. more extensive discussion of previous work can be found in the surveys Schiappa et al. (2023); Yu et al. (2023); Liu et al. (2022) for SSL and Le-Khac et al. (2020); Kumar et al. (2022) for CL. In this paper, we use SimCLR (Chen et al., 2020) and SimSiam (Chen & He, 2021) for CIFARIO dataset and SimCSE (Gao et al., 2021) for BIASBIOS dataset to learn CL representations.
Several works have theoretically studied the success of self-supervised learning (Arora al., 2019; HaoChen et al., 2021: Lee et al., 2020; Tian et al.. 2021; Tosh et al., 2021). Our theoretical analysis of CL. loss is partly motivated by Fang et al. (2021b), who showed that CL representations of a group collapse to single vector. This phenomenon is known as neural collapse, and it also occurs in supervised learning (Papyan et al., 2020; Zhu et al., 2021; Fang et al., 2021a;b; Hui et al., 2022; Han et al., 2021) and transfer learning (Galanti et al., 2021). Our analysis differs from Fang et al. (2021b) because they study neural collapse when the dataset is balanced, while we focus on imbalanced datasets. This is crucial for studying allocation and representation harms caused by underrepresemmation. We show that when the dataset is imbalanced, neural collapse still occurs, but the collapsed points no longer possess symmetric simplex structure. Instead, the relative positions of
2

---

Published as a conference paper at ICLR 2024
the collapsed points depend the degree of underrepresentaation in intricate manner; explore the resulting structure in Section 3.2.
Liu et al. (2021c) showed that under group imbalance, predictive models based on SSL representations cause less allocation harm than models trained a supervised end-to-cat manner. However, their work does preclude allocation harms by SSL. We build their work by studying the allocation and representation harms caused by SSL.
Previous work mitigating representation harm is discussed in Section 5.
UNDER INDER-REPRESENTATION CAUSES STEREOTYPING
In Figure 1 we observe that underrepresentation of automobiles les causes its CL representations to collapse with t trucks. To delve deeper into this phenomenon, in this section, we consider two cases of underreenesentatiio:: (i) controlled study with CIFARIO dataset, where we emulate the effect of underrepressetation by systemically subsampling a class, and (ii) on BIASBIOS dataset, which is naturally imbalanced.
2.1
CONTROLLED STUDY OF CLASS UNDERREPRESENTATIIN
2.1.1
EXPERIMENTAL SETUP
airplane
-0 10011
377
+0 01
0.94
+0.01
0.9
+0
0.967
00.00
+0.07
0.973
+0.007
3.975
Dataset: For controlled study, we consider CIFAR 10 (Krizhevsky et al., 2009), a well-known visual benchmark dataset that has 10 classes (airplane, cars, etc.) and both the training and the test datasets are balanced (equal number of images all classes). To simulate underrepresentation, we randomly subsample 1% of the images for one of the classes when training our CL models. We denote this dataset as Dr. where & is the class that is undersampled. Furthermore, we train a CL model on the balanced dataset (Dnat) as a baseline.
ship
887
1:154
1.004
1.007
automobile
0.985
0.914
1.041
0.78
1.022
1.028
2:0 00.000
10.01
20.01
00.00
10.000
80.009
truck
0.96
0.916
1.001
0.001
912
deer
0. 995
1.0
1.00
1.005
1.6
20.00
0.000
20.000
200022
+0,009
horse-
0.99
1.001
1.01
0.893
0,852
1.000
airplane
ship automobile
truck
dees
hore
Models and representations: We the ResNet-34 back- bone and train on both balanced and underrepresented train-
Figure 2: Representation harms in CI- FARI6 10 over 10 repetitions.
ing datasets using two well-known CL approaches: SimCLR
(Chen et al., 2020) and SimSiam (Chen & He, 2021). We will refer to the model trained with a balanced (resp. underrepresented) dataset balanced (resp. underrepresented) model. In the main text, we present results for SimCLR. The results for SimSiam are deferred to Appendix A.2 and are quite similar to the SimCLR results. Further details and results of the SimCLR implementation can be found in Appendix A.
2.1.2 REPRESENTATION HARM
In Figure I we see that the underrepresemtation in automobile class causes its CL representations merge with those of t truck,, erasing the distinction between automobiles and trucks. We suspect similar phenomenon occurred when Google mistakenly tagged black people are Gorillas: the embeddings of black people merged with those of Gorillas, erasing the distinction between the two classes. Here, we further investigate this behavior, i.e., whether this phenomenon persists among other semantically similar pairs.
Metric:
We measure the representation harm between pair of classes (1, m) when the 1e-th class is underrepresented as
RH(I, m; k)
D(C.mffa). D(l. D(I, m: fa) mym...
Σ (e(n JE(mm
{1 cos (fe(F))) (f
(2.1)
where fea (resp. fk) is the balanced (resp. underrepresented) CL model trained on Doal (resp. D&). RH(1, m;;)) < implies that underrepresentation in class / causes its CL representation to collapse with class m. In Figure we plot the representation harm metrics, where the (l, m)-th entry is RH(1,m;;) We present RH metrics for the four vehical classes (a rpl rplane, shi p, automoble
3

---

Published as a conference paper at ICLR 2024
and truck), and two animal classes (deer and horse). The other RH metrics can be found in Appendix A.1.1.
Results: In Figure 2 observe the worst of representation harm between aut omobi i le and truck classes. The RH metric for this pair is the lowest (0.78 + 0.009) when aut omobi le is undersampled, and second lowest (0.832 = 0.012) when truck is undersampled. This means un- derrepresentation in either aut e or t ruuc results in a collapse in their CL representations with each other. Similar behavior is also observed between pairs (airpl ane, ships) (RH metrics are 0.887 + 0.008 and 0.94 + 0.01) and (deers, horses) (RH metrics are 0.852 + 0.008 and 0.953 + 0.009). Since (aut ootoooooss.....o. trucks) are vehicles on road, (ships, airplanes) are vehicles with blue (water/sky) backgrounds, and (deers, horses) are mammals with green backgrounds, these pairs are semantically similar pairs, which affirms our intuition that CL rep- resentations of an underrepresented class collapse with the representations of a similar class. We shall see later in Section 4.2 that these representation harms cause allocation harm, i.e. reduction in downstream classification accuracies.
2.2 EFFECTS OF
IN THE WILD
An important premise of self-super learning is the ability to train on large amounts of data from the Internet with little data curation. Such data will inevitably contain many underrepresented groups. In this experiment, we study the potential harms of CL applied to data obtained from Common Crawl, popular source of text data for self-supervised learning.
2.2.1 EXPERIMENTAL SETUP
Dataset: We consider BIASBIOS dataset (De-Arteaga et al., 2019) which consists of around 400k online biographies English extracted from the Common Crawl data. These biographies represent 28 occupations appearing at difffrrnnt frequencies (professor being the most common, and rapper the least common). In addition, many occupations are dominated by biographics of either male female gender (identified using the pronouns in the biographies), mimicking societal gender stereotypes. Please see Appendix B for additional details.
Model and representations: We obtain CL representations by fine-tuning BERT (Devlin al., 2018) with SimCSE loss (Gao et al., 2021), an analog of SimCLR for text data, that uses dropout in the representation space instead of image augmentations. We use random 75% of samples for training the SimCSE representations and the remaining data for computing all metrics reported in the experiments. SimCSE representations are trained using the source code from the authors in the unsupervised mode and with the MLP projectio layer (Gao et al., 2021).
2.2.2 REPRESENTATION HARM
attommy (f)
3
0.46
0.91
3.05
0.58
1.01
1.02
1.00
1.23
3.63
As discussed previously, we expect the representa- tions of underrepresented groups collapse towards the groups that are most similar to them. For exam- ple, consider the class surgeon which consists of 85% male and 15% female biographies. Will the bi-
paralegal
surgeon (F) 1,23
1.05
1.92
0:96
1.50
1.55
1.09
1.82
3.65
dentis
1.00
3.00
1.04
1.09
2.75
2.05
physicinn (M0 1.22
1.0
0:36
0.25
1.02
1.02
1.52
3.62
ographies of underrepresented female surgeons be assigned representations similar to male surgeons or to female biographies of different but related oc- cupation such as dent ist? Although none of these outcomes are desirable, the latter can cause represen- tation harms associated with gender stereotyping.
IPX
1:10
3.15
1:0
0.99
2.29
2.09
1:2
1.22
1:22
3.56
1.1
0.95
1.20
5.12
compeser 1..0
1.12
1.12
3.03
1.26
1:11
105
130
3.34
3.30
modessor
sanlgga
suppoon
dands-
ahyoltar
oeper
mode:
Metric: To measure the representation harm and
answer the aforementioned question we consider Figure 3: Gender RH in BIASBIOS dataset.
variant of the metric based on the average cosine distance used in the CIFARIO experiment. Let D(Ig, ) be the average cosine distance between the representations of gender g biographies from occupation and gender g' biographies from occupation m (analogous to equation 2.1 where we omit the dependency the model f since have single CL. model in this experiment). Let g
4

---

Published as a conference paper at ICLR 2024
be the underrepresented gender for occupation 1, we define gender representation harm (GRH) for occupations and m as:
GRH(L, m) D(l, D(ly)
(2.2)
GRH(I, m) implies that the learned representations for the underrepresented gender in occupation I are closer to representations of occupations m for the same gender than they are to the representations for the different gender within the same occupation. GRH(1, m) < 1 for I o m can be interpreted as a warning sign of gender stereotyping in representations.
We present the GRH results for a subset of occupations in Figure 3 (F and M in the row names indicate the underrepresented gender for the corresponding occupation; complete results are in Appendix B.3). We focus on the off-diagonal entries which should be >1 when there is no representa- tion harm. We observe several deviations from this rule, especially for occupation pairs (at (attorney, oreyy paralegal) and (surgeon, dent ist.. Specifically, suggests that representations of female attoreey are closer to representations of female paralegals than they are to that of male attorneys. The two occupations are highly related and such representation harm can result disadvantaging female attorneys. Analogously, GRH( surgeon, dent st) of 0.95 corresponds to similar problem for pair of occupations in the medical domain.
Results:
paraleçal of 0.94
3 ASYMPTOTIC ANALYSIS OF CL REPRESENTATIONS
In this section, show theoretically that the representation harms in contrastively learned represen- tations are generic and are not specific to certain datasets. Here, we focus on a generic contrastive learning loss (Chen al., 2020; Wang & Isola, 2020):
ming.a x LCL ( (xX)),
1 (V) - = E(EJ) Eje(n) e) log
explxT (xpt" s//+)
(3.1)
Eiem exp(s, mi/+)
where Sd-1 the unit sphere in Rd, the rows of (XX) E Raxd are the learned representations of the samples X,, and 222 IR is the (learedd representation of the i-th sample. Note that the loss depends on the raw inputs X2 's only through their similarities j). We shall exploit this fact to simplify the subsequent theoretical developments. In practice, t is usually parameterized as neural network.
To isolate the effects of the loss (from other inductive biases encoded in the architecture of & and the training algorithm), we make the ppere- assumption (Fang et al., 2021a;b) that & can produce any point in $d-1, This assumption simplifies the problem to optimizing directly on the outputs of (instead of the parameters of ④)
min,,eso-1 (V).
(3.2)
where the optimizers of corresponds to the learned representation of X,, Since the CL loss depends only the similarities between samples, directly impose a probabilistic model the similarities (instead of imposing assumptions on the distribution of samples). This simplifies the theoretical developments.
Stochastic block model: We formalize similarities between samples similarity graph on the samples and impose a stochastic block model (SBM) (Holland et al., 1983) on the similarity graph. Typically, an SBM with K blocks is described by two parameters: (1) II [T....., nx] the probabilities of data point or node belonging to each of the blocks, and (2) = [[ox,,>RR.. a matrix that describes connectivity between blocks, where 04, is the probability that node from block k is connected to a node from block k'. The ITy controls the size of k-th block and au.k's 'ss determines the similarity between blocks. A sample of size is drawn from SBM A) as:
1. We generate nodes from multinomial distribution with probability vector II. Let denote (Y, (r C [K] as the block annotations, i.e. the i-th node belongs to the Y,-th block. Note that Y; are i.i.d. (TTe)
2. For each pair of nodes, we generate e(i, E' Y,, Yv Beroulli(ay,, where e(i, i') indicates whether the nodes i and are connected by an undirected edge. In the context of CL from images, such edges imply that neighboring nodes and i' augmentations of the same similar images.
5

---

Published as a conference paper at ICLR 2024
The SBM has two desirable properties:
1. An inherent group structure that is often associated with downstream classification tasks. For example, the classes in CIFARI 10 dataset can be considered as groups.
2. A notion of connectivity among groups that explains which of them similar. As shall see later, this is a key factor in identifying the majority groups to which representations of an underrepresented group can collapse. Recall, in the example of CIFARIO (Section 4) we that the aut omobil le and ruck classes are connected to each other, and underrepresentation of either of them results in the collapse of their CL representations.
3.1 SIMULATIONS WITH THE STOCHASTIC BLOCK MODEL
To understand how underrepresentaaion may cause representation harm in CL representations, conduct a simple experiment on SBM data. We consider an SBM with three blocks. The connectivity probabilities are provided in the left plot of Figure 4, where only the first two groups connected with each other. Additional details are provided in Appendix D.
In experiment, underrepresent the first block by factors of 2-2 and 2-1 (the sam- ple size for the first group is reduced from 26 to 2 and 2²) and in- vestigate its represen- tation harm. For this purpose, we obtained the CL representations from equation 3.2. Note that the generic CL cost function is exactly the popular node2vec cost function (Grover & Leskovec, 2016) for graph representation learning, so our results also have implications in graph representation learning. In Figure we present the means and standard deviations for the cosines of the CL representations between groups. Some notable observations are described below.
connectivity (cu.))
29 [2
28
2F
0.95
0.4
+0.00
0.00
+0.03
0.57
+0.04
-087
+0.0
1.
077 +0.07
+3.03
-0.87
+0.0
1.0
-0.03
0.83
+0.00
-0 85
0.4
0.95
1:0.00
0.52
10.0
1.9
10.03
9.57
1.0
9.7
00.01
0.99
1000
0.98
1.0.0
0.33
00.0
1.0
1.000
1.0
1.0
1:0
10000
Figure 4: Left: connectivity across groups, middle-left, middle-right and right: average cosines and their standard deviations across groups.
The diagonals in Figure 4 demonstrate that all of vectors within block collapse into a single vector, they have cosines very close one. This phenomenon is known neural collapse (Papyan et al., 2020) and has been observed by Fang et al. (2021b) in analyzing CL representation with balanced data. Further related discussions are deferred Section 3.2.
The same plots suggest that the CL representations of the first two groups get closer when the first group is underrepresented, as seen their increased cosine similarity (it increases from 0.52(+0.08) to 0.7(+0.07) in the middle right and 0.83(=00.3)) in the right plot). This is form of representation harm, as the representations of the first two groups, which are similar to each other, partially collapse in terms of their CL representations (RH metric in equation 2. are RH(1, 2; 1) 0.48 and 0.35). Additionally, the second and third groups become further apart as their cosines decrease from - -0.87 to -0.99(+0.0) and 1++0000).
3.2 ASYMPTOTIC ANALYSIS OF CL REPRESENTATIONS FOR STOCHASTIC BLOCK MODELS
Here, we verify the two phenomena that we observe in our simulation with the SBM dataset: (1) the collapse of CL representations within block and (2) the representation harm between blocks with high connectivity. For this purpose, we perform an asymptotic analysis on the optimal CL representations o at n going to infinity. We require the following assumption in our analysis.
Assumption 3.1. Denoting the representation variables of the k-th block as Di where n4 = # (Y, =k and for each k we assume that 1/ns EØ=1 Uk.j converges as n&g + 0o.
The above assumption simply requires that the representation variables in the optimization of equa- tion 3.2 that belong to the same block do not fluctuate too much, which is a rather minimal assumption. With this assumption, state main result.

---

Published as a conference paper at ICLR 2024
Theorem 3.2. Assume that [VE(K Tyakke > 0 for each k. Then at - the optimum CL representations obtained from the minimization of problem in equation 3.2 satisfy the following:
2/ 4.
byp where a_ denotes almost sure equality and { [N([K] is minimizer for
-k Thi bg 18
min
Ebg= TKg (*hy (lgg-))
EE,= Thy log Ek
Tks Akal= }. (3.3)
Σx Zkg=1 Rkg R2
Note that the objective in equation 3.3 is a weighted version of the generic CL objective in equation 3.2 applied to common group-wise representations hy,
This theorem shows that all samples from the same block have the same representation as ni - oo; i.e. all of 's converge to their corresponding h This phenomenon is known neural collapse. It has been studied in the context of layer peeled model for supervised learning (Papyan et al., 2020; Zhu et al., 2021; Mixon et al., 2020; Lu & Steinerberger, 2020), and in CL representations with balanced data (Fang et al., 2021b). Fang et al. (2021b)'s results are most similar to ours, but they do not elicit the effects of dataset imbalance on the geometry of the collapsed representations.
-0.6
derstand the representation harm we reconsider the setup of the synthetic experiment in Section 3.1. The SBM has three blocks, where for a given 1 we set 72 = Ty = (1-m) /22 In the leftmost plot of Figure 5 we present the connectivity matrix, and be- fore, only the first two blocks are connected with probability a. In this setup, obtain representations hgg from equation 3.3 and plot their cosine similarities for pairs (1, 2) and (2, 3) in Figure 5. The harm in representation becomes more prominent between the first two groups with the severity of the underrepresenttation of the minority group, as their cosine increases in the middle plot of Figure 5. In fact, their representations become identical for Ti < 2-6 when there is sufficient connectivity between the first two groups (a = 0.4). Additionally, the two disconnected majority groups (second and third groups) get further apart, and at the extreme ( > 2- they become exactly opposite (cosine -1).
Representation harm: To un-
0.95
a
0.0
0.7
0.7
0.2
0.95
0.0
-08
0.1
0
0.2
0.0
0.95
-02
1.0
3
2
2
2
m1
Figure 5: Left: connectivity, and middle and right: cosine similar- ities between groups (1, 2) and (2, 3).
Finally, when the first two groups are less connected (a = 0.1), their representations do not collapse, even for a severe underrepresentation of the first group (cosine is less than zero when T1 < 2 6). This relates to observation in CIFAR 10 case-study in Figure 8a in Appendix A.1.I, that the dog class, which is arguably disconnected from other classes, suffers the least allocation harm when is underrepresented.
Practical implications: Reiterating our previous discussions, our analysis suggests that harm in representation occurs between two semantically similar groups when either of them is underrepre- sented. Therefore, mitigate the issue, practitioners should consider a combination of the following strategies; 1) reweighing the loss to counteract underrepresentation. an approach considered in Liu al. (2021c); Zhou et al. (2022), and 2) a "surgery" on connectivity, similar to the approach in Ma et al. (2021). We defer our further discussion of the mitigation of harm to Section 5.
HARMS IN CL REPRESENTATION CAUSE ALLOCATION HARMS
In Section we observed that underrepresentattoon of a group causes CL representation to collapse with symantically similar groups. Through a causal mediation analysis (CMA) (Pearl, 2022), in this section, we show that this is partly responsible for allocation harm (AH) in downstream classification tasks. Thus, to fully mitigate downstream allocation harm, practitioners must address this representation harm.
M
Figure Basic model for causal mediation analysis

---

Published as a conference paper at ICLR 2024
4.1 BACKGROUND ON CAUSAL MEDIATION ANALYSIS
The goal of CMA is to decompose the causal effect of a treatment on an outcome into a direct (causal) effect that goes directly from the treatment the outcome and an indirect effect that goes through other variables in the causal graph. In this section, we consider the basic CMA graph shown in Figure Here, treatment T' whether the dataset undersampled, M is the CL representations, and Y is measure of downstream allocation harm (e.g. misclassification rate). To introduce our decomposition we require the following notation. The outcome under treatement T = t is denoted as (T)(T=)). and the same under treatment T = f and mediation M = m is denoted as Yao mediation under treatment is denoted as M,- = Mao(T=I). Note that Ydo(r=1) = Y2o(T=I,M=M))>
Similarly,
The two effects that we evaluate are the natural indirect effect (NIE) and the reverse natural direct effect (rNDE). It is easily seen that they add up to the total effect TE - E (door==i) - E oo
TE E (a) B[Yao(T=O.M=M)] + [
M(
-INDE
NIE
(4.1)
Here, the NIE is the downstream allocation that can attributed to (representation harms) in the CL representation, while -INDE is the downstream allocation harms directly caused by undersampling.
4.2 MEDIATION ANALYSIS ON CONTROLLED STUDY
In our controlled study with CIFARIO dataset in Section 2.1 we observed that semantically similar pairs such as (automobs les, t trucks), (ai rplares, shi ss), and (deers, horses) suffer from representation harm when one of the classes is underrepresented. Following this, using mediation analysis, we show that may cause allocation harm in a downstream classification, thus emphasizing its importance in mitigating allocation harm.
Setup: Suppose that treatment T 1 is the underrepresentation of class k. We train a linear head with a randomly chosen 75% of the test data, consider- ing two scenarios: (1) it is trained on balanced dataset (I" 0), and (2)
TE
NIE
airplane
.3563
0.254
0,232
0,031
5,022
0.055
5/011
6
0.000
0,502
0,203
0.001
ship
0862
.0012
0,292
0,730
0.118
0.502
0.002
0.001
automobile
=0000
25.000
-0.00
002
-00
+3.0
00 00)
01001
000
-0 00
10.0
10.
truck
=00002
0.089
=0.00
0.317
000
-0 008
20.
-00
0.007
=300
0.017
20.0
+0:00
55 000
00 000
+0.0
2-0
6m011
+0.3
is trained on an imbalanced dataset where the class A is subsampled to 1% of its original size (T = 1). Recall- ing that Shal (resp. SK) denotes the CL model trained on balanced (resp. class k underrepresented) training dataset, we denote fra as Ma and fú as -a- CL model coupled with a linear head builds the final image classifier, which we always evaluate on the remaining 25% of the test dataset.
deer
0.003
0.004
=3.0
0.301
0.027
10,00
0.35
2012
+00
0.00
0 0.000
=3.0
0.23
=0.0
8.555
20021
6.00
0.0
0.00
0/007
077
5:00
0.901
0.991
1.0
200001
aisplane
automoble
brack
deer
horm
airplane
dap automobile
truck
deer
horge
Figure 7: TE (left. -NDE (middle) and NIE (right) for CIFARIO dataset over 10 repetitions.
Metrics: We consider three classifiers, where linear head is trained on top of: (1) fual using balanced data (T = 0, MM = Mao(T=o)). which we denote as No (2) fa using balanced data (T = 0, M = Mao(T=1)), denoted as 1o 1. and finally (3) fa using imbalanced data (T = 1, M = Mao(T=1)), denoted as Ø1.1- With all the notations in hand, the total effect (resp. natural indirect effect) due to the underrepresentation of the class k in classifying sample x with the true class
y = 1: as 7
TE 1) = P(ol)) (x) = 1 = k) - P(io.o (x) = =1 -))
(4.2)
NIE(K, P(661) (x) 1 y = k) - P(io,o (x) =1 =1 = k:)
Finally, the corresponding reverse natural direct effect is calculated as - -NDE(KK 1) TE(K, NIE(k,I). I). For + I the TE(k, 1) > 0 indicates how much more often the class k is mistaken for the class 1 when & is underrepresented and NIE(k, 1) is the part that is caused by representation harm.
8

---

Published as a conference paper at ICLR 2024
We present the TE, -INDE and NIE heatmap plots in Figure 7, where TE(k, 1) is the entry of the (ke, -) cell and similarly for -INDE and NIE. As in Section 2.1.2, we present the metrics for only six classes (four vehicles and two animals). The remaining ones provided in Appendix A.1.1.
Results: In Figure we mainly focus on the NIE, this part of allocation harm caused by harm in CL representation. When underrepresenteed. aut outoo les are mistaken as t truck most often (off-diagonal TE is highest at 0. 0.798 + 0.002). Additionally, the allocation harm caused by harm in CL representations is the highest in this case (highest observed value of the NIE metric at 0.062 + 0.001), which is related to the highest representation harm observed between them (RH metric 0.78++0.000) Furthermore, the harm in CL representations causes a significant allocation harm for semantically similar pairs (aut omobiles trucks), tairplanes NIE metrics are significantly higher. This aligns with our observations in Section 2. 1, as these pairs suffer significant representation harm due to their underrepresentation.
ships), and (deers, horses) as their
Additionally, representation harm causes significant reduction in downstream accuracy for these classes. This is observed in the diagonal NIE metrics in Figure 7, which is the highest for deer (15.5 + 0.2%) and is at least 7.9 + 0.1%. This emphasizes that one cannot completely mitigate allocation harm without addressing the harm in CL representations.
For the BIASBIOS case study performing mediation analysis is challenging due to the existing natural imbalance, which makes it infeasible to create a balanced dataset and thus train a balanced CL model. It is, however, possible to quantify allocation harm, which we investigate in Appendix B.4.
5 SUMMARY AND DISCUSSION
We studied the effects of underrepresentations on contrastive learning algorithms empirically on the CIFARIO 10 and BIASBIOS datasets and theoretically in stochastic block model. We find (both theoretically and empirically) that the CL representations of an underrepresented group collapse to a semantically similar group. Although prior work shows that classifiers trained on top of CL representations is more robust to underrepresentation than supervised learning (Liu al., 2021c) in terms of downstream allocation harms, we show that the CL. representations themselves suffer from representation harms: the CL. representations of an underrepresented group collapse semantically similar group. To reconcile our results with prior work, we decompose the downstream allocation harms in classifiers trained on top of CL representations into a direct effect and an indirect effect mediated by the CL representations via a causal mediation analysis. Our results show that it is necessary to address the representation harms in CL representations in order to eliminate allocation harms in classifiers built on top of CL representations.
Attempts to mitigate representation harms in CL: tation in conflict with one the key advantages of CL or SSL: their ability to use large uncurated datasets from the Internet. Our results suggest that broad adoption of SSL can lead to representations harms to underpriviledged groups This corroborates recent empirical results on algorithmic biases (Johnson, 2022; Naik & Nushi, 2023) in prompt-guided generative models powered by CLIP (Radford et al., 2021) (e.g., DALL-E 2), which uses a variant CL loss. To address these issues, we need SSL methods that can account for underrepresentation without requiring group annotations. Prior works have proposed several such methods (Liu et al., 2021c; Zhou et al., 2022; Assran et al., 2022), but their evaluation metrics are limited downstream accuracy, and thus their effectiveness in mitigating representation harms remains unexplored. In Appendix A.3 we investigate the representations learned with one of these methods, boosted contrastive learning (BCL) (Zhou al., 2022), which turns out not to be sufficiently effective to completely mitigate the representation harm. Thus, the elimination of representation harms in CL. remains a pressing open problem.
Broadly speaking, the issue of underrepresen-
Our theoretical analysis suggests that practitioners should consider a combination of the following strategies to mitigate harm in representation; (1) reweighing the loss to counteract underrepresentation and (2) "surgery" on connectivity, both of which require group annotations. Since they are not available in most CL applications, it would be interesting to attempt to combine the two using a proxy for group annotations. There have been many efforts improve performance in minority groups without group annotations in the supervised learning setting (Hashimoto et al., 2018; Liu et al., 2021b; Zeng et al., 2022), but it remains be seen whether these techniques can be transferred to SSL..
9

---

Published as a conference paper at ICLR 2024
ACKNOWLEDGMENTS
This paper is based upon work supported by the NSF under grants no. 2027737 and 2113373.
REFERENCES
Mohsen Abbasi, Sorelle A. Friedler, Carlos Scheidegger, and Suresh Venkatasubramanian. Fairness in representation: Quantifying stereotyping representational harm. arXiv:1901.09565 [cs, stat], January 2019.
Alekh Agarwal, Alina Beygelzimer, Miroslav Dudík, John Langford, and Hanna Wallach, A Reductions Approach Fair Classification. arXiv: 880.00233333 [cs], July 2018.
Shin Ando and Chun Yuan Huang. Deep over-sampling framework for classifying imbalanced data. In Machine Learning and Knowledge Discovery in Databases: European Conference. ECML PKDD 2017, Skopje, Macedonia, September 18-22, 2017, Proceedings, Part 10, pp. 770-785. Springer, 2017.
Julia Angwin, Jeff Larson, Surya Mattu, and Lauren Kirchner. hwtp:ppppbbciicoog/arrcccceaaaaa saathiine-bb---iis--a-sse--e aaaar-iisskssssmentts tetttss-ii-iiii--seeeeeeeee May 2016.
Machine Bias.
Sanjeev Arora, Hrishikesh Khandeparkar, Mikhail Khodak, Orestis Plevrakis, and Nikunj Saun- shi. A theoretical analysis of contrastive unsupervised representation learning. arXiv preprint arXiv:1902.09229, 2019.
Mahmoud Assran, Randall Balestriero, Quentin Duval, Florian Bordes, Ishan Misra, Piotr Bojanowski, Pascal Vincent, Michael Rabbat, and Nicolas Ballas. The hidden uniform cluster prior in self- supervised learning, arXiv preprint arXiv:2210.07277 2022.
Arun Babu, Changhan Wang, Andros Tjandra, Kushal Lakhotia, Qiantong Xu, Naman Goyal, Kritika Singh, Patrick von Platen, Yatharth Saraf, Juan Pino, et al. XIs-r: Self-supervied cross-lingual speech representation learning at scale. arXiv preprint arXiv:2111.0929) 2021.
Alistair Barr. Google mistakenly tags black people as 'gorillas," "showins limits of algorithms. The Wall Street Journal, 1(7):2015, 2015.
Yoshua Bengio, Aaron Courville, and Pascal Vincent. Representation learning: A review and new perspectives. IEEE transactions on pattern analysis and machine intelligence, 3588):1798-1828, 2013.
Jonathon Byrd and Zachary Lipton. What is the effect of importance weighting in deep learning? In International conference on machine learning. pp. 872-881. PMLR, 2019.
Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In International conference on machine learning, pp- 1597-1607. PMLR, 2020.
Xinlei Chen and Kaiming He. Exploring simple siamese representation learning. In Proceedings of the JEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 15750-15758, 2021.
Kate Crawford. The Trouble with Bias, December 2017.
Jeffrey Dastin. Amazon scraps secret Al recruiting tool that showed bias against women. Reuters, October 2018.
Maria De-Arteaga, Alexey Romanov, Hanna Wallach, Jennifer Chayes, Christian Borgs, Alexandra Chouldechova, Sahin Geyik, Krishnaram Kenthapadi, and Adam Tauman Kalai. Bias in Bios: Case Study Semantic Representation Bias in a High-Stakes Setting. Proceedings of the Conference on Fairness, Accountability, and Transparency FAT* '19, pp. 120-128, 2019. doi: 10.1145/3287560.32875777
Jacob Devlin, Ming- Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. arXiv: [cs], October 2018.
10

---

Published as a conference paper at ICLR 2024
Cong Fang, Hangfeng He, Qi Long, and Weijie J. Su. Layer-Peeled Model: Toward Understanding Sell-Trained Deep Neural Networks. arXiv:2101.12699 [cs, math, stat], January 2021a.
Cong Fang. Hangfeng He, Qi Long, and Weijie J Su. Exploring deep neural networks via layer-peeled model: Minority collapse imbalanced training. Proceedings of the National Academy of Sciences, 118(43))22103011 18, 2021b.
Hongchao Fang, Sicheng Wang, Meng Zhou, Jiayuan Ding, and Pengtao Xie. Cert: Contrastive self-supervised learning for language understanding. arXiv preprint arXiv:2005. 12766, 2020.
Tomer Galanti, András György, and Marcus Hutter. On the role neural collapse in transfer learning. arXiv preprint arXiv:21/2.55/22 2021.
Tianyu Gao, Xingcheng Yao, and Danqi Chen. SimCSE: Simple Contrastive Learning of Sentence Embeddings. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pp. 6894-6910, Online and Punta Cana, Dominican Republic, November 2021. Association for Computational Linguistics. doi: 10. 20.1886311/200 16655/1/002...mmmmmmaa mainp-
Aditya Grover and Jure Leskovec. Node2vec: Scalable Feature Learning for Networks.
arXiv [es, stat), July 2016.
XY Han, Vardan Papyan, and David L Donoho. Neural collapse under mse loss: Proximity to and dynamics on the central path. arXiv preprint vrXiv:2106.02075 2021.
Jeff Z. HaoChen, Colin Wei, Adrien Gaidon, and Tengyu Ma. Provable Guarantees for Self- Supervised Deep Learning with Spectral Contrastive Loss. 1.Xiv:2006.04156 [cs, stat], June 2021.
Moritz HardT, Eric Price, and Nathan Srebro. Equality of Opportunity in Supervised Learning. arXiv:1610.0241 [cs], October 2016.
Tatsunori B. Hashimoto, Megha Srivastava, Hongseok Namkoong, and Percy Liang. Fairness Without Demographics in Repeated Loss Minimization. arXiv: 08010 [cs, stat], June 2018.
Haibo Hc, Yang Bai, Edwardo A Garcia, and Shutao Li. Adasyn: Adaptive synthetic sampling approach for imbalanced learning. In 2008 IEEE international joint conference on neural networks (IEEE world congress computational intelligence), pp. 1322-1328 IEEE, 2008
Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum Contrast for Unsupervised Visual Representation Learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9729-9738, 2020.
Paul W. Holland, Kathryn Blackmond Laskey, and Samuel Leinhardt. Stochastic blockmodels: First steps. Social Networks, 5(2):109-137, June 1983. ISSN 03788733. doi: 10.1016/0038--8733333 90021-7.
Wei-Ning Hsu, Benjamin Bolte, Yao-Hung Hubert Tsai, Kushal Lakhotia, Ruslan Salakhutdinov, and Abdelrahman Mohamed. Hubert: Self-supervised speech representation learning by masked prediction of hidden units. JEEE/ACM Transactions on Audio, Speech, and Language Processing. 29:3451-3460, 2021.
Like Hui, Mikhail Belkin, and Preetum Nakkiran. Limitations of neural collapse for understanding generalization in deep learning. arXiv preprint 2rXiv:2202.08384, 2022.
Badr Youbi Idrissi, Martin Arjovsky, Mohammad Pezeshki, and David Lopez-Paz. Simple data balancing achieves competitive worst-group-accuracyy arXiv:21 14500 [cs], February 2022.
Longlong Jing and Yingli Tian. Self-supervised visual feature learning with deep neural networks: A
survey. IEEE transactions on pattern analysis and machine intelligence, 43(1 ():4333 4058, 2020.
Khari Johnson. DALL-E 2 creates incredible images-and and biased ones you don't see. woo.-attttet---------------naeae May 2022.
Alex Krizhevsky, Geoffrey Hinton, al. Learning multiple layers of features from tiny images. 2009.

---

Published as a conference paper at ICLR 2024
Pranjal Kumar, Piyush Rawat, and Siddhartha Chauhan. Contrastive self self-suuervised learning: review, progress, challenges and future research directions. International Journal of Multimedia Information Retrieval, 11(4):461-45) 2022.
Phuc H Le-Khac, Graham Healy, and Alan Smeaton. Contrastive representation learning: A framework and review. Ieee Access, 8:193907-193934, 2020.
Jason D. Lee, Qi Lei, Nikunj Saunshi, and Jiacheng Zhuo Predicting What You Already Know Helps: Provable Self-Supervised Learning. arXiv:2008.0106 [cs, stat), August 2020.
Andy T Liu, Shang-War Wen Li, and Hung-yi Lee. Tera: Self-supervised learning of transformer encoder representation for speech. IEEE/ACM Transactions Audio, Speech, and Language Processing, 29:2351-2366, 2021a.
Evan Z. Liu, Behzad Haghgoo, Annie S. Chen, Aditi Raghunathan, Pang Wei Koh, Shiori Sagawa, Percy Liang, and Chelsea Finn. Just Train Twice: Improving Group Robustness without Training Group Information. In Proceedings of the 38th International Conference on Machine Learning, pp. 6781-6792. PMLR, July 2021b.
Hong Liu, Jeff Z HaoChen, Adrien Gaidon, and Tengyu Ma. Self-supervised learning is more robust to dataset imbalance. arXiv preprint arXiv: 10.05025, 2021c.
Yixin Liu, Ming Jin, Shirui Pan, Chuan Zhou, Yu Zheng, Feng Xia, and S Yu Philip. Graph self- supervised learning: A survey. IEEE Transactions on Knowledge and Data Engineering, 35(6): 5879-5900, 2022.
Jianfeng Lu and Stefan Steinerberger. Neural collapse with cross-entropy loss. arXiv preprint arXiv:2012.08465 2020.
Martin Q Ma, Yao-Hung Hubert Tsai, Paul Pu Liang, Han Zhao, Kun Zhang, Ruslan Salakhutdinov, and Louis-Philippe Morency. Conditional contrastive learning for improving fairness in self- supervised learning, arXiv preprint arXiv:2 2021.
Ishan Misra and Laurens van der Maaten. Self-supervised learning of In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 6707- 6717, 2020.
representations.
Dustin G Mixon, Hans Parshall, and Jianzong Pi. Neural collapse with unconstrained features. arXiv preprint 2rXiv:2011.1 1619, 2020.
Ranjita Naik and Besmira Nushi. Social biases through the text-to-image generation lens. arXiv preprint FrXiv:2304.0603. 2023.
Cathy O'Neil. Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy. Crown, September 2017. ISBN 978-0-553-41893-5.
Vardan Papyan, XY Han, and David L Donoho. Prevalence of neural collapse during the terminal phase of deep learning training. Proceedings of the National Academy of Sciences, 117(40); 24652-244 -2663,, 2020.
Judea Pearl. Direct and indirect effects. In Probabilistic and causal inference: the works of Judea Pearl, pp. 373-392. 2022.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pp. 8748 8763. PMLR, 2021.
Shiori Sagawa, Aditi Raghunathan, Pang Wei Koh, and Percy Liang. An Investigation of Why Overparameterization Exacerbates Spurious Correlations. 0444442005004345 [cs, stat], August 2020.
Madeline Schiappa, Yogesh S Rawat, and Mubarak Shah. Self-supervised learning for videos: A survey. ACM Computing Surveys, 55(13s):1-37, 2023.
12

---

Published as a conference paper at ICLR 2024
Junjiao Tian, Yen-Cheng Liu, Nathaniel Glaser, Yen-Chang Hsu, and Zsolt Kira. Posterior re- calibration for imbalanced datasets. Advances in Neural Information Processing Systems, 33: 8101-8113, 2020.
Yuandong Tian, Xinlei Chen, and Surya Ganguli. Understanding self-supervised learning dynamics without contrastive pairs. In International Conference Machine Learning, pp. 10268- 10278. PMLR, 2021.
Christopher Tosh, Akshay Krishnamurthy, and Daniel Hsu. Contrastive learning, multi-view redun- dancy, and linear models. arXiv:2008. 10150 [cs, stat], April 2021.
Laurens Van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of machine learning research, 9(11), 2008.
Changhan Wang, Anne Wu, Juan Pino, Alexei Baevski, Michael Auli, and Alexis Conneau. Large- scale self-and semi-supervised learning for speech translation. arXiv preprint 2021.
Tongzhou Wang and Phillip Isola. Understanding Contrastive Representation Learning through Align- ment and Uniformity on the Hypersphere. In Proceedings of the 37th International Conference on Machine Learning, pp.
PMLR, November 2020.
Junliang Yu, Hongzhi Yin, Xin Xia, Tong Chen, Jundong Li, and Zi Huang. Self-supervised learning for recommender systems: A survey. IEEE Trunsactions Knowledge and Data Engineering, 2023.
Yuchen Zeng, Kristjan Greenewald, Kangwook Lee. Justin Solomon, and Mikhail Yurochkin. Outlier- robust group inference gradient space clustering. arXiv preprint arXiv:2210.06759 2022.
Zhihan Zhou, Jiangchao Yao, Yan-Feng Wang, Bo Han, and Ya Zhang Contrastive learning with boosted memorization. In International Conference on Machine Learning, pp. 27367-27377. PMLR, 2022.
Zhihui Zhu, Tianyu Ding, Jinxin Zhou, Xiao Li, Chong You, Jeremias Sulam, and Qing Qu. A geometric analysis of neural collapse with unconstrained features. In M. Ran- zato, A. Beygelzimer, Y. Dauphin, P.S. Liang and J. Wortman Vaughan (eds.), Advances in Neural Information Processing Systems, volume 34, pp. 29820-29834, Curran Asso- ciates, Inc., 2021. URL https: t hww.htttpps//procceeenngeges 00207//1...f22222222222001116110 Paw.eededsr..--P..pp..... Paper.
neurips cccppppr_aiiiiceeeeeeeeeeeeeeepe files//plpppe files/paper// paper/
A SUPPLEMENTARY DETAILS FOR UNDER- -ERRREEETEEETOOOO STUDY WITH CIFARI0
A.1 SUPPLEMENTARY DETAILS FOR SIMCLR
We the implementation in simelr. py file of the training of contrastive learning (CL) models with the SimCLR training protocol. Please see simclr. . py in supplementary codes for parameter values. We use the same parameters in both training cases with balanced and imbalanced datasets.
for
A.1.1 REPRESENTATION AND ALLOCATION HARM
The representation and allocation harms for all 10 classes are provided Figures 8a and 8b. Here, each corresponds underrepresentation to 1% for the corresponding class. Additionally, in Figures 9a and 9b we present similar plots when classes are underrepresented at 5% of their original sizes.
13

---

Published as a conference paper at ICLR 2024
TC
100
025
0,60
3602 2017
CART
3411
20.
1.84
180
900 1110000--00 40000000000000
200 100000
EC00 0.77
8250
246
096
2.03
3603
80
3005
+86
-80
3.00
10
Mr.
-0000
18:0
18.0
10.00
2.39
-36
16.0
10.3
180
201 1E 1:56 1811
5:000
0.00
0.30
44g
-80
+00
+50
-00
(a) Representation harm
(b) Allocation harm
Figure 8: Representation harm and total effect and natural indirect effect for allocation harm for SimCLR representations in CIFAR10 10 where the classes are undersampled to 1%.
1004 OMT CGO AST CA
ET5 EGU
GUM GAET 3822
R012
ARL
BOUS
KGS 3415
202
204
3.44
-80
10000
(441 20.
6:65 6.00
6.33
401
0.81
9.11
86
800
0.300
4.95
0.94
36
3.05
195
4.00
11.0
16.0
180
0:00
18.00
20
-34
0.000
06.0
20.0
6:04
0.30
-90 400
600
0.35
909
(a) Representation harm
(b) Allocation harm
Figure 9: Representation harm and total effect and natural indirect effect for allocation harm for SimCLR representations in CIFAR 10 where the classes are undersampled to 5%.
A.2 SUPPLEMENTARY DETAILS FOR SIMSIAM
We use the implementation in ma i py file of acc-on- -IIFRROOO for the training of CL models with the SimSIAM protocol. Please see our jobs py and main py for the specification of hyperparameters, which are kept the same in both training cases with balanced and imbalanced datasets. We use 3 repetitions of each setting with three different seed values. The rest of the details are the same as in SimCLR training.
91.9-top1-
LIU EVR LIGH
COGE
EOG
STDE
ROW
$550
14:00
103
503
103
114
114
0.30
1.000
304
-04
134
234
180
6:00
36
390
2-001 68:0
8.00
61000
0.300
3001
2009
$6.00
34.0
0.204
6006
480
4600
40.6
01144 2E1
1xm
K103
1ED
K0M
K22
6:28
0301
24
20.00
2
(a) Representation harm
(b) Allocation harm
Figure 10: Representation harm and total effect and natural indirect effect for allocation harm for SimSIAM representations in CIFAR 10 over three repetitions.
14

---

Published as a conference paper at ICLR 2024
A.2.1 REPRESENTATION AND ALLOCATION HARM
We plot the representation harm metrics equation 2.1 in Figure 10a. Similar to SimCLR representations in Section 2.1.2, representation harm is observed between semantically similar pairs (automobil les,, trucks) (RH between them are 0.834 0.025 and 0.882 + 0.0), (airplanes, ships) (RH is 0.878 = 0.0) and (deer, horse) (RH is 0.827 + 0.003).
Representation harm:
Allocation harm: In Figure 10b we plot the TE and NIE metrics equation 4.2, where we observe that representation harm causes allocation harm in downstream classification. This is evident from the NIE metrics in Figure 10b, as the NIE between pairs (a sirples ane, ship) are 0.111 + 0.001 and 0.072 + 0.001, between (automobil lle truck) are 0.145 + 0.002 and 0.061 + 0.002, and finally between (deer, horse) are 0.075 = 0.0 and 0.039 = 0,001, Additionally, these classes suffer a non-trivial reduction in their prediction accuracies due to harm in representation. These observations are similar to our findings regarding allocation harm for SimCLR representation (Section 4).
A.3 BOOSTED CONTRASTIVE LEARNING WITH SIMCLR
We implement the boosted contrastive learning (BCL) algorithm (Zhou et al., 2022) on SimCLR protocol using the implementation in t train. py file in http:/iilubb.cmm/MediBBaaaaa Specifically, we use the BCL_I version of the algorithm, whose hyperparameter values can be found in the files jobs. py and train. py in the codes /c / far far/BCL/ folder of our supplementary materials. Each setup is training for three repetitions with different seed values. The representation and allocation harm metrics for 10 CIFARI 10 classes are provided in Figure 11.
TF
CON
63%
CAH
GALT
XJLS REST
200 BOGS EGO
4018
603
402
AG4
K00 8110
64.9
0130
0181
0.01
100
RGO
XGU
CIKE
6.79
48.0
5000
DED
5179
PM
Aug
340
MAC COL
MY MELI 1M4
280
ZED
24.0
SCON
124
260
340
305
ICTN
204
TM
(a) Representation harm
(b) Allocation harm
Figure 11: Representation harm and total effect and natural indirect effect for allocation harm for BCL representations in CIFAR10 over three repetitions.
Attempts to mitigate representation harms in CL with boosted CL (Zhou et al., 2022): In our case study on CIFARIO dataset in Figure 11, we find that BCL may not be sufficiently ef- fective to completely mitigate the representation harm. Although the RH metrics between the pairs (automobilee 0.829 + 0.001 and 0.897 + 0.001 (in Figure I Ia), BCL does not completely mitigate the harm in representations. As a result, accuracies for these classes still suffer non-trivial reductions, as observed in the diagonal NIE metrics in Figure 1lb (the reduction in accuracy is 5.3 0..2% (resp. 0.01% for aut omobiles (resp. trucks)). These results show that the elimination of representation harms in CL remains a pressing open problem.
t trucks) increase from 0.78 + 0.009 and 0.832 + 0.012 (in Figure 2) to
15

---

Published as a conference paper at ICLR 2024
B SUPPLEMENTARY DETAILS FOR BIASBIOS
STUDY WITH
B.1 BIASBIOS DATASET DETAILS
In Figure 12, we provide the sample counts and frequencies for the 28 occupations and 2 genders (Male and Female) along with Total counts. We note that the dataset is imbalanced both in the occupation di- mension as well as the gender dimension within particular occupation. Particularly, the Professoz occupation the most occurring, while the Rapper occupation is the least occurring. Within each occupation, the proportion of samples of the two genders also varies mimicking societal gender stereo- types. For example, the occupations dieticies etc. are dominated by Female samples, while the occupations composer, DJ, pastor, rapper, software engineer, surgeon, etc. are dominated by Male samples.
interior desi
model, nurse,
2014
6::33
1:14
2-2
2003
1184
6214
1.S
14:77
A14:
11:44
50609
2759
5000
LA4T9
1073
1402
1443
399.0
00673
2510
3432
30909
9330
33/333
00039
3437
6993
33279
333423
1.2%
2.6
KM
144
AM
2.1%
4.A%
4.4%
4.%%
1.N%
Figure 12: BIASBIOS dataset counts (above) and frequencies (below) by occupation and gender. The Y-axis labels are genders (M)ale, (F)emale, and (T)otal counts.
B.2 SIMCSE EXPERIMENTAL SETUP
Table 1: Training parameters for SimCSE.
We randomly divide the 4000 BIASBIOS dataset into the following three splits: 65% as training set, 10% as validation set, and 25% as test set. We use the official SimCSE imple- mentation' to train the embedding model, with the optimal parameters used in the reported experiments listed in Table 1. To find these optimal parameters, we perform a grid search over the following parameters: training epochs 3, 5). learning rate E (3e le 5e -3. batch size E {64, 128, 256, 512]. and sequence lengths (32, 64, 128, 256, 512), and then train
model parameters
value
batch size
64
sequence length
512
learning rate l -5
training epochs 1
a Logistic Regression model on top of each these embedding models. We select the parameters which
result in the best occupation prediction accuracy on the validation set.
B.3 REPRESENTATION HARM
We present the GRH results for all occupation in the BIASBIOS dataset in Figure 13 (the F and M in the row labels indicate the under-representee gender for the corresponding occupation). We focus on the off-diagonal entries which should be >1 when there is representation harm. We observe several de- viations from this rule, broadly classifying into two types of deviations: (1) when representations of the redre--epreeented group for an occupation collapse with similar occupation, and (2) when represen- tations of the under-repeesented group for an occupation collapses with several occupations. For the first type of deviation, we especially note GRH (architect (F), interior derdddess (err_desigeer) gner). 0. 95, GRH (chi ropractor (F), personal (triierr) = 0. 96, GRH (nurse (M),
hhttps:////t thubithub.ccoo wttpttttt...oooooocooooooommmmme S/CkK
16

---

Published as a conference paper at ICLR 2024
architect
0.93
attomay (F)-
0.95
0.58
2.24
chiropracter (F].
0.95
0.99
0.97
111
0.56
3.3
0.99
0.59
comodian [F)-
0.94
1.1
1.1
1.1
1.1
14
2.2
1.1
1.
2.3
2.3
composer (F]
0.94
1.1
1.1
14
1.1
14
dentist
0.92
3.3
1.1
(M]
himmaker
Interior designer [M] 1.1
1.1
1.1
11
1.1
1.1
1.1
joumnlistt [F]
0.96
0.98 0.98 0.9 3.57 0.94
5.94
0.49
0.50
0.97
0.97
0.99
0.99
model [M)
0.90
0.95
0.99 0.92 0.5
3.58 0.97
5,59
0.95
0.95
0.97
0.07
0.88
5.97
nurse [M]
0.96
0.97
0.99
0.96
0.59
0.98
0.59
0.59
painter (F)-
1.1
1.1
1.1
1.1
paralegal (M] 0.97
pastor
personal
1.3
0.92
photographer (F)
3.59
0.50
0.92
1.1
0.59
physician [M]
0.97
0.57 0.95
0.39
0.95
0.36
poet IFI-
1.1
3:3
1.1
1.1
1.1
1.1
1.1
0.92
1.3
3.1
1.1
1.1
professor (F1- 1
0.98
0.35
0.99
0.98
0.09
0.96
1
0.36
0.96
0.93
0.55
0,59
0.99
0.37
9.26
0.59
0.99
0.39
0.95
psychologist [M]
0.99
0.00
0.59
0.00
rapper
1.2
12
1.2
0.05
1.1
software engleenniieee
teacher [M]
0.98
0.97
097
3:58
5.54
0.47
0.05
0.04
teacher (M].
11
322
111
1.3
11
3.1
12
11
12
12
1.1
1.1
1.1
11
1.1
111
sccommlan
chroproco
comedise
composer
deniat
dakktar
fimmarer
mode
Figure 13: Gender RH for all occupations in BIASBIOS dataset.
17

---

Published as a conference paper at ICLR 2024
dentist) 0.97, GRH (physician (M), surgeon) = 0 96, and GRH (surgeon (F), dentist) 0. 95 among others. These mentioned deviant groups are of occupations that are highly related. For the second type of deviation, note that the GRH of Models (M), Journalists (F), and Teachers (M) have values < 1 for many occupations in the dataset.
B.4 ALLOCATION HARM
We consider the task of occupation prediction from the SimCSE representations we learned on the BIASBIOS dataset using a logistic regression model trained on the same data as we used to learn the representations. Such decision-making system may be used to assist in recruiting or hiring. applications where allocation harm can exacerbate gender disparity. To counteract the effect of underr-eepresentation the supervised learning stage (thus focusing our attention on the effect of under-representation on CL., we reweigh the samples to achieve gender parity within each occupation when training the logistic regression following prior work (Idrissi et al., 2022; Sagawa et al., 2020). See Appendix B.5 for results with other weighting strategies.
We define gender allocation harm (GAH) similarly to Equalized Odds, popular group fairness criteria in the algorithmic faimess literature (Hardt et al., 2016):
Metric:
GAH(L, m) Più m y female) P m y I, male).
(B.1)
GAH can be understood as the difference of confusion matrices corresponding to each gender with entries far from O implying allocation harm. The average of the absolute values of the GAH diagonal a common way to quantify violations of Equalized Odds.
Results: In Figure 14 we present the GAH results for a subset of occupations. The complete set of GAH results is pro- vided in Appendix B.5, Inspecting the di- agonal entries,
professar 0.01
00,00
0.000
0.,00
attorney 0.00
-0.01
0.00
.0.00
-0.00
.0.00
0.00
-0.00
0.00
.0.00
paralegal - 0.00 0.05
0.03
0.00
0.00
-8.02
0.00
0.00
0.00
-0.0
large performance gaps between genders for occupations such rapper, DJ, and model 1. (within each of these occupations the majority gender corresponds to least 85% of the sam- ples; these occupations are also underrepre- sented the data with representation rang- ing from 0.4% for rapper to 1.9% for model). These results demonstrate that reweighing for gender parity when train- ing the predictor may be insufficient to re-
surgeon - 0.03
-0.00
0.000 00.01
0.09
0.12
0.00
0.00
0.00
0.00
dentist 0.00 0.00
0.00
0.00
0..00
physiclan 0.03 00,00
0.00
-0.00
0.02 0.00
0.21
0.04
rapper--0.01 -0.01
-0.02 0.00
.0.00
0.00
0.03
0.08
-0.17
-0.02
0.08
composer- 0.01
-0.00
0.00 0.00
0.00
-8.00
0.01
-0.01
0.04
-0.01
pair harms due to underrepresentatiin the CL stage of the pipeline (it does, how- ever, help mitigate some of the biases of the unweighted model, as we show in the Appendix B.5). We also note the mistake patterns for related occupations: female DJs are predicted as composers lot more often than male ones, and female surgeons tend to be mistaken for dentists while male surgeons for physicians, despite similar performance surgeons across genders. The surgeon example demonstrates that it may be insufficient to compare only class accuracies across genders (as is often done to measure group fairness violations via Equalized Odds) when quantifying allocations harms.
model 0.06
0.00
0.00
0.00
0.00
-0.02
-0.00
0.00
-0.01
0.32
professar
attomey
paralegal
surgcon-
dentist-
physician
di
rapper-
composer
model
Figure 14: Gender AH in BIASBIOS dataset.
Compared to gender representation harms, we note that while the two have some overlap in terms of genders and occupations they are affecting, there are also differences, e.g., representations for female attorneys are closer to female paralegals than male attorneys, but it does not manifest in the allocation harm analysis. We hypothesize that differences in which groups were affected by allocation and representation harms in our experiments might be due to the logistic regression model used for prediction utilizing only a subset (or subspace) of features most relevant to the task, while representation harm analysis takes into account all features.
18

---

Published as a conference paper at ICLR 2024
The allocation harm does not have be gender-specific. Similar to CIFARIO 10 case study, samples from an underrepresented occupation can also be mistaken for a related occupation at a similar rate between genders. We observe this for the paralegal occupation which corresponds to about 0.4% of the training samples. Despite the gender imbalance (85% of paralegals in the data are female), both male and female class accuracy is the worst across occupations (14% for females and 1 11% for males) and the majority of them (66% for females and 61% for males) is predicted as attorneys, which is more frequent class (8.3% of the samples are attorneys; see Appendix B for extended allocation harm analysis for occupations).
Overall, similar to the CIFAR Oexperiment, underrepresentatton leads to the allocation harms of mistaking underrepresented groups for related group. In some cases, the related group may be the same for both genders (e.g.. in the case of paralegals), but in others, can differ across genders (e.g., in the case of surgeons and DJs). Both cases can cause allocation harms for people from underrepresented occupations, whereas the latter additionally exacerbates gender stereotypes.
B.5 OTHER DETAILS FOR ALL.OCATION HARM
We experiment with three weighting strategies to counter the gender imbalance in the BIASBIOS dataset, namely: (I) Each sample is cqually weighted, (2) We balance for gender imbalance within each class by weighting each sample as W 2N where N is the total number of samples in the dataset, Ny is the number of samples within a class and Ný is the number of samples of gender in class y. and (3) We balance for gender and class imbalance within the dataset by weighting each sample as W 2-/p1-Ny where yyl is the number of classes and NØ is the number of samples of gender in class y.
In Figure 15 we provide the Gender Allocation Harms for all occupations in the BIASBIOS dataset and for the three weighting strategies. Similar to our observations in Section 2.2, inspecting the diagonal entries, three weighting strategies. We do, however, see that reweighing for gender parity (Figure 15b) does mitigate gender bias to certain extent. For example, for the occupation nurse the GAH is mitigated, but is merely reduced for DJ and model. Balancing for class frequencies in addition to gender (Figure 15c) has little effect on the GAH.
large performance gaps between genders of the same occupation across all
In Figure 16 we present confusion matrices for the three weighting strategies to quantify allocation harms (AH) for occupations (irrespective of gender). For occupations such as paralega and interior designer we observed a small amount of GAH, however, we see that performance on these occupations is poor for the unweighted and gender-balanced balanced weighting strategies (Figures 16a and 16b), as they are often confused with the related occupations (attorney and archites tect respectively). In this case, the representation harm is due to the under-reppeesentation of these occupations as opposed to the gender imbalance within occupations that we observed in the GAH experiments. Accounting for class imbalance the weighting strategy helps to mitigate some of these biases (Figure 16c), e.g., the AH for interior designer is largely mitigated, while AH for paralegal is reduced.
Overall, conclude that allocation harms due under-repressenatiio in the contrastive learning stage can be partially mitigated during the supervised learning stage by curating/reweighing the data to have equal representation of classes and groups.
C A PROOF OF THEOREM 3.2
We recall the CL loss equation 3.2 and restate the theorem 3.2 for the reader's 's convenience. The minimization problem in a layer-peeled setting
min LCL(V),
(V) -
Σye( e(t.)) EyE(N) e( log expl% 9/-)
Eeen exp(=s))1/.) (
(C.1)
where Sdd-1 is the unit sphere in Rd. Theorem C.1. Atn the optimum representations obtained from the minimization of CL loss in
equation 3.2 satisfy the following: of 0.5.
My. where a.S. denotes almost sure equality and (hitkelK)
19

---

Published as a conference paper at ICLR 2024
(a) Equal weight for each sample
(b) Gender- baranaaldeed
(c) Gender and class- blassbbalanced
Figure 15: Gender AH for all occupations in BIASBIOS across the three weighting strategies.
(a) Equal weight for each sample
(b) Giender-balaneed
(c) Gender and class- ballneb
Figure 16: AH for all occupations in BIASBIOS across the three weighting strategies.
is a minimizer for
"I Mag
min hyE
-E Tk,
ΣE 1k An
Eki=1 Thy log { E (istt }. (C.2)
Ling-1 Thy .22
Note that the objective in equation 3.3 is a weighted version of the node2vec objective in equation 3.2 applied to common group-wise representations hy,
For our convenience, we index the i-th node in k-th block as (k, i) where i e [me] and & E [K]. We start with a simplification of the CL. loss.
C. A SIMPLIFICATION OF THE LOSS
We divide the loss into two parts: a linear and a logarithmic sum exponential part.
exp( "No 42
La (V) a n 1 d(ksids) E(kasta) (1 log
(kg. expl 2 "K3.ng
+ E(min T.in Exprrss ((hi.d))((kkaaa) {
(C.3)
A (V)
n E(kit) 1 E(eross) log (s " Dky 1s d(ki))) 2 L
20

---

Published as a conference paper at ICLR 2024
We simplify the log-sun sum exponential part
LLSE(V) + (11,11) (1) E (Mav))) log ( (kg. exp
Chani Skggfa ( exxp
(C.4)
n1 (k log [ expl
For k-th group, we define 7& _ 2e as the sample proportion and hy _ 1 Ei= Uks as the sample average the representation vectors. We further define
EMEINI Aar be Evee aara
Linear (V) A + EkE(K] th
and modified loss:
Leu. Liineu (V) LLSE(V).
(C.5)
According to Lemma C.3 the linear part of the loss has the following & convergence:: as n + 0o
max
Linear (V Llinear (V)l 4$ 0,
(C.6)
and thus
Ler (V) (V)((V))
max
= maxx [ (V) + LLSE (V) IItincrr (V) LLSE(V) (V)
(C.7)
V
Limear Liinarr (V)
$ 0.
max V
In Sections C.2, C.3, and C.4 we perform a finite sample analysis on the modified CL loss in eq.
equation C.5 and establish its neural collapse property.
C.2 A LOWER BOUND
For studying the minimization modified CL loss in equation C.5 we derive an achievable lower bound for Ler. (V). We expand the log-sum exponential part
(VV = (( log { 2 (k) exp(
o Penay Shavin
=
Mus E(mid log 1 (kig exp(
E(trdda))
M hNg
exp
Ve reging
1i hug
"ky fy 1 hay
Eminit log
E(egdda) exp
Jexp
hag
Likg (s) exp RT,, hay
(C.8)
E(kis)) log 1 ((aovss) exp(
Eiggiia expl "I Mak JoND di Wigi
Aig Akg
Eia log
Egg teyexp My hNs }}
+ E,, nं. log EnSE(K) th, exp
and provide a Jensen's 's incquality-based lower bound for
1 Eung exp(
"I "Kg exp
Σ (1) log 7
I
" lkg
Σng Ray exp
Since log is a strictly concave function, we apply Jensen's 's inequality that log(E(X)) 2 Bllog X] to each summand of (k1, i1) with respect to the probability measure
1 1 n2
exp("i "kg/s)) Exe A Axrexp heefs) 8ka
21

---

Published as a conference paper at ICLR 2024
where 8, is the Dirac-delta measure (i.e. X exp( "K Muk )) and obtain
log n Eigginn "h "eaht exp
"he Maigiiy
"eyy hig
Miy Mik
Eikging expl
(C.9)
exp(h heaft)
Tk Leyyy Weaii - - hug
H (a) Eng Fang exp(" by mxa/+)
which we combine over (h1,, (1) obtain the following
Linn ia) 1 Kka/ Texpl
"in is -22 ANB
Σ (k1.))) log
E(kging "i Meg/t
Σ knn nt
1 1 kaids Exg fag mxp(t (e//t)
exp(A1 hag/t)
)
"ky Wag.dg Nug hkg
(C.10)
exp(" Meg//)
Kk1 D'z Wky
0. Deg ohaie Phging NZ, hus
En. 7ki Eky Tka {
Ety (xp Axg/t)
The above simplification implies
LLSE(V) 2 En, mRy log Exn mky exp( Nr hiss } LLSE
(C.11)
and thus
(V) = -iinear (V) + LISE(V) 2 Liinear (V) + LiSE(V) & (V)
(C.12)
C.3 EQUALITY CONDITION OF LOWER BOUND
Under what condition equality is achieved in the inequality equation C.12? To answer that we look the Jensen's inequality applied in equation C.9. Since log is strictly concave function, equality is achieved when for any (k3, iz) holds
c Ukai hus Thks = C
(C.13)
where Ch E RR is the constant associated with the equality constraint for Jensen's 's inequality for (ki, i1). Next, we establish that these constants Cky must be exactly equal to zero. For this purpose, we average over both i and ig in equation C.13
En, Eng
J. -" (of Ukais
has T hes
Cki.i1
Bkg
mkg
183
(C.14)
T hkg hhi his = 0. A further simplification of the above equation leads
hki
meg En, Chi.ir 0.
(C.15)
Then, we let ki = kg = k and i = ig = in equation C. 13 and average over i to obtain
1
Σn Ck,s
1 Σ, U/ Uk, - ha T ha
(C.16)
1. E. 1| h21l + Σ, (We,s h2) (Mu.1 ha) mR
n4
which finally concludes that the equality
Thtt h
(C.17)
We further notice that vk.s E Sdl1 which implies hy E Sd-1
C.4 MINIMIZATION OF EQ. EQUATION C.5
Further developing on eq. equation C.12 we provide a final lower bound for eq. equation C.5.
(V) (V)
= = #
Eveuy woa ENelK Farmee
+ Ao *े log { Eks #hy exp( 1i luna }} (C.18)
Every Evengrvan Roan has Exeeiy Faraew
Et, Ahi log Ex, expl h fig
2 E #t
22

---

Published as a conference paper at ICLR 2024
where
1 ENE[K] th Evern whhh 2 RarmaeN
{ha} arg esd-
(C.19)
+ Exy Thi log Ekg Thy expl hag
The lower bound in eq. equation C.18 does involve any optimization variables (vR.i or hę) and according to eq. equation C.17 the equality achieved when
Uk fAs-
(C.20)
Thus, the minimum of LCL (V) it holds UkKi fug-
C.5 FINAL CONCLUSION OF THEOREM 3.2
In Sections C.2, C.3, and C.4 we have established neural collapse for minimization of LCL(V). It remains how that translates to the minimization (V). In eq. equation C.7 we argued that
max (V) (V) $ 0.
(C.21)
Since both (V) and (V) are continuous, we use continuous mapping theorem conclude that
d(rrg m (V), arg LeL " 0
(C.22)
in any metric d for measuring set difference, or, alternatively speaking
arg min, LCL(() " arg minny-ccL(V).
(C.23)
Since, the loss in eq. equation C.19 convergences almost surely to the loss in eq. equation 3.3 we conclude the Theorem 3.2.
C.6 ADDITIONAL LEMMA
Lemma C.2. Consider a generic sequence of vectors (Gus)NG[K))sTTm)) C SP-1 and define the
group mean &
1. Σ, Ex,i and
<(3) a ((k.)s))
(22) Exs the Akr
(C.24)
Then
max [|< " 0.
(C.25)
Proof of the lemma C.2. Indexing the dependence of c(E) as en we notice that
(=) 1/ E (( (e2 En, #kg Okg Sa 1 ((ggis)) ((lk),))))(kkas))) - aus ns Ebzidz
(C.26)
and
(E),( = E(ks (e(k)))))(tass)) - Az INa.sz |*]
# E (k2 Bl[ Bl(e(t) a21
Σ (o E((e(hsnn)(kaiaa) - 2b 1 17 .kk 2
0 3 0 = (( (
(C.27)
Thus, for any 8 0 we have
8 Ensi P(le,, > 6) s > (5) co.
(C.28)
Next, we use the first Borel-Cantelli lemma to conclude that for any we have
||c, (2) " 0.
(C.29)
Finally, the continuity of (5) and separability of S conclude the statement of the lemma.
23

---

Published as a conference paper at ICLR 2024
Lemma C.3. Assume 3.1. Then as oc the following convergence holds.
max Linear - [ [V) " 0.
(C.30)
Proof of the Lemma C.3. We start with an expansion of (V) - LLincar (V).
Liinear (V) Liinear (V)
n E(hrrt)) a(11)11)) E(ka.ta) (ik sn (K22,))))
{ "ky 12
Eby this EMElAY Fardeys
Eng 0eg
ki, hag
(C.31)
" Emiid) ((birde)
((hawaa) (1k, (
7
Ees anganing
Tee hi2
- Enreini Haroray.
1 E(a.i)) (ii Edkrnnn) (1
Ev, R ony aey
d(R()1)
Exer Ferear
Following the expansion define
(v)) + E(kaa.))) (k),)))))))))))) Veaid Ekz Aky Kk,
(C.32)
22 d(kr.in) ΣE' te or
and note that c(V) 61 (V) and ((1) 62, where c(-) is defined in lemma C.2, As conclusion of the lemma we have
maxv ||e, (V)ll "$ 0, and |ea| " 0.
(C.33)
Next, we notice that
max Liinear (V) Liinea(())
Eig thy an big
< max E(nis Ak. |1 ll2
d(kr,i))
Eec( W
(C.34)
E Eneni 60 (2 (2)
Ex, FNo oeigg
$ E Rk. maxy
where we focus on each term within the sum
Enkgn)) (1). (22.t2) d(kerts)
Dey feg Maghin Evenn Ayaay
max
Ery fig heg Evciy W
Ex, Reg aigiiggl EMcIRe ANooeyy 1
max
|eatiin Aegae, da + ter the apa 20)
(C.35)
maxx
V
EEEEE &wak w )ea) ( Enrep (
le2|-maax
En, tuy tugaeninhh,
maxy (V)(()) } of ExEIx Awar
Evreini a
Evern A
Here, both lezl and maxy |ee (V)ll2 convergence almost surely to zero. Since ΣVE(K) Tg > 0 both ΣME(K) Awak, > 0 and Σ EME(K] twak, x + eg > 0 for sufficiently large n. Finally,
max Ex, t ak, hn 1l < Ex, fh ak As maxv [[hxal12] S En, ak (C.36)
Thus have
1 Eug (1) (12 Phavii d00uta
Eng hhg Evee Agray
2 0
(C.37)
which proves the lemma that
max
Linear (V)
Linnear
(iinear(V) (V) 5
(C.38)
24

---

Published as a conference paper at ICLR 2024
D SUPPLEMENT FOR SBM SIMULATION IN SECTION 3.1
We generate an SBM dataset using graspologic sesic.siime simulmmmmllm..... at attions sbm function. We obtain the representation by optimizing the node2vec loss eq. equation 3.1 using gradient descend algorithm that has step size for (-th step as Ir, = 0.1 x t 0.2 and 7" 30000 as the number of iterations.
25


Reviews:
[{"review_id": "oo8f5Jz3zD", "reviewer": "Reviewer_X2wa", "summary": "The study presented in this paper concentrates on examining how contrastive learning (CL) can cause representation harm, particularly when certain groups are not adequately represented in the training data. The researchers demonstrate that these underrepresented groups often merge into other semantically similar groups that are better represented. Further, through a theoretical analysis involving graphs, it is shown that increased connectivity between two groups of nodes leads to their convergence. Lastly, a causal analysis reveals that the detrimental effects on representation caused by CL are irreparable for subsequent tasks, even when a probe is trained using the CL representations.", "questions": "See weaknesses above", "limitations": "", "rating": 5, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "1.\tThe composition of the paper is clear and comprehensive, discussing results in depth and highlighting representational harm. The paper offers practical insights to diminish the adverse effects of CL techniques.\n2.\tSection 4 presents an intriguing analysis, exploring how representational biases can lead to allocation disparities.\n3.\tThe paper's empirical research is commendable, and its theoretical framework provides a robust underpinning for the empirical observations. The outcomes from both empirical and theoretical perspectives appear promising.", "weaknesses": "1.\tThe text in the figures is too small for easy readability and needs enlargement for better clarity.\n2.\tA more thorough justification is needed for the chosen metric of representation harm, given its critical importance to the paper's analysis.\n3.\tThe necessity for different data setups in Sections 5 and 6, particularly the choice of causal mediation analysis for Section 6, lacks proper motivation and explanation.\n4.\tThe paper's primary limitation is the absence of a comprehensive large-scale study, especially in the realm of self-supervised learning (SSL).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "pJoU3OqVVP", "reviewer": "Reviewer_n65V", "summary": "This paper studies the effect of under-representation on the performance of minority groups in the context of self-supervised learning (SSL), specifically contrastive learning (CL). They show that CL tends to collapse representations of minority groups with certain majority groups, leading to representation harms and downstream allocation harms even when labeled data is balanced. Theory and experiments are presented to support their results.", "questions": "see weaknesses above", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "1. This is a well written paper that discusses an important topic that is well motivated. This rigorous analysis is likely of interest to practitioners and useful to mitigate the potential harm from CL methods.\n2. The empirical study is good and the theoretical study adds a solid foundation to the empirical results. Both empirical and theoretical findings are promising.\n3. Section 4 is also quite interesting, showing how representation harms can cause allocation harms.", "weaknesses": "1. There needs to be an intuitive definition of allocative harms and representation harms when they are first mentioned in the intro, which matches the precise definition in section 2.\n2. Figure text is generally too small to see clearly.\n3. Why does this metric for representation harm make sense? It seems like an important decision critical for the rest of the paper, so needs better justification.\n4. The examples used in section 2 are not very useful - sure automobiles and trucks could collapse but is it the worst thing - are there more compelling real-world examples to illustrate these problems?\n5. Why are the metrics for representation harm in 2.1.2 and 2.2.2 different? This seems weird. Even if the exact distance (eg cosine vs l2 for image or word embeddings) are different - why are the metrics different?\n6. Why do sections 5 and 6 require a different data setup? Specifically, why is causal mediation analysis the right framework to study section 6 - this was not motivated.\n7. The empirical analysis is conducted only on CIFAR10 and BIASBIOS datasets - I would have liked to see more to further strengthen the results in the paper, such as including celebA which is quite standard for studying biases in vision models, or perhaps even with results on a CLIP model for results on image-text models.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "6hblUiXMm5", "reviewer": "Reviewer_F3P4", "summary": "This paper focuses on studying the issue of representation harm in contrastive learning (CL) that arises when some groups are underrepresented in the training corpora. In this case, the authors show that the underrepresented groups tend to collapse into semantically similar groups (that are not underrepresented). In a follow up theoretical analysis on graphs, the authors show that two groups of nodes tend to collapse as their connectivity increases. Finally, through a causal analysis, the authors show that the representation harms caused by CL cannot be mitigated for downstream tasks when training a probe on top of the CL representations.", "questions": "1. Typo “a” at the end of line 6 in page 2\n2. You could use diverging palettes (with a neutral color at 1.0) in the heatmaps, to clearly distinguish HR<1. Having a colorbar next to each heatmap figure would also improve readability.\n3. In Figure 2, do you have any idea why there’s such a large difference (10%) between deer and horse collapsing?\n4. In line 4 of Sec 2.1.2, you can add “of classes” after “a pair” to help the reader understand better \n5. In Sec 2.2.2, when you define the GRH metric, I would have found it useful to have a sketch of a plane with boundaries that show what each region means. It’s something you could consider adding when you get an extra page\n6. In Sec 4.2, why do you train on 75% of the Test set?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "1. The paper studies an important problem in self-supervised learning through contrastive objectives (that of representation collapse of groups that are underrepresented in the training data)\n2. It does so in a sound and systematic way, by providing evidence on controlled images (CIFAR10), imbalanced text (BiasBios) and a theoretical analysis with artificial graphs.\n3. The paper is well written, with a thorough discussion of results and potential insufficiencies of an existing class of algorithms in overcoming representation harm, showing how more work is needed in the area of CL to result in algorithms that are robust to underrepresentation of certain groups (which can be hard to identify at scale to begin with).", "weaknesses": "The main weakness I can find from this paper is the lack of a large-scale study. SSL, and CL in particular, are most effective when used on large amounts of data. The controlled studies in this paper allow us to analyze the existing representation harms. However, it can be seen that these values are generally much lower for BiasBios than for CIFAR10. On the other hand, this could be due to the different domain (text). A study on ImageNet (larger number of both samples and classes) could potentially disentangle this confound. In particular, it would be interesting to know whether having more diversity in the data alleviates learning spurious features (e.g. collapsing classes with similar background colors), and reduces collapse of underrepresented groups.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (poster)