Contrastive learning (CL) has achieved remarkable success in learning data representations without label supervision  . However, the conventional CL loss is sensitive to how many negative samples are included and how they are selected. This paper proposes contrastive conditional transport (CCT) that defines its CL loss over dependent sample-query pairs, which in practice is realized by drawing a random query, randomly selecting positive and negative samples, and contrastively reweighting these samples according to their distances to the query , exerting a  greater force to both pull more distant positive samples towards the query and push closer negative samples away from the query . Theoretical analysis shows that this unique contrastive reweighting scheme helps in the representation space to both align the positive samples with the query and reduce the mutual information between the negative sample and query . Extensive large-scale experiments on standard vision tasks show that CCT not only consistently outperforms existing  methods on benchmark datasets in contrastive representation learning but also provides interpretable contrastive weights and latent representations. PyTorch code will be provided .
Contrastive learning (CL) is effective in learning data representations without label supervision , where the encoder needs to contrast each positive sample over multiple negative samples via a one-vs-many softmax cross-entropy loss . However, conventional CL is sensitive to how many negative samples are included and how they are selected. Proposed in this paper is a doubly CL strategy that contrasts positive samples and negative ones within themselves separately. We realize this strategy with contrastive attraction and contrastive repulsion (CACR) makes the query not only exert a greater force to attract more distant positive samples but also do so to repel closer negative samples . Theoretical analysis reveals the connection between CACR and CL from the perspectives of both positive attraction and negative repulsion and shows the benefits in both efficiency and robustness brought by separately contrasting within the sampled positive and negative pairs . Extensive large-scale experiments on standard vision tasks show that CACR not only consistently outperforms existing CL methods on benchmark datasets in representation learning, but also provides interpretable contrastive weights , demonstrating the efficacy of the proposed doubly contrastive strategy .
R
has achieved remarkable success
is effective
A
, where the encoder needs to contrast each positive sample over multiple negative samples via a one-vs-many softmax cross-entropy loss
R
the conventional CL loss
conventional CL
R
This paper proposes contrastive conditional transport (CCT) that defines its CL loss over dependent sample-query pairs, which in practice is realized by drawing a random query, randomly selecting positive and negative samples, and contrastively reweighting these samples according to their distances to the query , exerting
Proposed in this paper is
A
doubly CL strategy that contrasts positive samples and negative ones within themselves separately. We realize this strategy with contrastive attraction and contrastive repulsion (CACR) makes the query not only exert a
R
both pull
attract
R
towards the query and push
but also do so to repel
D
away from the query
R
shows that this unique contrastive reweighting scheme helps in the representation space to both align the positive samples with the query and reduce the mutual information between the negative sample and query
reveals the connection between CACR and CL from the perspectives of both positive attraction and negative repulsion and shows the benefits in both efficiency and robustness brought by separately contrasting within the sampled positive and negative pairs
R
CCT
CACR
A
CL
R
contrastive representation learning
representation learning,
R
and latent representations. PyTorch code will be provided
, demonstrating the efficacy of the proposed doubly contrastive strategy