Variational Supervised Contrastive Learning
Authors: Ziwen Wang
Jiajun Fan
Thao Nguyen
Heng Ji
Ge Liu
Venue: NeurIPS 2025 poster ()
Keywords: Contrastive Learning
Variational Inference
Representation Learning
Evidence Lower Bound
Confidence-Adaptive Temperature
Image Classification
Temperature control
Primary area: deep_learning
TL;DR: Variational supervised contrastive learning maximizes a posterior-weighted ELBO, replacing pairwise comparisons with class-level interactions for SOTA performance on image classification tasks.

Abstract: Contrastive learning has proven to be highly efficient and adaptable in shaping representation spaces across diverse modalities by pulling similar samples together and pushing dissimilar ones apart. However, two key limitations persist: (1) Without explicit regulation of the embedding distribution, semantically related instances can inadvertently be pushed apart unless complementary signals guide pair selection, and (2) excessive reliance on large in-batch negatives and tailored augmentations hinders generalization. To address these limitations, we propose Variational Supervised Contrastive Learning (VarCon), which reformulates supervised contrastive learning as variational inference over latent class variables and maximizes a posterior-weighted evidence lower bound (ELBO) that replaces exhaustive pair-wise comparisons for efficient class-aware matching and grants fine-grained control over intra-class dispersion in the embedding space. Trained exclusively on image data, our experiments on CIFAR-10, CIFAR-100, ImageNet-100, and ImageNet-1K show that VarCon (1) achieves state-of-the-art performance for contrastive learning frameworks, reaching 79.36% Top-1 accuracy on ImageNet-1K and 78.29% on CIFAR-100 with a ResNet-50 encoder while converging in just 200 epochs; (2) yields substantially clearer decision boundaries and semantic organization in the embedding space, as evidenced by KNN classification, hierarchical clustering results, and transfer-learning assessments; and (3) demonstrates superior performance in few-shot learning than supervised baseline and superior robustness across various augmentation strategies.

Variational Supervised Contrastive Learning
Ziwen Wang" University Illinois Urbana-Champaign
Jiajun Fan University of Illinois
Thao Nguyen
University of Illinois
Uirbana-Champar jiajunf3eillino edu
TribanaCChappaige chaotn28illinoe edu
ciwen20illinois..
Heng Ji
Ge Liu
University of Illinois
University of Illinois Chhapaa-Champaign geliu@illinois edu
Oabana-Chaamaigee
hengj Lingji@illln ieliiiiillniiiii edu
Abstract
Contrastive learning has proven be highly efficient and adaptable in shaping representation spaces across diverse modalities by pulling similar samples together and pushing dissimilar ones apart. However, two key limitations persist: (I) Without explicit regulation of the embedding distribution, semantically related instances can inadvertently be pushed apart unless complementary signals guide pair selection, and (2) excessive reliance on large in-batch negatives and tailored augmentations hinders generalization. To address these limitations, we propose Variational Supervised Contrastive Learning (VarCon), which reformulates su- pervised contrastive learning as variational inference over latent class variables and maximizes wosterior-weighted evidence lower bound (ELBO) that replaces exhaustive pair-wise comparisons for efficient class-aware matching and grants fine-graines control over intra-class dispersion in the embedding space. Trained exclusively on image data, our experiments on CIFAR-I0, CIFAR-I00, ImageNet- 100, and ImageNet- 1K show that VarCon (1) achieves state-of-ther of-the-art performance for contrastive learning frameworks, reaching 79.36% Top- accuracy on ImageNet- CIFAR-100 with ResNet-50 encoder while converging in just 200 epochs; (2) yields substantially clearer decision boundaries and seman- tic organization in the embedding space, evidenced by KNN classification, hierarchical clustering results, and transfer-learning assessments; and (3) demon- strates superior performance in few-shot learning than supervised baseline and superior robustness across various augmentation strategies. Our code is available httph:://// http:::///gtubt /ttg//ttttut//../ com/ http:///iwwwaggg///// httm//iieeeaagg2//rrrrtttaa /aa.s////taaaaatttt...om
IK and 78.29%
Introduction
Ever since its introduction, contrastive learning has become central paradigm in representation learning, enabling advances across computer vision, natural language processing (NLP), speech, mul- timodal understanding, and applications in natural sciences [62, 65. 24. 64, 3. 41, 35). Foundational models such as SimCLR [10], MoCo [31], BYOL [29], and the fully supervised SupCon [37] have enabled capabilities ranging from zero-shot image classification to state-of-the-an sentence embed- dings, demonstrating that the simple "pull-together, push-apart" principle scales effectively across domains. Despite their empirical success, these objectives function as heuristic energy functions with opaque statistical meaning. While recent analyses have made progress [32, 46, 53, 47, 61]. we still
"Corresponiing author.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

VarCon Torst features,, labels3: normalize features LT noesed Lf nerrailles
reatures nora rormlteeefestures
par-class controids
Wdog
centroids get controlds(feettun festures, labels)
calculate logits and conf idfence scores Logsts features centroids.1) tomperature toftaud get_conf iidence for, labelstp,
Ximg
(x)
Zuna
Vartonn
dycanie resperature based on dence
Weat
Lspel teaps = t (t,na t.ein] cont
target labele)
q_5oft soften d listr ibutiond ag hannd, [abps]t
temps]
rital losss KL divergence neg Tog 1iket thood return KL. divElogits, a a.sorc) avg. lag probrlogits, labelsy
Input images are processed through an encoder network to produce fz-normalized embeddings z. Class-level centroids w,, are computed dynamically from mini-batch embeddings. The model determines sample's 's classification difficulty and applies confidence-adaptive temperature scaling (2(2). which tightens constraints challenging samples and relaxes them for well-classified examples. Right: Pseudocode implementation of our ELBO-deriver loss function combining KL divergence and negative log- ong-likelihod terms.
Figure 1: VarCon architectural flowehart and Pseudocode. Left:
lack principled account how contrastive interactions shape embeddings that capture relational structure among data samples. This gap motivates our work to establish a rigorous foundation for contrastive learning through variational inference.
Generative models provide a complementary perspective: they introduce latent variables and estimate parameters via maximum likelihood, assigning explicit probabilistic semantics the representation space [51]. In Variational Autoencoders (VAEs) [38] and flow-based architectures [42, 12, 22, 60), Euclidean proximity in the latent space corresponds to regions of high data likelihood, endowing distances with a rigorous statistical interpretation. Although contrastive learning is not inherently likelihood-basee based, recent theory frames InfoNCE as density-ratio maximization: it optimizes the ratio of joint probability over marginal probabilities for positive pairs relative to negatives, prioritizing genuine pairs. Both paradigms ultimately seek embeddings where "near" implies "probable," though via different routes generative models integrate over latent variables to maximize likelihood, while contrastive methods directly sculpt the space through positive- negative comparisons. This concep- tual convergence invites our unified framework, where variational inference provides probabilistic grounding for contrastive objectives while preserving their proven geometric inductive biases.
In this work, we present Variational Supervised Contrastive Learning (VarCon), a probabilistically grounded framework that preserves the geometric structure of conventional contrastive objectives while endowing them with explicit likelihood semantics. VarCon treats the class label as latent variable and maximizes a cesterior-weigheee evidence lower bound (ELBO), replacing hard labels with soft probabilities reflecting the model's current belief. This formulation confers three key benefits. First, each embedding interacts with single class-level direction computed on the fly, reducing computation from quadratic to nearly linear in batch size. Second, it employs confidence- adaptive temperature that tightens pull strength for hard samples and relaxes for confident ones, providing fine-grained control over intra-class compactness. Third, we develop novel objective with two synergistic terms: a Kullback-Leibler divergence aligning the auxiliary posterior with the model's is class posterior, and negative log-likelihhe likelihood term for the ground-truth label. This approach simultaneously aligns distributions, maximizes class likelihood, and prevents representational collapse. Our main contributions are summarized as follows:
1. We propose a novel formulation of contrastive learning in variational inference with an explicit ELBO framework, deriving the VarCon loss that explicitly regulates embedding distributions and enforces appropriate semantic relationships between samples.
2. We propose a confidence -adaapivee temperature scaling for label softening strategy that pushes the edge of learning hard positives and negatives, proven through gradient derivation analyses.
We advance state-of-the-art contrastive learning performance across multiple architectures, achiev- ing 79.36% Top-1 accuracy ImageNet with ResNet-50 (vs. 78.72% for SupCon) and 81.87% with

---

ResNet-200, while converging in 200 epochs versus SupCon's 's 350 and using smaller batch sizes (2048 vs. 4096).
4. Our learned representations demonstrate superior semantic organization with clearer hierarchical clustering, and maintain strong performance in low-data settings and various augmentation strength.
2 Preliminaries
Contrastive Learning. Early metric-learning objectives, the pairwise contrastive loss [13], triplet margin loss [54], and N-pair loss [56), laid the groundwork for modern contrastive frameworks. In contrastive learning, given an input sample æ with ground-truth class label an encoder maps to an embedding RR typically fg-normalized. The field subsequently converged on noise-contrastive formulations like InfoNCE [46]:
exp(z] 2,/T7)
LIafNNCE = - llog E 1 [e7]] exp(z, xx/T)
where 2; and 2 are positive pairs and 5 is a temperature parameter. InfoNCE drives self-supervised systems like SimCLR [10] and MoCo [11, 31], and scales to modal setups in CLIP [48]. For supervised learning, SupCon [37] extends this by treating same-class samples as positives:
exp(z) (2] (//2p/T)
Suppoon
log PEP(()
iET P())
Eneste exp(z, Zn/T)
where P(t) contains samples of the same class as i, and A(i) includes all samples except i. Recent theoretical advances [68, 34] and methods like X-CLR [55] further improve these frameworks by analyzing optimization dynamics and introducing graded similarities. Our approach builds on these foundations by reformulating supervised contrastive learning through variational inference while preserving semantic linkages across classes. Additional related work is provided in Appendix E.
Variational Inference. Variational methods provide principled probabilistic frameworks for learning latent representations by maximizing an evidence lower bound (ELBO). Variational Autoencoders (VAEs) [38, 50] pioneered this approach with amortized encoder-decoder architectures learning continuous latent spaces. Later advances improved flexibility through normalizing flows [39, 49) and learned priors, while Bayesian approaches like Beayes-by-Backjjp [7] and Monte-Carlo dropout [23] incorporated parameter uncertainty. Recent work has integrated contrastive learning with variational objectives: Coitt-Coote Prior [2] separates between posteriors and priors to tighten latent fit; variational crfornation-bottleneckk augments SimCLR [59]; and Recognition-Parammttriiie Models reformulate the ELBO to yield InfoNCE-style objectives [1]. These integrations create latent spaces that are simultaneously generative, predictive, and uncertainty- aware.. Our approach builds on these foundations by maximizing a class-conditional data likelihood with label-softened targets, preserving contrastive learning's ' geometric properties while incorporating explicit probabilistic semantics.
Soft Labeling. Soft labeling replaces hard one-hot targets with distribution-bart supervision, first popularized through label-smoothing in Inception-v3 [57] and knowledge distillation [33]. Recent advances employ soft labels to improve representation learning [21. 20]: for noisy targets, smoothed distributions enhance calibration across facial expressions [43] and enlarge inter-class margins in partial-label settings [28]. This concept extends remi-supervised learning where ESL [44] treats uncertain pixels as soft pseudo-labels. SoftMatch [9] applies confidence weiegteewwedddde pseudo-labelo and ProtoCon [45] refines labels through online clustering. For cross-modal learning, SoftCLIP [25] strengthens image-text embeddings by weighting related captions. Our approach employs soft labeling as a aanfidence-adaptiiee variational distribution that dynamically adjusts class distributions, more uniform for confident samples and more peaked for challenging ones, to enhance representation learning through targeted supervision.
3 Method
Throughout this study, we use the following notation: Let I&d denote the fz-normalized embedding produced by the encoder (e.g., ResNet-50) for an input sample x with ground-truth class index E {1...., C). Each class is represented by unit-norm reference vector w, e Rd
3

---

2 where 2, 1B,1-IE 2; and B, = { = r; r, i E B denotes the index set of samples from class in mini-batch B. We set 71 be the fixed temperature that scales the logits to adjust the sharpness of the class distribution and T2(2) the confidence-adappe addativve temperature that softens each sample's target distribution to regularize learning and control intra-class dispersion. With 0 being the set of learnable weights of the encoder, pe(r z) denotes probability the model assigns to the correct class for embedding z, po(z rr) the class-conditional likelihood of observing 2, and p=(=) = Σr' Po ( r')p(r") with the dummy class index, gives the marginal embedding density used in the variational derivation. We introduce (r z) as a confidence-uee andidence-adaptive label distribution parameterized by Ø, which plays the role of an adaptive temperature for accentuating the probability assigned to the ground-truth class while allocating the remaining mass uniformly across the other C' classes.
obtained by normalizing the class mean:
3.1 The Variational Bound of Class-Conditional Likelihood
To formulate variational approach, need establish evidence lower bound (ELBO) for the class-conditional likelihood of embeddings. We begin with Bayes' rule [26, 6], which allows us to express the class-conditional likelihood in terms of the posterior probability and marginal densities:
Po(r 2) pz(z) p(r)
pa zz
(r)
(1)
Taking logarithms on both sides gives us:
log Pa(z |r) Po 22) + loggo(=) llgg p(r).
(2)
The marginal embedding density Pz( integrates over all possible class assignments. By introducing
our auxiliary distribution 90 (r' 2) that sums 1, and applying the identity po(z) Σ,, Po r') "(("), we can rewrite the marginal density as:
(2 = Σ po(z rr')((r)) ga(r') (2)
(3)
go(r' 2))
Substituting Eq. (3) into Eq. (2), we obtain:
(o r) = log x) + (z) logp(r)
C
p= p(') 90(r') x))
log x) + log
- log
go(r" x)]
(4)
(r 2x) po(z r') p(r')
logpe(r 2) log Σ
log p(r).
gr(r' 2)
From Eq. (I), we know that (z () p() po(r (z). Using this relation to substitute for
po(z Ir') p(r") in Eq. (4), and applying Jensen's 's inequality to the logarithm of a sum (since log is concave function), we derive the ELBO for log pa (z r):

---

) po(2 [2] r²)) p(r')
logpo(z r)) log 22) + log Σ (o(r' 1 12)
log p(r)
(r(r' 2)
1o(r' 1 = =) p)())
= =logpo(r]
2) + log
Σ
loo p(r)
(r' 2)
p/=l
qs(r' 2) po (r' 22) p)(2))
2 logpo(r 2)) Σ
logp(r)
2o(r' z)
r'=l
logpo(r) 12) Σ
90(r'
2) Mlogpo(r'
2) log po()) log go(r') 22] logp(r)
(5)
rlll
= logpe(r) |z) Σ
go(r'1
z) log po(z)
r==1
c
+
Σ
go(r' (x) [log (o x) log go x)] - p(r)
r=
C
log 2) + log po (z)
9o(r') 12) - DRa(us(r' 22) po(r' |=)) - log
r==1 (r'(go(r') x) po 2)) p(r).
= Pogp x) + log pa (x)
This derived bound consists of several interpretable components: (1) encouraging the model to correctly classify the input embedding, (2) the log marginal probability of the embedding, (3) KL divergence that aligns our auxiliary distribution with the model's class posterior, and (4) a constant class prior. This ELBO serves as the foundation for our variational contrastive learning objective.
3.2 Variational Contrastive Learning
Having derived the ELBO for log po(= | in Eq. (5), observe that under contrastive learning settings, certain terms are either intractable or uninformative: log log (z) would encourage high likelihood throughout the embedding space without class distinction, while lor((r) is fixed constant based on the dataset's 's class distribution. Therefore, instcad of directly maximizing the full ELBOO:
LELBO = Dxi(go(r' z) (r =)) log - log po log(o
(6)
we focus on minimizing the contrastive-relevant components:
LvarCon = DKL (r' x) (o(r') 2)) lo x))
(7)
This formulation balances two complementary objectives: the KL divergence term aligns our auxiliary distribution with the model's predictive distribution, while the log pog-pottrrior term encourages correct class assignments. To compute class probabilities efficiently, we leverage class centroids rather than pairwise comparisons. For each class we calculate reference vector where B, {i r; T, B contains indices of samples with class in batch B. The posterior probability then:
|B,| 18,1- EiEB, 2i
exp(z Tw/TT) po(r 2) = Σ,- exp(x> "w,/TT)²
(8)
with logarithm:
exp(x exp(xTur/TI)
w//T1)
x w. log Σ exp
z W.
log (r x) log
(9)
exp(z "w,//// [TT]
TI
Ti
where is a fixed temperature parameter. For the target distribution go. we start with one-hot distribution:
I, if
Qone-bool (r' 1 2) 0, otherwise.
(10)
5

---

We then apply adaptive softening using temperature 72:
loxp (r 2) + (exp(1/75) ) 1] gone hot (r 2),
(11)
and normalize to obtain the final distribution:
gexp (r x)
(r(r z) En 2oxp (r' 2]
(12)
key innovation in our approach is using a confidence-adaptive temperature 72 that varies between
bounds 71 E and 71 +
(Ti -) + 2epa(r =),
(13)
where is learnable parameter. When po [r] 2) is high (confident prediction), 72 approaches ti + making (r(r) z) more uniform. Conversely, for difficult samples with low pa 2). 72 approaches
TI 6, creating a sharper distribution. This adaptive mechanism dynamically adjusts supervision intensity-relaxing constraints
well-classifedd samples while focusing learning challenging ones (see gradient derivation C.I). By minimizing Lvaroon we simultaneously control intra-class dispersion through the KL term and enhance the distinguishing power of embeddings through the posterior term, creating a representation space with clear semantic structure and decision boundaries.
Experiments
We evaluate our VarCon loss four standard benchmarks: CIFAR-10, CIFAR-100 [40], ImageNet- 100, and ImageNet [52, 18], using the official test splits. Our experiments demonstrate rapid convergence under various training configurations, including different batch sizes, epochs, and hyperparameter settings. For downstream classification, we freeze the pretrained encoder and train only a linear classification layer, and additionally apply KNN-classifier to the embeddings to investigate the learned semantic structure. To isolate the contribution of our learning objective, we employ the same data augmentation techniques and encoder architectures used previous contrastive learning models [37]: augmentation strategies including SimAugment, AutoAugment [14], and StackedRandAAggment [15] with ResNet-50 [30], ResNet-101,
2000, and ViT-Base [19] architectures. We set the learning rate according to the thear-scaling rule Ir о B B/256 with cosine decay. Our best results are achieved with batch sizes of 512 for CIFAR- (200 epochs), 1,024 for ImageNet-100 100 (200 epochs), and 4,096 for ImageNet (350 epochs). Throughout, we use SGD) with momentum 0.9 and weight decay 10-4 for smaller datasets including CIFAR-10 10, CIFAR-100, and ImageNet- 100 and LARS [63] optimizer for ImageNet training to ensure stability. Full hyperparameter settings and optimization details are available in Appendix B. Analyses of the training dynamics, such as loss convergence and memory efficiency, are provided in Appendix D.
4.1 Classification Performance
To evaluate our proposed VarCon, we conducted extensive experiments on multiple benchmark datasets for image classification. Table 1 presents performance comparisons against statc-of-the-a
self-supervised methods (SimCLR [10], MoCo V2 [31, 11], BYOL [29], Sw [8], VicReg [5], and Barlow Twins [66]) and supervised approaches (standard Cross-Entropy and SupCon [37]). VarCon consistently outperforms all competing methods across all datasets. Compared SupCon, VarCon achieves 0 0.43% higher Top-1 accuracy on CIFAR- 10 (95.94% vs. 95.51 1)), 1.72% higher on CIFAR- 77% vs. 76.57%) ), 1. 1.28% higher on ImageNet- 100 (86.34% vs. 85.06%), and 0.64% higher on ImageNet (79.36% vs. 78.72% The advantage is more pronounced when compared to self- supervised methods. On ImageNet-1066 VarCon surpasses the best self supervied method (Barlow Twins at 80.83%) by 5.51%, and ImageNet, it outperforms all self- 4.07% in Top-1 accuracy. VarCon also demonstrates superior performance on ImageNet- ReaL (see B.2), re-annotation with more accurate multi-label ground truth, achieving 84.12% Top-1 accuracy compared to SupCon's 83.87% They validate that by explicitly modeling feature uncertainty through our BerO-derived loss function, VarCon learns more well-defined, generalizable representations.
methods by at least
4.2 Few-Shot Learning Performance
Given the practical importance of learning from limited labeled data, we evaluate VarCon in few-shot learning scenarios by comparing with SupCon across different per-class sample sizes. Table

---

Table 1: Classification performance comparison across benchmark datasets. We report Top-1 I and Top-5 accuracy (%) (mean + standard eттor) for VarCon versus state-of-the self self-supe and supervised methods. All models utilize the ResNet-50 architecture for a fair comparison. Best scores are highlighted in blue second-best in green
Dataset
Categury Methood
CIFARIO
CIFAR100
ImtgeNNtt- 1000
ImageNet
Top-1
Top-5
Top-I f
Top-5
Top-1 f
Top-5
Top-1
Top-5
SimCLR
91.52
99. .9.78_...1
70:67 70.67.0.13
92.01,0a:
71:54,0.00
91. 50
70.51,cas
90.37
Superseal Self-upertted
MoCo
82. ge
98. 0.00
70.01 o0
91.08
78 98
95 20
g1.06
90. 90.400.0.06
BYOL
a2.57
99.71
10.50a0.11
g1.05
80, 80.18
94 1600
74.28
91. 16 6..0
SwAV
69.14g
90.00 090.99_...
64. 11.77.m
88.81 88.81,005
74.07
52. 77
75.29 go.ar
91.881_00
VieReg
92. 20000.
51
20.01
26
95 06
79. 79.25
01.06.0..0
Barlow Twins
92.70 a0
99. pu.0s
71 12.022
91.95 91.95
80. 80.80a
56, 24gaum
73. 2a.20
91. 10s00
Cross-Farropy
96. goues
90 20.
24.0
g1.89
83.17
55. 78
78. goue7
98.77 20.04
SupCoo
96.51ac.e
89.80
76.57a0.1
93.50 98.50
85. 86.06anor
56. 84mnma
78.72ac.a
94.31:
VarCon (Ours)
955 gooor
78 78.29ga.0 28.
98. goucs
86.344m00
96 go
79. g9.30
94.37
Table 2: Few-shot learning performance on ImageNet with varying training data availability. We report Top- accuracy (mean + standard error) on ImageNet using ResNet-50 for both SupCon and
our proposed VarCon method across different per-class sample sizes (N). Best highlighted in blue.
Method
N 50
N 100
N 200
N 500
N = 700
N 1000
SupCon
2.47
36.57
50,25
70.1220.16
23.04
VarCon (Ours)
2.53
37.8140.2
51.10
60.83
70.61
=3.21
presents this comparison on ImageNet using ResNer-50, with sample sizes ranging from extremely limited (NV 50) to moderate (N 1000). Results are averaged across five random subsets per configuration to account for sampling variability. VarCon consistently outperforms SupCon across all sample sizes, with advantages more pronounced in data-scarce settings: With 100 samples per class, VarCon achieves 37.81% Top-1 accuracy, surpassing SupCon (6.57%) by 1.24%. This advantage persists N = 200 (51.10% vs. 50.25%) and N 500 (65.83% vs. 64.91% 1)). As training data increases, VarCon maintains superior performance, though the gap narrows slightly (73.21% % vs. 73.04% N 1000). These results suggest our variational framework provides greatest benefit when training data is scarce, where robust representation learning is most challenging. By modeling feature uncertainty explicitly, VarCon learns more effective representations from limited data, making it well-suited for real-world applications where large labeled datasets are often unavailable.
4.3 Effect of Data Augmentation and Encoder Architecture
To investigate the robustness of VarCon across different neural architectures and data augmentation strategies, we conducted experiments on ImageNet using various combinations, as shown in Table VarCon consistently outperforms SupCon across all tested architectures. With ResNet-50 and AutoAugment, VarCon achieves 79.36% Top-1 accuracy, surpassing SupCon (78.72%) by 0.64%. This advantage persists with ResNet- (80.58% vs. 80.24%), ResNet-200 (81.87% vs. 81.43%), and extends to the fundamentally different architecture of ViT-Base (78.56% vs. 78.21%). Importantly, VarCon demonstrates strong performance even with simpler augmentation strategies. Using only SimAugment with ResNet-101, VarCon achieves 79.98% Top-1 accuracy, which is competitive with (80.24%). This reduced dependency on
SupCon using the more complex StackedRand sophisticated, task-specific augmentations represents a significant practical advantage, as identifying optimal augmentation strategies often requires extensive tuning and domain expertise.
4.4 Transfer Learning Performance
A critical measure of representation quality is how well features transfer to new tasks and domains. Table 4 presents transfer learning results across 12 diverse datasets spanning fine-grained recognition, scene classification, and general object recognition tasks. While supervised contrastive learning typically faces challenges in transfer learning due to supervised signals potentially limiting rep- resentation generality, VarCon demonstrates improved transferability compared to SupCon. With
7

---

Table 3: Adaptability analysis across architectures and augmentation strategies on ImageNet. We evaluate VarCon against SupCon using multiple encoder backbones (ResNet-50/ 50/101/200 and ViT- Base) and augmentation techniques (SimAugment, AutoAugment, and StackedRandAugmenn)) The highest performance values for each encoder architecture are highlighted in boldface.
Loss
Architecture
Feat. Dim
Params
Augmenlation
Top-1
Top-5
SupCon
ResNet-50
2048
25.6
SimAugment
77.82
93.61
SupCon
R5
2048
AutoAugment
78.72
94.27
VarCon (Ours)
ResNet-50
2048
25.6
SimAugment
78.23
93.67
VarCon (Ours)
ResNer-50
2048
25.6 M
AutoAugment
79.36
94.33
SupCon
ResNet-101
2048
SimAAgment
79.64
94.85
SupCon
ResNet-101
2048
44.5
StackedRundAugmeent
80.24
94,82
VarCon (Ours)
ResNet-101
2048
44.5 M
SimAAugmet
79.98
94.89
VarCon (Ours)
ResNet-101
2048
44.5 M
StackelRandAAugment
80.58
94.87
SupCon
ResNet-200
2048
65
StackedRundAAugmentt
81.43
95.93
VarCon (Ours)
ResNet-200
2048
65 M
StackedRandAugnen
81.87
95.95
SupCon VarCon (Ours)
ViT-Base
768
SimnAggmnn
78.21
94.13
ViT-Base
768
86
SimAugment 78.56
94.17
Table 4: We compare the transferability of representations learned by VarCon against SimCLR RR and SupCon. Metrics reported are mAP for VOC2007, mean-per-class accuracy for Aircraft, Pets, Caltech-101. and Flowers, and top-1 accuracy for remaining datasets.
generalization evaluation across 12 diverse visual recognition benchmarks.
Method
Food
CIFAR10
CIFARI00
Birdsnap SUN397
Cars
Aircraft
VOC2007
DTD
Pets
Caltech-101
Flowers
Mean
SimCLR-50
88.21
97.62
85,86
75.91
63.52
91.33
87.40
83,98
73.23
89.22
92.11
97.02
85.45
SupCon-50
97.43
84.26
75.20
91.71
84.08
93.45
91.07
96.08
$4.88
55
87.31
97.65
84.39
75.27
57.96
91.79
84.25
85.07
74.79
93.52
91.19
96.14
84.94
SupCon 200
98.32
87.25
60.44
91.83
88.53
74.62
93.09
94.87
96.94
$6.33
VarCon-200
88.68
98.34
87.30
76.29
60.39
91.85
88.61
65.02
74.47
93.26
94.95
96.96
86.34
ResNet-50, VarCon achieves a mean accuracy 84.94% across all datasets, outperforming SupCon (84.88%) and approaching the self-supervise SimCLR (85.45%) VarCon-50 surpasses SupCon-50 on several datasets, including CIFAR-10 (97.65% vs. 97.43%) Cars (91.79% vs. 91.71 1)), and Pets (93.52% vs. 93.45%). With 2esNet- VarCon achieves mean accuracy of 86.34% and outperforms SupCon-200 on out of 12 datasets. These results validate that VarCon learns a more adaptable embedding space that generalizes better across diverse visual domains. Our variational ELBO mechanism effectively mitigates potential overfitting to source domain labels, leading to more transferable representations while maintaining strong performance the source task.
4.5 Ablation Studies
Effect of Temperature. The temperature parameter in contrastive learning significantly impacts model performance by controlling the concentration of the distribution in the embedding space. Figure 2a shows T accuracy on ImageNet for both VarCon and SupCon across temperatures from 0.02 0.14. Both methods achieve optimal performance at 7 0.10, with VarCon reaching 79.38% accuracy compared to SupCon's 's 78.8%. However, VarCon exhibits much greater robustness to temperature variations, particularly in higher temperature regimes. While SupCon's is accuracy drops sharply after - 00 (Top-1 Accuracy declining by 2% at = 0.14), VarCon maintains stable performance up to т = 0.12 before showing a comparable decrease. This enhanced stability can be attributed to VarCon's 's adaptive temperature mechanism (72), which dynamically adjusts based on sample confidence, providing an additional layer of robustness against suboptimal temperature settings.
Number of Training Epochs. Figure 2b shows the Top-1 accuracy on ImageNet for VarCon and SupCon across different training durations. VarCon demonstrates consistently superior performance and faster convergence throughout training. At just 50 epochs, VarCon achieves 75.3% accuracy (vs. SupCon's 's 74.5%), reaching 77.85% by 100 epochs and 79. 79.15% by 200 apochs-alreade approaching its peak performance. VarCon achieves optimal accuracy of 79.36% at 350 epochs and maintains better stability during extended training, with less performance degradation after 700 epochs (78.80%
8

---

Imggeee Temperskur
Tope: www.en/////eeeeeee..
VerCoe
SacCen
Close beikey
AM
20
30
500
Daleh
400
500
000
(a)
(b)
Figure 2: (a) Top-1 accuracy on ImageNet versus temperature parameter; (b) Top-1 accuracy on ImageNet versus training epochs; (c) Top-1 accuracy on ImageNet versus batch size.
Accunacy Irageékel
Tre-1 Acausacy on trasgohet vn Comptt Severty
754
inspeleet
80.
Suptian
Supion Flaahee c0
Vax Reclae San
150
K.Neanod hiightors of
000
Comption Serveity
(a)
(b)
(c)
Figure 3: (a) KNN classifier accuracy on ImageNet embeddings; (b) Effect of adaptive temperature parameter on ImageNet Top-1 accuracy; (c) Robustness evaluation on ImageNer-C across different corruption severity levels.
vs. 78.23% for SupCon). These results show that our variational approach both converges faster and provides enhanced robustness against overfitting.
Batch Size Sensitivity Analysis. Figure 2c illustrates the effect of batch size on ImageNet Top-1 accuracy. While large batches are typically crucial for contrastive learning. they significantly increase computational requirements and can lead training instability. VarCon demonstrates reduced dependency on large batch sizes-with just 2048 samples, it achieves 79. 79.26% accuracy, already outperforming SupCon at 4096 (78.7%). VarCon's performance plateaus earlier, showing minimal improvement (0.01% () when increasing from 4096 to 6144, while SupCon continues to benefit from larger batches. This indicates that VarCon learns a more effective embedding space with fewer negative examples, enabling efficient training with more limited computational resources.
Analysis of Embedding Space Organization. To directly evaluate embedding quality without additional parameterized classifiers, we employed K-nearest neighbor classification, which better reflects the intrinsic structure of the feature space. Figure 3a shows that VarCon consistently outperforms SupCon across all K values on ImageNet. For Top-1 accuracy, VarCon's 's advantage increases with larger values (from 77.4% vs. 77.32% at K=5 to 78.07% vs. 77. 77.86% at K 80). For 5op-5 accuracy, VarCon maintains a consistent 0.27% advantage (83.88% % vs. 83.61 %)) across all values. This superiority confirms that VarCon produces embedding spaces with clearer decision boundaries and better structured class relationships. This improved embedding organization is further confirmed by t-SNE visualizations of the embedding space evolution in Appendix B.3 and by quantitative hierarchical clustering metrics in Appendix B.4.
Effect of Epsilon Parameter. In VarCon, we introduce an adaptive temperature mechanism where T2 varies within bounds determined by 71 + c. Figure 3b shows the effect of different values on ImageNet Top-1 accuracy when T1 0.1. At = (equivalent fixed temperature), the model achieves 78.87% accuracy. Performance improves
increases, peaking at 79.36% with e 0.02, before declining to 78.42% E = 0.08. This indicates that while some temperature adaptability benefits learning, excessive deviation becomes detrimental, with 0.02 providing optimal flexibility to adjust confidence levels based on sample difficulty.
4.6 Robustness to Image Corruption
To evaluate robustness to real-world image degradation, we tested VarCon on ImageNet-C, which applies 15 different corruption types at increasing severity levels. Figure 3e shows Top-1 accuracy on clean ImageNet ("test") and across all corruption severity levels. VarCon consistently outperforms

---

Distribution of Adaptive Temperature (Ta) Key Epochs
100
Epoch
Evolution Adaptive Temperature (Ty) on ImageNet
Epoch 20
Fpoch
0.105
80
Epoch 40
Average VValue
Epoch 50
Density
0.095
0.090
0.085
Confidence-adaptive Temperature (72)
0.080
20
30
50
0.000 0.005 0.090
0.095
0.100 0.105 0.110 0.115 0.120
Training Epochs
Value of F2
(a) Mcan T2 evolution
(b) T2 density distributions
Figure 4: Evolution of adaptive temperature 72 during full 50-epoch ImageNet training with ResNet-
50 (c = 0.02, -1 = 0.1, batch size 4096). (a) Mean 72 increases from 0.09378 to 10656 over the complete training, indicating systematic confidence growth. (b) Density distributions (epochs 10-50) show rightward shift with initial broadening (std: 0.00961 + 0.01092) then stabilization, reflecting heterogeneous confidence development as the model distinguishes easy from hard samples.
SupCon with both ResNet-50 and ResNet-200 architectures. With ResNet-50, VarCon maintains a performance advantage ranging from 0.3% to 1.2% as corruption severity increases from level 1 to 5. This graceful degradation pattern can be attributed to our probabilistic framework, which explicitly models feature uncertainty. By representing features as distributions rather than points, VarCon better accommodates input variations caused by corruptions. These results highlight that variational approach not only performs well on clean data but also demonstrates enhanced robustness to low-quality inputs, which valuable property for real-world applications.
4.7 Adaptive Temperature Analysis
To investigate our aeffidence-adaptive temperature mechanism, we analyze 72 distribution across ImageNet samples during ResNet-50 training (batch size 4096, 72 E [0.08, 0.12)). We fix 0.02 and тi = 0.1 to ensure comparable temperature spans across epochs, though e is learnable in our best configuration. Figure 4 shows systematic evolution: initially (epoch 10), over 25% of samples receive near-maximan gradients with 72 near lower bound, indicating widespread uncertainty. Mean T2 increases 13.6% during training while maintaining persistent challenging samples near minimum. This heterogeneous evolution demonstrates differential treatment, strong gradients for difficult samples and relaxed constraints for well-learned ones, emerging naturally from KI divergence and confidence-based temperature interaction. This demonstrates our self-adaptive temperature simultaneously adjusts both target sharpness and gradient magnitude based per-sample confidence (see Appendix D.4 for comparisons with other regularization methods [57, 67, 27, 58, 4, 36]). Detailed T2 evolution statisties for fixed (Table 7) and learnable (Table 8) in Appendix D.2.
5
Conclusion
In this work, we introduced Variational Supervised Contrastive Learning (VarCon), probabilistically grounded framework that reformulates supervised contrastive learning through variational inference over latent class variables. By deriving and maximizing posterior posterior-weighted evidence lower bound, VarCon overcomes key limitations of conventional contrastive approaches: explicitly regulates embedding distributions through principled KL divergence, replaces exhaustive pairwise comparisons with efficient class-level interactions, and employs confidence-adaative temperature scaling to control intra-class dispersion. Our extensive evaluation demonstrates VarCon consistently outperforms leading contrastive methods across diverse benchmarks, achieving superior classification accuracy CIFAR- 10/100 and ImageNet while requiring fewer training epochs. Moreover, VarCon exhibits enhanced robustness to hyperparameters, reduced dependency on large batches, stronger low-data performance, and improved corruption resilience, all while maintaining better cross-domaie transfer- ability. Beyond empirical advantages, VarCon bridges the theoretical gap between distinguishing and generative paradigms by endowing contrastive objectives with explicit likelihood semantics.
10

---

References
[T] Laurence Aitchison and Stoil Krasimirov Ganev. InfoNCE is variational inference in a recogni- tion parameterised model. Transactions Machine Learning Research, 2024. ISSN 2835-8856.
URL hooptttt///ppereeiieee chepaarreeeee.eeeeoooooooddddcdr chbhsss
[2] Jyoti Aneja, Alex Schwing, Jan Kautz, and Arash Vahdat. A contrastive learning approach for training variational autoencoder priors. Advances in neural information processing systems, 34: 480-493, 2021.
[3] Alexei Baevski, Yuhao Zhou, Abdelrahman Mohamed, and Michael Auli. wav2vec 2.0: framework for self-supervised learning of speech representations. Advances in neural information processing systems, 33:12449-12460, 2020.
[4] Dara Bahri and Heinrich Jiang. Locally adaptive label smoothing improves predictive churn. In International Conference on Machine Learning, pages 532-542. PMLR, 2021.
[5] Adrien Bardes, Jean Ponce, and Yann LeCun. Vicreg: Variance-iivvaranee-cvvar regular- ization for self-supervised learning. arXiv preprint arXtv:2105.04996, 2021.
[6] Christopher M. Bishop. Pattern Recognition and Machine Learning. Information Science and Statistics. Springer, New York, 2006. ISBN 978-0-387-310734
[7] Charles Blundell, Julien Cornebise, Koray Kavukcuoglu, and Daan Wierstra. Weight uncertainty in neural network. In International conference on machine learning, pages 1613-1622, PMLR, 2015.
[8] Mathilde Caron, Ishan Misra, Julien Mairal, Priya Goyal, Piotr Bojanowski, and Armand Joulin. Unsupervised learning of visual features by contrasting cluster assignments. Advances in neural information processing systems, 33:9912-9922 2020.
[9] Hao Chen, Ran Tao, Yue Fan, Yidong Wang, Jindong Wang, Bernt Schiele, Xing Xie, Bhiksha Raj, and Marios Savvides. Softmatch: Addressing the quantity-quality tradeoff in semi- supervised learning. In The Eleventh International Conference on Learning Representations, 2023. URL https: http///pppeeeeeeeeeee/eeeeeeeeet net www.ee/wwweeeettonnddei//eeeeeee htte//oormm/ddmmmm11ttb----
[10] Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In Hal Daumé III and Aarti Singh, editors, Proceedings of the 37th International Conference on Machine Learning, volume 119 of Pro- ceedings of Machine Learning Research, pages 1597-1607. PMLR, 13-18 Jul 2020. URL .html.
https: https://proceedingg mlr http://nnns....pppee///////o
[11] Xinlei Chen, Saining Xie, and Kaiming He. An empirical study of training self supe-suue vision transformers. In Proceedings of the IEEE/CVF international conference on computer vision, pages 9640-9649, 2021.
[12] Chaoran Cheng, Jiahan Li, Jian Peng, and Ge Liu. Categorical flow matching statistical manifolds. In Annual Conference on Neural Information Processing Systems 2024, NeurIPS 2024, 2024.
[13] Sumit Chopra, Raia Hadsell, and Yann LeCun. Learning similarity metric discriminatively, with application to face verification. In 2005 IEEE computer society conference on computer vision and pattern recognition (CVPR'0S) 005, volume 1, pages $39-546. IEEE, 2005.
[14] Ekin D Cubuk, Barret Zoph, Dandelion Mane, Vijay Vasudevan, and Quoe V Le. Autoaugment: Learning augmentation policies from data. arXiv preprint 18055 09500, 2018.
[15] Ekin D Cubuk, Barret Zoph. Jonathon Shlens, and Quoc V Le. Randaugment: Practical automated data augmentation with a reduced search space. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops, pages 702-703, 2020.
[16] Jiequan Cui, Zhisheng Zhong, Shu Liu, Bei Yu, and Jiaya Jia. Parametric contrastive learning. In Proceedings of the IEFF/CVE international conference on computer vision, pages 715-724, 2021.
11

---

[17] Jiequan Cui, Zhisheng Zhong. Zhuotao Tian, Shu Liu, Bei Yu, and Jiaya Jia. Generalized para- metric contrastive learning. IEEE Transactions on Pattern Analysis and Machine Intelligence. 46(12))443-7477, 2023.
[18] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large- scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pages 248-255. Iece, 2009.
[19] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenbom, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition scale. arXiv preprini arXiv:2010. 1929, 2020.
[20] Jiajun Fan and Changnan Xiao. Generalized data distribution iteration. In Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvari. Gang Niu, and Sivan Sabato, editors, Proceedings of the 39th International Conference on Machine Learning, volume 162 of Proceedings of Machine Learning Research, pages 6103-6184 PMLR, 17-23 Jul 2022. URL https:// mlr. wtt.sesw.....eeeeeeeeeeeeeeeeee. arressv162/fan2222 html.
proceedings.o mml.eddiinn.mm.......mmmm..
[21] Jiajun Fan, Yuzheng Zhuang, Yuecheng Liu, Jianye Hao, Bin Wang, Jiangcheng Zhu, Hao Wang, and Shu-Tao Xia. Learnable behavior control: Breaking atari human world records via sample-efficient behavior selection. arXiv preprint arXiv:2305.05239 2023.
[22] Jiajun Fan, Shuaike Shen, Chaoran Cheng, Yuxin Chen, Chumeng Liang, and Ge Liu. Online rewght-weighted fine-tuning of flow matching with wasserstein regularization. In The Thirteenth International Conference on Learning Representations, 2025. URL
eet/foormmid=21onm ie=iiid
[23] Yarin Gal and Zoubin Ghahramani. Dropout as a bayesian approximation: Representing model uncertainty in deep learning. In international conference on machine learning, pages 1050-1059. PMLR, 2016.
[24] Tianyu Gao, Xingcheng Yao, and Danqi Chen. SimCSE:
Simple contrastive learning of sentence embeddings. In Maric-Francine Moens, Xuanjing Huang, Lucia Specia, and Scott Wen-tau Yih, editors, Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 6894-6910, Online and Punta Cana, Dominican Republic, November 2021. Association for Computational Linguistics. doi: 10.18653/v1/221 122/3/11/2221.. wem.o.eeemnlpppmme... main.552. URL https://aclantholoog org/2021
omnlp-main... 552/
[25] Yuting Gao, Jinfeng Liu, Zihan Xu, Tong Wu, Enwei Zhang, Ke Li, Jie Yang, Wei Liu, and Xing Sun. Softclip: Softer cross-modal alignment makes clip stronger. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 1860-1868, 2024.
[26] Andrew Gelman, John B Carlin, Hal S Stern, and Donald B Rubin. Bayesian data analysis. Chapman and Hal//CRC, 1995.
[27] Arindam Ghosh, Thomas Schaaf, and Matthew Gormley. Adafocal: Calibration-aware adaptive focal loss. Advances in Neural Information Processing Systems, 35:1583-1595, 2022.
[28] Xiuwen Gong, Nitin Bisht, and Guandong Xu. Does label smoothing help deep partial label learning? In Forty-first International Conference on Machine Learning, 2024.
[29] Jean-Bastien Grill, Florian Strub, Florent Altché, Corentin Tallec, Pierre Richemond, Elena Buchatskaya, Carl Doersch, Bernardo Avila Pires, Zhaohan Guo, Mohammad Gheshlaghi Azar, Bilal Piot, koray kavukcuoglu, Remi Munos, and Michal Valko. Bootstrap your own latent a new approach self-supervised learning. H. Larochelle, M. Ranzato, Hadsell, M.F. Balcan, and H. Lin, editors, Advances in Neural Information Processing Systems, volume 33, pages 21271-21284. Curran Associates, Inc., 2020. URL https: httpp://pprooceeiingg neurips cc/ paper files
Paper pdf.
[30] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770-778, 2016.
12

---

[31] Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie. and Ross Girshick. Momentum contrast for unsupervised visual representation learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2020.
[32] Olivier Henaff. Data-efficient image recognition with contrastive predictive coding. In Interna- lional conference on machine learning, pages 4182-4192. PMLR, 2020.
[33] Geoffrey Hinton. Oriol Vinyals, and Jeff Dean. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.0223 2015.
[34] Cheng Ji, Zixuan Huang. Qingyun Sun, Hao Peng, Xingeheng Fu, Qian Li, and Jianxin Li. Regel: rethinking message passing in graph contrastive learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 8544 88522, 2024.
[35] Chao Jia, Yinfei Yang, Ye Xia, Yi-Ting Chen, Zarana Parekh, Hieu Pham, Quoc Le, Yum-Hsuan Sung. Zhen Li, and Tom Duerig. Scaling up visual and vision-language representation learning with noisy text supervision. In International conference on machine learning, pages PMLR, 2021.
[36] Can Jin, Tong Che, Hongwu Peng, Yiyuan Li, Dimitris Metaxas, and Marco Pavone. Learning from teaching regularization: Generalizable correlations should be easy to imitate. Advances in Neural Information Processing Systems, 37:966-999 994, 2024.
[37] Prannay Khosla, Piotr Teterwak, Chen Wang, Aaron Sarna, Yonglong Tian, Phillip Isola, Aaron Maschinot, Ce Liu, and Dilip Krishnan. Supervised contrastive learning. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin, editors, Advances Neu- ral Information Processing Systems, volume 33, pages 18661-18673, Curran Associates, Inc., 2020. URL http::://pprceeedingggge wwwpeeediiiiinnnnnnneeeeeeeeeeee cc/paper..... files/ www.ffelweeeeeaaaeeeaaeaaaaaaeee httpp/ppapppe/////ppppp/ Paper.pde
[38] Diederik Kingma, Max Welling, et al. Auto-encoding variational bayes, 2013.
[39] Durk P Kingma, Tim Salimans, Rafal Jozefowicz, Xi Chen, Ilya Sutskever, and Max Welling, Improved variational inference with inverse autoregressive flow. Advances in neural information processing systems, 29, 2016.
[40] Alex Krizhevsky and Geoffrey Hinton. Learning multiple layers of features from tiny images. Technical report, University of Toronto, 2009.
[41] Michael Laskin, Aravind Srinivas, and Pieter Abbeel. Curl: Contrastive unsupervised repre- sentations for reinforcement learning, In International conference machine learning, pages 5639-5650. PMLR, 2020.
[42] Yaron Lipman, Ricky T. Q. Chen, Heli Ben- Ben-Hamu, Maximilian Nickel, and Matthew Le. Flow matching for generative modeling. The Eleventh International Conference on Learning Representations, 2023. URL https: http///eeeeeeeeeeeeeeeeeeeeeeeee wwwpnnrwweeeeeeeeeeeeeeeeeeeeeee wtt.//eetoornnee owffm wow.mmiddPyyMMD..TTSS
[43] Tohar Lukov, Na Zhao, Gim Hee Lee, and Ser-Nam Lim. Teaching with soft label smoothing for mitigating noisy labels in facial expressions. In European Conference on Computer Vision, pages 648-665. Springer, 2022.
[44] Jie Ma, Chuan Wang, Yang Liu, Liang Lin, and Guanbin Li. Enhanced soft label for semi- supervised semantic segmentation. Proceedings of the IEEE/CVE International Conference on Computer Vision, pages 1 1185-1195, 2023.
[45] Islam Nassar, Munawar Hayat, Ehsan Abbasnejad, Hamid Rezatofighi, and Gholamreza Haf- fari. Protocon: Pseudo-label refinement via online clustering and prototypical consistency for efficient semi-supervised learning. In Proceedings of the IEEE/CV conference on computer vision and pattern recognition, pages 11641-11650, 2023.
[46] Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predictive coding, arXiv preprint arXiv: 2018
13

---

[47] Senthil Purushwalkam and Abhinav Gupta. Demystifying contrastive self sulf-supe learning: Invariances, augmentations and dataset biases. Advances in Neural Information Processing Systems, 33:3407-3418, 2020.
[48] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference machine learning,
pages 8748-8763 PmLR, 2021.
[49] Danilo Rezende and Shakir Mohamed. Variational inference with normalizing flows. In International conference machine learning, pages 1530-1538. PMLR, 2015.
[50] Danilo Jimenez Rezende, Shakir Mohamed, and Daan Wierstra. Stochastic backpropagation and approximate inference in deep generative models. In International conference on machine learning, pages 1278-1286. PMLR, 2014.
[51] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pages 10684-10695, 2022.
[52] Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma. Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, et al. Imagenet large scale visual recognition challenge. International journal of computer vision, 1 115:211-2521 2015.
[53] Nikunj Saunshi, Orestis Plevrakis, Sanjeev Arora, Mikhail Khodak, and Hrishikesh Khande- parkar. A theoretical analysis of contrastive unsupervised representation learning. In Interna- tional Conference on Machine Learning pages 5628-5637. PMLR, 2019.
[54] Florian Schroff, Dmitry Kalenichenko, and James Philbin. Facenet: A unified embedding for face recognition and clustering. In Proceedings of the IEEE conference on computer vision and pattemn recognition, pages 315-823, 2015.
[55] Vlad Sobal, Mark Ibrahim, Randall Balestriero, Vivien Cabannes, Diane Bouchacourt, Pietro As- tolfi, Kyunghyun Cho, and Yann LeCun. $lmathbb (X) X-SSmapele contrastive loss: Improving con- trastive learning with sample similarity graphs. In The Thirteenth International Conference on Learning Representations, 2025. URL https://openreviii
.et formm? iidmm?dddcccnnii
[56) Kihyuk Sohn. Improved deep metric learning with multi-class n-pair loss objective. Advances in neural information processing systems, 29, 2016.
[57] Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jon Shlens, and Zbigniew Wojna. Re- thinking the inception architecture for computer vision. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2818-2826, 2016.
[58] Linwei Tao, Minjing Dong, and Chang Xu. Dual focal loss for calibration. In International Conference on Machine Learning, pages 118833-33849 PMLR, 2023.
[59] Haoqing Wang, Xun Guo, Zhi-Hong Deng, and Yan Lu. Rethinking minimal sufficient rep- resentation in contrastive learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 16041-16050, 2022.
[60] Luran Wang. Chaoran Cheng, Yizhen Liao, Yanru Qu, and Ge Liu. Training free guided flow-matching with optimal control. In The Thirteenth International Conference on Learning Representations, 2025. URL /ttps://openrewe http:/wwwwwwweeeeeeeeeeeeeee net//orrmm? id-
[61] Tongzhou Wang and Phillip Isola. Understanding contrastive representation learning through aligmment and uniformity on the hypersphere. In Hal Daumé III and Aarti Singh, editors. Proceedings of the 37th International Conference on Machine Learning, volume 119 of Pro- ceedings of Machine Learning Research, pages 992999999999999999 9939.999...... PMLR, 13-18 Jul 2020. URL https: attps://proceedings mlr wwt.ssaw..........ee.ee.eeeereee https///111///gn200e html.
[62] Ziwen Wang. Jojosel: Shrinkage contrastive learning for single-cell ma sequence clustering. In 2024 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), pages IEEE, 2024.
14

---

[63] Yang You, Igor Gitman, and Boris Ginsburg. Large batch training of convolutional networks. arXiv preprint arXiv: ww0..0000000 03888, 2017.
[64] Yuning You, Tianlong Chen, Yongduo Sui, Ting Chen, Zhangyang Wang, and Yang Shen. Graph contrastive learning with augmentations. Advances in neural information processing systems, 13:5812-5823, 2020.
[65] Tianhao Yu, Haiyang Cui, Jianan Canal Li, Yunan Luo, Guangde Jiang. and Huimin Zhao. Enzyme function prediction using contrastive learning. Science, 379(663))11388-133 1368, 2023.
[66] Jure Zbontar, Li Jing, Ishan Misra, Yann LeCun, and Stéphane Deny. Barlow twins: Self- supervised learning via redundancy reduction. In International conference on machine learning, pages 13110-1230. PMLR, 2021.
[67] Hongyi Zhang, Moustapha Cisse, Yann Dauphin, and David Lopez-Paz. mixup: Beyond empirical risk minimization. arXiv preprint 1710.9921 2017.
[68] Xiong Zhou, Xianming Liu, Feilong Zhang, Gang Wu, Deming Zhai, Junjun Jiang, and Xiangyang Ji. Zero-mean regularized spectral contrastive learning: Implicitly mitigating wrong connections in positive-pait graphs. In The Twelfth International Conference Learning Representations, 2024.
15

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
The abstract and introduction clearly outline our paper's main contributions: (I) a variational reformulation of supervised contrastive learning as inference over latent class variables (3); (2) confidence-adaptive temperature scaling strategy (3, paragraph 5); and (3) empirical validation across multiple benchmarks showing improved performance and semantic organization (4). The claims aligned with the actual results presented throughout the paper, particularly in Tables and
Justification:
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: Our paper discusses limitations in Section A. of the appendix, where we address computational complexity considerations, batch size dependency, and potential challenges in cross-domain applications. Throughout the experimental section (4), we also analyze performance boundaries under various conditions, such as reduced data availability (Table 2) and different augmentation strategies (Table 3), providing clear understanding of our method's limitations.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested few datasets with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they seale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimess.
16

---

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?
Answer: [Yes]
Justification: Our paper provides full mathematical derivations for the variational bound in Section 3. We clearly state our assumptions and provide step-by-step proofs from Bayes' rule (Eq. 1) through the ELBO derivation (Eq. 5), to our final VarCon loss formulation (Eq. 7). All equations are properly numbered and cross-referenced, with intermediate steps clearly shown. We also define the mathematical notations and variables the beginning of Section 3 to establish our theoretical framework.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: Section 4 provides comprehensive details on datasets CCIFAR-10/108 ImageNet-100, ImageNet), data preprocessing, network architectures (ResNet-50/101/200, ViT-Base), and optimization parameters. We specify batch sizes (512 for CIFAR 10/100, 1,024 for ImageNet- 100, 4,096 for ImageNet), learning rate scheduling (linear scaling with cosine decay), optimizers (SGD with momentum 0.9 and weight decay 10-4, LARS for ImageNet), and data augmentation strategies. Our ablation studies in Section 4 (subsections "Effect of Temperature," "Batch Size Sensitivity Analysis," etc.) further detail parameter choices and their effects.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access to a hosted model (c.g., in the case
17

---

of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily new algorithm, the paper should make it clear how to reproduce that algorithm. (b) the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is new model (c.g.. . a large language model), then there should either be way to access this model for reproducing the results or a way to reproduce the model (e.g.. with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may tricky some cases, in which case authors welcome describe the particular way they provide for reproducibility. the case of closed-source models, it may be that access to the model is limited in some way (e.g.. registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: We have detailed our code in Fig. 1. And plan release full code and trained models upon publication, including implementation details for VarCon loss, training scripts, and evaluation pipelines. Our code will cover all experiments presented in the paper, utilizing standard publicly available datasets (CIFAR-10/100) ImageNet) and common network architectures (ResNet, ViT).
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.cm cc/ public/guides/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https: / /nips.
sww.nnosssippcccc for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Question:
Answer: [Yes]
18

---

Justification: We provide detailed experimental settings in Section 4, including dataset splits (using official test splits), architectures, data augmentation techniques (SimAugment, AutoAugment, StackedRandAAngment, optimizer configurations (SGI) with momentum 0.9, weight decay 10- learning rate schedules (linear scaling with batch size), and training durations (200 epochs for CIFAR-10/100 and ImageNet-100 100, 350 epochs for ImageNet.. Key hyperparameters like the temperature parameter Ti and adaptive range e are specified in our ablation studies, where we show the effect of various values (c.g.. optimal
Ti = 0.1, 0.02).
Guidelines:
The answer NA means that the paper does include experiments.
The experimental setting should be presented the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: All our experimental results include appropriate statistical information. Tables and 2 report accuracy measures with standard error (e.g., 95.94.0.077% for VarCon on CIFAR-I0), calculated across multiple runs with different random initializations. For the few-shot learning experiments in Table we explicitly state that results are averaged across five random data subsets for each sample size to account for sampling variability, The error bars represent standard error of the mean, providing a clear measure of the statistical reliability of our results.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma eтror bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
Question:
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Answer: [Yes]
Justification: We provide sufficient information about computational resources Section B and B. of the appendix. Our experiments were conducted on NVIDIA GPUs, with
19

---

larger experiments (ImageNet) utilizing multi-GPU setups. We specify in Section 4 that we employed different batch sizes for different datasets (512 for CIFAR-10/100, 1024 for ImageNet-100, and 4096 for ImageNet), which implicitly indicates the computational requirements. The reduced batch size dependency of our method, as shown in Figure 2c, also demonstrates our attention to computational efficiency.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't 't make it into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics https: https://neut httt:p:///eurips.... httppppulic..hiccacuuiieeeeeee
Answer: [Yes]
Justification: Our research fully complies with the NeurIPS Code of Ethics. We use standard, publicly available benchmark datasets (CIFAR-10/100, ImageNet) that are widely accepted in the machine learning community. Our VarCon method focuses on improving represen- tation learning without raising specific ethical concerns. We report results transparently with appropriate error statístics, avoid misleading claims, and compare fairly with baseline methods. All experimental comparisons were conducted under equitable conditions using the same computational resources for both our method and baselines.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No. they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: We discuss the broader impacts of our work in Section A.2 of the appendix, where address both potential positive impacts (improved performance for visual recogni- tion systems in healthcare, autonomous vehicles, etc.) and possible negative implications (such as potential use in surveillance systems). We also discuss how our method's 's improved efficiency with smaller batch sizes contributes to reducing computational resources, which has positive environmental implications through reduced energy consumption.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (c.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out, For example, it is legitimate to point out that improvement in the quality of generative models could be used to
20

---

generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Our work does not involve models datasets with high risk for misuse. VarCon is representation learning method that improves embedding spaces for classification tasks, not a generative model capable of producing deceptive content. We work with standard benchmark datasets (CIFAR- CCIFAR-10/000 10/100, ImageNet) rather than scraped web data. Our method does not enable capabilities that would require specific safeguards beyond standard practices in machine learning research.
Justification:
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging. and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: All datasets and baseline methods used in our research are properly cited. We reference the original papers for LIFAR-10/100 [40], ImageNett [52, 18], and the comparison methods (SimCLR, SupCon, MoCo, etc.) in Section 4. We use these standard benchmark datasets according to their intended research purposes. For the encoder architectures, we cite and use the standard implementations of ResNet and ViT models following the settings in previous works [37]. We do not modify or repackage any existing datasets.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
21

---

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and the documentation provided alongside the assets?
Answer: [NA]
Justification: Our paper does not introduce datasets, tools, models standalone assets that would require specific documentation. We present new methodology (VarCon) that builds upon existing frameworks and benchmark datasets, but this is fully described within the paper itself. While we plan to release our implementation code in the future, we are not introducing new assets as part of this paper submission.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moddl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and sereenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: Our research does not involve crowdsourcing, human subjects, or human evaluations. All experiments are computational, using established benchmark datasets (CIFAR-10/100, ImageNet) and algorithmic evaluations. No human participants were involved in the research process beyond the authors themselves.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Question:
Answer: [NA]
22

---

Justification: Our research does not involve human subjects, participants, or human data collection of any kind. All experiments use standard machine learning benchmark datasets (CIFAR-10/100, ImageNet) and computational evaluations. Therefore, no IRB approval or equivalent was required for this work.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: Our research on Variational Supervised Contrastive Learning (VarCon) does not involve Large Language Models (LLMs) in any capacity part of our methodology, periments, or analysis. Our work focuses on representation learning for image classification through variational approach to contrastive learning, without any LLM components.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to our LLM policy (https atttp::/neurips: cco.oooeeeecccc/22255555 for what should or should not be described.
23

---

Supplementary Material
A Discussion
A.1
Linitation
While Variational Supervised Contrastive Learning (VarCon) demonstrates significant improvements existing methods, acknowledge several limitations. First, approach relies supervised learning, which limits its applicability where labeled data is scarce or expensive to obtain. Second, although VarCon reduces comparison complexity from quadratic nearly linear through class-level centroids, computing these centroids still introduces overhead for datasets with numerous classes. Third, cross-domain adaptation may require recalibration of mechanism when facing substantial distribution shifts between source and target domains.
sefffidnneeddaaptiv temperature
Future work could address these limitations by: 1) extending VarCon to self-supervisee settings by leveraging augmentation-bast positive pairs with a variational formulation of latent pseudo-labels, which would maintain our probabilistic framework while removing the need for explicit supervision; 2) exploring amortized computation of class centroids through memory banks or representative embedding techniques to reduce computational overhead for large-scale applications; and 3) incor- porating domain adaptation techniques such as distribution alignment constraints in the variational objective to improve cross-domain robustness. The core variational formulation of VarCon is flexible enough to accommodate these extensions with relatively modest modifications to the underlying framework.
Despite these limitations, VarCon successfully addresses a fundamental challenge in representation learning by providing principled probabilistic interpretation of contrastive objectives while simulta- neously improving empirical performance across diverse benchmarks. Our approach offers a new theoretical perspective that bridges the gap between generative and distinguishing paradigms, opening promising directions for developing more robust, interpretable, and efficient representation learning methods.
A.2 Broader Impact
VarCon offers significant potential for broader impact across multiple domains of deep learning research and applications. By establishing a principled probabilistic foundation for contrastive learning and demonstrating substantial empirical improvements, our work contributes to several important areas:
From a scientific perspective, VarCon bridges the theoretical gap between generative and contrastive approaches to representation learning. This unification provides researchers with a more coherent understanding of how different learning paradigms relate development of hybrid approaches that leverage the strengths of multiple frameworks. The explicit modeling of uncertainty through our confidence-adaptive temperature mechanism also contributes to the growing body of work on uncertainty quantification in deep learning, which remains a critical challenge for reliable AI systems.
another, potentially accelerating the
From an applications standpoint, VarCon's improved performance and robustness make it particularly valuable for domains where high-quality representations essential. In medical imaging, more distinctive features could enhance diagnostic accuracy and treatment planning. In computer vision for autonomous vehicles, better representations may improve object detection and scene understanding under varying conditions. For scientific applications like protein structure prediction or molecular property estimation, our approach's 's ability to capture fine-grained semantic relationships could lead to meaningful discoveries.
The environmental impact of our approach is noteworthy. VarCon's 's reduced dependence large batch sizes and faster convergence translate to lower computational requirements and energy consumption compared to existing contrastive methods. As the carbon footprint of machine learning research grows increasingly concerning, techniques that maintain or improve performance while requiring fewer computational resources represent important direction for sustainability.
Our method's 's enhanced interpretabillty through uncertainty modeling provides foundation for more transparent embedding learning. By explicitly quantifying confidence in learned representations,

---

VarCon could help identify potential failure modes or biases in downstream applications, particularly important as deep learning systems increasingly influence high-staker decisions.
In summary, VarCon represents not only a technical advancement representation learning but also step toward more principled, efficient, and transparent deep learning systems, which can be responsibly deployed across a wide range of applications.
B Experimental Analysis and Additional Results
B.1 Hyperparameter Settings
In this section, we provide additional hyperparameter settings used our VarCon implementation.
Optimization Details For all experiments, used SGD with momentum 0.9 and weight decay 1 x 10-1 We employed a cosine learning rate schedule with initial learning rate 0.05, and for experiments with batch sizes greater than 256, incorporated 10-epoch warm-up phase. For ImageNet experiments, we used the LARS optimizer to maintain training stability at large batch sizes. We utilized mixed-precision training (AMP) with gradient scaling to improve computational efficiency while maintaining numerical stability. All experiments were conducted on 8 NVIDIA A100 GPUs. Training for 350 epochs on ImageNet required approximately 54 hours.
Data Augmentation Strategies Data augmentation plays crucial role in contrastive learning, as it defines the invariances that the representation should capture. We experimented with three augmentation strategies: SimAugment (random cropping, flipping, color jitter, and grayscale conver- sion). AutoAugment (which uses reinforcement learning to discover optimal transformation policies), and StackedRandAuggenn (which applies multiple random transformations sequentially). While previous contrastive learning methods often require strong augmentations to achieve competitive performance, VarCon demonstrates superior results even with simpler augmentation strategies. This is particularly significant because finding appropriate augmentation policies is typically domain-specific and timecconumming: augmentations that work well for images may not transfer other modalities such as text or audio. VarCon's 's reduced dependency on aggressive augmentation makes it more adaptable across different domains and reduces the need for extensive hyperparameter tuning.
Positive and Negative Sample Definition Different contrastive learning frameworks define positive and negative samples distinctively, which significantly impacts their learning dynamics. SimCLR, positive pairs consist solely of different augmented views of the same instance, while all other instances in the batch serve as negatives, regardless of their semantic similarity. SupCon extends this definition by considering samples from the same class as positives, leveraging label information to create semantically meaningful groupings. Our VarCon framework further refines this approach by encoding these relationships through probabilistic lens. Rather than relying on explicit pairwise comparisons, VarCon infers class-conditiona likelihoods and minimizes the KL. divergence between the model's posterior and adaptive target distribution. This formulation naturally handles both same-instance positives (through augmented views) and same-class positives (through class centroids) while maintaining appropriate separation from samples of different classes. The key distinction is that VarCon's 's confidence-adaptic temperature mechanism dynamically adjusts the "strength" of these positive relationships based on classification difficulty, providing fine-grained control over the embedding space organization. This probabilistic treatment of sample relationships contributes significantly VarCon's superior performance and robustness across various experimental settings.
These implementation details contribute significantly to VarCon's performance and stability across different datasets and architectures, highlighting the importance of careful hyperparameter selection in representation learning systems.
B.2 Extended Evaluations
To further validate the robustness of our proposed approach, we evaluated VarCon against leading self- supervised and supervised methods on the ImageNet-Real, which addresses the inherent limitations of the original ImageNet validation set by providing higher-quality multi-label annotations. Table presents these results.
25

---

Table Classification performance comparison on ImageNet-Real We report Top-1 accuracy (%) (mean + standard eтror) for VarCon versus state-of of seeeeoofthe-arte the-art supf-supe and supervised methods. All models utilize the ResNet-50 architecture for a fair comparison. Best scores are highlighted in blue second-best in green
Dataset
Category
Method
ImageNet-Real
Top-1
Supervised Self-supervised
SimCLR
75.30+0.05
MoCo V2
78.221.004
BYOL
81.10+0.07
SwAV
81.5610004
VicReg
79.454..07
Barlow Twins
80.09
Cross-Entropy
83.4710.06
SupCon
83.87 10.04
VarCon (Ours)
84. 84.12+0.04
On this more rigorous dataset, VarCon achieves 84.12% Top-1 accuracy, outperforming all baseline methods by statistically significant margin. The performance gap between VarCon and the strongest self-supervised method (SwAV 81.56%) is significant, showing solid 2.56% improvement, suggesting that our probabilistic formulation effectively leverages label information to learn more distinctive representations that better align with human perception of image content.
Even within the supervised category, VarCon demonstrates clear advantages over conventional cross- entropy (83.47%) and the previous state-of-the-an SupCon approach (83. 883.87). This improvement is especially meaningful on ImageNet-ReaL, where evaluation more accurately reflects model perfor- mance on actual visual content rather than potentially noisy single-label annotations. The consistent performance gains across both traditional ImageNet and ImageNet-Real datasets demonstrate that VarCon's 's improvements are not merely artifacts of label noise but represent genuine advances in representation quality. addition, the reduced standard errors (=0.04) compared to other methods indicate that VarCon's 's probabilistic approach not only improves accuracy but also yields more stable predictions across evaluation runs. This enhanced stability, combined with superior accuracy, high- lights the effectiveness of our variational formulation in capturing the underlying structure of visual data while maintaining robust decision boundaries.
B.3 Embedding Space Evolution Analysis
To investigate how VarCon shapes the embedding space during training, the progressive evolution of learned representations visualized using t-SNE (perplexity 50) on ImageNet validation set. Models are trained for 200 epochs with checkpoints saved at epochs 50, 100, 150, 200. Figure 5 reveals systematic refinement of semantic organization throughout training.
The visualization demonstrates clear progression from loosely organized clusters at epoch 50 (KNN classifier: 52.74%) highly structured, rell-separated semantic groups by epoch 200 (79.1 11). The most significant improvement occurs between epochs 100-150, where the KNN classifier performance jumps from 59.57% 71.12%, corresponding to the convergence our ELBO-derived loss function. During this phase, the KL divergence term effectively aligns the auxiliary posterior with the model's class posterior while ensuring correct class assignments. By epoch 200, VarCon achieves optimal embedding organization with distinct, compact clusters and clear decision boundaries. We employ KNN classification to directly evaluate the quality of pretrained representations without introducing additional trainable parameters that could mask the intrinsic separability of the learned embeddings. The high KNN classifier performance (79. %) demonstrates that our confidence-adaptive temper- ature mechanism successfully provides fine- fine-grained control over intra-class dispersion, creating
26

---

Epoch 50: KNN-classifier Top-1 52.74%.
(h) Epoch 100: KNN-classifier Top-1 59.57%.
(c) Epoch 150: KNN-classifier Top-1 71.12%.
(d) Epoch 200: KNN-classifier Top-1 79.11%.
Figure 5: Progressive evolution of VarCon embedding space visualization through t-SNE [10] during training on ImageNet validation set. Our variational formulation demonstrates systematic improve- ment in semantic organization, with KNN-classifier accuracy increasing from 52.74% at epoch to 79.11 1 % at epoch 200 as clusters become increasingly well-separated and semantically coherent. The confidence-adaptive temperature mechanism enables fine-grained control over intra-class dispersion, resulting in embedding spaces with clear decision boundaries and hicrarchical semantic structure that facilitate effective nearest- neighbr classification without additional parameterized classifiers.
embedding spaces where semantic similarity corresponds to geometric proximity and enabling effective classification through nearest-neighhor search.
The superior embedding quality achieved by VarCon stems from three synergistic mechanisms that fundamentally improve upon conventional contrastive learning. As demonstrated in Figure 6, VarCon achieves 79.11% 11% KNN classifier performance in just 200 epochs, outperforming SupCon's 's 78.53% accuracy obtained after 350 epochs of training. First, by replacing exhaustive pairwise comparisons with class-level centroids, each sample directly learns to align with its corresponding class center, enabling more efficient identification cluster centroids and reducing the quadratic computational complexity inherent in traditional contrastive methods. Second, our variational inference formulation preserves the possibility of inter-class linkages during the learning process, allowing the model to maintain nuanced relationships between semantically related classes rather than forcing rigid
27

---

(a) SupCon (350 epochs): 78.53%.
KNN-classifeer Top-1
(b) VarCon (200 epochs): 79.11%.
KNN-classifier Top-1
Figure 6: Embedding space comparison between SupCon and VarCon through t-SNE visualization on ImageNet validation set. Despite training for significantly fewer epochs (200 vs. 350), VarCon achieves superior KNN classifier performance (79.1 199.11% 1 vs. 78.53%) and demonstrates clearer cluster separation. This comparison validates our findings in Figure 2b that VarCon converges faster than SupCon while simultaneously learning higher-quality representations with better-defined decision boundaries. The variational formulation enables more efficient optimization dynamics, achieving better semantic organization in substantially reduced training time.
Table 6: Hicrarchical clustering evaluation ImageNet embeddings. We perform Ward linkage clustering on the ImageNet validation set. All metrics: higher better (t).
Method
ARI NMI
Homogeneity Completeness V-measure
Purity
SupCon
0.613
0.888
0.886
0.890
0.888
0.755
VarCon (e 0)
0.627
0.892
0.890
0.892
0.891
0.766
VarCon
0.634
0.895
0.893
0.896
0.895
0.774
separations. This probabilistic treatment enables the embedding space capture gradual transitions and hierarchical relationships that are often lost in hard contrastive objectives. Third, the confidence- adaptive temperature mechanism provides dynamic regulation of learning intensity: for well-classified samples, prevents overfitting by relaxing constraints, while simultaneously promoting continued learning on challenging examples by tightening supervision. This adaptive strategy ensures that computational resources are allocated efficiently, focusing learning capacity on samples that require additional refinement while maintaining stability for already well-separated instances.
B.4 Hierarchical Clustering Results
To further investigate the semantic organization of learned representations, we perform hierarchical clustering analysis on the embedding space. We extract features from the entire ImageNet validation set and apply Ward linkage clustering, which iteratively merges clusters based minimum variance criteria. This approach reveals the natural hierarchical structure present the learned representations without imposing predefined category boundaries.
Table 6 presents comprehensive clustering quality metrics. VarCon consistently outperforms SupCon across all evaluation criteria, with progressive improvements observed through our ablation. The Adjusted Rand Index (ARI), which measures the similarity between cluster assignments and ground- truth labels while correcting for chance, increases from 0.613 (SupCon) to 0.627 (VarCon with 0) 0.634 (VarCon). This progression suggests that the discovered clusters align increasingly closer with true semantic categories. Similarly, the purity metric, which quantifies the fraction of samples in
28

---

each cluster belonging to the most common class, improves from 0.755 to 0.766 to 0.774, indicating that individual clusters contain progressively fewer mixed-class samples.
The Normalized Mutual Information (NMI) score, measuring the mutual dependence between cluster assignments and true labels, increases from 0.888 to 0.892 to 0.895. This gain reflects stronger statistical relationship between the discovered structure and semantic categories. The V-measure, which provides the harmonic mean of homogeneity and completeness, shows balanced improvements (0.888 to 0.891 to 0.895). Specifically, homogeneity, measuring whether each cluster contains only members of single class, improves from 0.886 to 0.890 0.893, while completeness, measuring whether all members of a class are assigned to the same cluster, increases from 0.890 to 0.892 0.896. These metric improvements align with our theoretical framework. Notably, VarCon with fixed temperature (e) already outperforms SupCon. demonstrating that the ELBO-derived loss itself enhances representation quality. The full VarCon with confidence-adaptive temperature mechanism further improves performance by applying stronger supervision on difficult samples while relaxing constraints on well-classified examples. This adaptive behavior, combined with the KL divergence term that prevents distributional collapse, appears to preserve more nuanced semantic relationships during training. The clustering analysis thus provides empirical support for our variational formulation's ability to maintain meaningful structure in the representation space, with both components contributing to the superior performance.
C Theoretical Analysis
C.1 Gradient Derivation w.r.t. Embedding
To understand the training dynamics of VarCon, we derive the gradient of the VarCon loss with respect the fz-normalized embedding z E IRd, Our analysis reveals how the confidence-adaptive temperature mechanism influences learning through gradient modulation.
Recall that our VarCon loss is:
L = DKL (gs 1l 2)) - log Po 2),
(14)
where r is the ground-truth class, ds is the confidence-adaptive target distribution, and pe is the model's posterior distribution.
The gradient decomposes into two terms:
V
0KI.. Oz
i log pe (r 2)
(15)
Oz
Oz Log-posterior
KL divergence
Gradient of the KI. Divergence.
The KL. divergence between qa and po is:
DKL (g6 1l Pp) = Σ (r' 2) log 2o(r') 2) pa (r' x)
(16)
To compute its gradient in Eq. (16), we apply the product rule to each term:
i
(r'r' 2)
(qs
(z)
(r 2))
log
log
Oz
po (r' z)
Oz
Po(r' 2)
log (r' 2)
(r' 2))
(17)
Oz po(p" 22
For the second term in Eq. (17), have:
log 40 (r' z)
1
Oyo (r' [2)
1 Opo(r' |x]
(r' 2) Oz po 2)
2o(r') | 2)
(r(r' 2)
Oz
po(r' z) Oz
@rs(r'
x)
(r'
x) Ope(r' 12)
(18)
dz
(r(r'
2)
Oz
29

---

In our implementation, we compute class centroids as Therefore, the complete gradient of each term in Eq. (16) is:
log 66(r' (2)
Dqu(r' [x)
log To (r (r' x))
9o(r' log
Öz
po(r' 2)
Oz
Pe (r' (2)
to (r'rr) x)
Öpp (r' x2)
(19)
Po (r' z)
Oz
Since Ene 4á (6 2) 1, we have the constraint Σ, classes:
dqa(r(lz) dz
Thus, when summing over
0D
C
Oqo(r' x)
( 2
+ - Σ
(r 2)
Ops (r' x)
log
dz
Üz
Po 2
Po (rd 2))
Ox
M=
r'ol
0qs(r' |x)
(r'
22)
dqs (r' (x)
(o(r' (x)
apa (r'
(x)
log po (r' [2]
8z
Pa(r" (z)
Oz
=0
04s (1(12) [²)
log (r 22)
(r' 2)) Ope(r' z)
(20)
Oz
po (r' 2)) po (r' 2)
Oz
Computing (r | z)/Oz.
The target distribution go depends on & through the confidence-
adaptive temperature:
(2 = (n - <) + 2cpo 2),
(21)
where is learnable parameter controlling adaptation strength. From our formulation, qá has the closed form:
exp(1/72) c-1+oxp(1/r3)
if =
1a(r' 12)
(22)
if tr
(-1ooxp(1/r5)
For the ground-truth class r'
0qs(r 2)
(C- - 1) exp(11T2)
(23)
OTz
12 C - 1 exp (1/2)11
For other classes r' # r
Og ( (r)
exp(1/T2)
(24)
OT2
T2 + exp(L//2))
By the chain rule:
@qs [x)
0go(r' 22
dT2 0qo(T') 1z)
Öpe(r x)
2e
(25)
OT2
Oz
OT2
8z
Computing 0pe(r' 22)Ozz The model's posterior follows softmax distribution:
exp(2 w., /TT)
(r(r' 2))
(26)
1= exp(# T w
Remark on Class Centroid Computation. the normalized average of embeddings:
Σ,
where E,
1
Σ Zi.
(27)
- |
B₁
B.B.
and B, = (i:r; r; r. B} denotes the set of samples with class r in the mini-batch. We treat these centroids as constants during backpropagation (i.e., we detach them from the computational graph). This design choice:
30

---

Simplifies gradient computation and improves training efficiency
Avoids cyclic dependencies that could lead to training instabilities
Aligns with the interpretation of centroids as fixed reference points representing the current state of each class
Therefore, when computing V /2z, we do not propagate gradients through wy,, treating them as constants in the following derivations.
Its gradient is:
Op z)
Pa
x)
(w,
Eyo [w]],
(28)
where Epulas) E po (A: 2)wx the expected class centroid under the current model distribu- tion.
Gradient of the Log-Pusterior. From the log- prgppoottrror definition:
log Po == wr
log Σ
exp
= T
Wr
(29)
Taking the gradient of Eg. (29):
log (r 2)
[w
Epo [w)]
(30)
Oz
Complete Gradient and Interpretation.
Combining both Eqs. (20) and (30), the complete gradient
is:
VL
Σ
Ogo z)
log Lo (T' 2) Oz Po (r' z)
(r'r' 2)
Ope(r'
z
Oz
Po (r' z)
Zz
Distribution alignment
Weighted gradient
W. Epp [w]]
(31)
Centroid
Since embeddings are no-omaallieed ([|=|12 = 1), effective gradient must projected onto tangent space of the unit sphere:
LarCoon
VarCoon
[I
zz
(32)
dz
Oz
efffective
where I - zz is the projection operator.
This gradient analysis reveals the dual mechanism of VarCon:
The KL divergence term aligns the auxiliary distribution %s with the model posterior through confidence-adaptive weighting.
The log-posterior term creates an attractive force toward the true class centroid w, and repulsive force from the expected centroid Ep, [w].
The menfidence-adaptve temperature T2(2) modulates the alignment strength based on classification confidence, providing sharper supervision for difficult samples.
C.2 Gradient Derivation w.rtt Adaptation Strength Parameter E
To understand how the adaptation strength parameter influences the training dynamics, we derive the gradient of the VarCon loss with respect to e. This analysis reveals how the confidence-adaptive mechanism automatically adjusts the strength of supervision based on sample difficulty.
31

---

Recall that appears in the monfidence-daapiiv temperature:
T2(2) = (Ti - <) + 2cps(r =);
which in turn affects the target distribution qι- The gradient of the VarCon loss with respect is:
Varrroon
0Dk. Ô log logpo((r z)
(33)
de
de
dc
We compute the instantaneous gradient with respect to while treating the encoder parameters 0 as fixed during this computation. This is consistent with how gradients are computed in backpropagation, where each parameter's gradient is calculated independently. Under this assumption, pe (r x) is treated constant with respect to e, yielding:
a log po 2)
0.
(34)
0 fixed
Therefore, the gradient in Eq. (33) simplifies to:
i LavrCon 0DKL
(35)
de
Computing the KI. Divergence Gradient. From the KI. divergence definition in Eq. (16):
DKL. Σ (o('' z) log go(r' 2) po (r' z)
apply the product rule compute its derivative with respect to
0DKL.
C
Equlrr' z)
Go(r' z) de Po(r' (r' z)
i
log (a(r' 2))
Σ
log
+ go(r'
x)
(36)
dE
De
Po(r' 2)
For the second term in Eq. (36), since po (r| 22) ) is treated as constant with respect to e:
a log (r (2) 1 Ogo(r' =))
(37)
de po(r' 2)) g0(r" 2)
de
Substituting back in Eq. (36):
0DxL.
C
Oqa(r' [z)
log Gorr'' 2))
Zgo (r' (2)
Σ
+ go(('' (z).
de
OE (r(r'
2))
40(r'
z)
de
r=1
C
Ogslt (r'
[z]
log (r(r') 2))
+1
(38)
de
(r(r' (r' 2)
1=1
Since
(6(r' 22) go('
1, we have the constraint
Σ,
@ga(r'|z)
Thus:
Be
θDkL
Oyolr' z)
log (r 2)
+
de
de
pr(r'
22)
Oqs(r' 2))
log ga(r'
())
0g (2)
+
de
Pr
2)
de
Oqs(r' 1z)
log (r(r') 2))
(39)
de
Po
x)
32

---

Computing 07s(r' z)/0e.
Using the chain rule:
Oqa(r' x))
Oqs(r' 2) Q72
(40)
de ÔT2 de
First, we compute 072/ Oe:
de [(T1 - -c) + 2epe(r xx]
Ōe
=-1 + 2pa (r z)
2p 2) 1.
(41)
Note that pr(r 2) treated as constant with respect under our instantancous gradient assumption.
This derivative reveals key insight:
When (r x) > 0.5 (confident predictions): 0rz/0e > 0
When (r 2) < 0.5 (difficult samples): 072/Oc
The derivatives 04 (r' / 0rz were computed in Section C.1. For the ground-truth class r' r:
(r =)
(C - 1) exp(1/12)
(42)
iT2
T2 - exp exp(1/T2)1
and for other classes r' * r:
dq (z)
cxp(1/T2) 1 exp(L/T2)]
(43)
OT
T2
Complete Gradient Expression.
Combining all terms in Eqs. (39) and (41):
VLvarCon
0qa(1' 12)
0TZ
ls =))
log
de
OT2
po |z)
1=1
dg 2))
log 20(r' 2)
(12po(r) -1]
(44)
po (r' 2)
Substituting the specific derivatives in Eqs. (42) and (43):
[2po(r 22) - x
de
exp(1/m3
log (o(r') 2)
Σ *'
2 + exp [xp2)1 po (r' x)
-C-1) (C-1) exp(p//7)
log (r(r z)
(45)
T C - e
po (r 2)
Factoring out common terms:
[2pa(r 2) - exx((L/TT))
dc
T - exp(1//pp)) T2
log zo(r') =))
(r z)
x
(C 1) log
(46)
(r z)
(r (r z)
33

---

Interpretation and Learning Dynamics. This gradient reveals the self-regulating nature of the sunidencceadaapiie mechanism:
For confident samples (po(r 2) > 0.5): The factor [2po(r =) - 0. When Pr(r 2) is high (confident prediction), 72 approaches T1 + c, making 9s more uniform.
The gradient direction is determined by the aggregate log-ratio term Er log 2o (r((2)
pe(r"
(C (C-1] log po(r|=] This uniform 9s typically satisfies g 2) < p 2) for the ground-truth class and (o(r' 22) >>> po(r' 2) for other classes, making the aggregate term positive. Consequently, the gradient pushes c increase, which in turn increases T2, making 4o even more uniform and effectively reducing supervision strength for already well- classified samples. This reff-reinooriing mechanism prevents overfitting on easy samples while maintaining stable learning dynamics.
(a(r)=)
For difficult samples (po(r (r =)) < 0.5): The factor [2po(r 2) - 1] < O. When the model is uncertain, 72 approaches T1 e, creating sharper distribution. The gradient behavior reverses, causing e to adjust such that 72 decreases. This reduction in e leads to a smaller T2, creating a sharper 96 distribution that provides stronger supervision preciscly where the model needs it most. The same aggregate log-ratio term that increases e for confident samples now decreases it due to the negative sign factor. The magnitude of this effect scales with the degree of distributional mismatch, ensuring proportional adaptation to the model's uncertainty.
Dynamic equilibrium: During training, e and are updated jointly through gradient descent. While our analysis considers their instantaneous gradients separately, their co-evolution creates a dynamic equilibrium where e continuously adapts to the current state of the model, automatically balancing exploration and exploitation throughout the learning process. As training progresses and the model's predictions align better with ground truth, the distributional differences driving the gradient naturally diminish, stabilizing Although is learnable and updated via gradient descent according to the derived gradients, we enforce hard bounds during optimization to ensure numerical stability. Specifically, after each gradient update, é is clamped to predefined range (e.g.. [0, 0.08] as illustrated in Figure 3b). This constraint ensures that the confidence-adde adaptive temperature 72(2) = (ni -e) + 2epo (r z) remains within valid bounds throughout training, preventing degenerate solutions while still allowing sufficient flexibility for adaptation.
This analysis confirms that the learnable parameter provides an elegant mechanism for adaptive supervision strength, contributing to VarCon's superior performance and faster convergence compared to fixed-temperature approaches.
D Training Details
D.1 Loss Convergence Analysis
To empirically validate VarCon's 's training efficiency, we analyze the loss trajectories in a controlled 50- epoch training experiment on ImageNet with ResNet-50 and overall batch size of 4096 distributed across 8 A 100 GPUs, as demonstrated in Figure 7. VarCon demonstrates dramatically faster loss convergence compared to SupCon: starting from comparable initial losses (VarCon: 9.997, SupCon: 11.142), VarCon's loss drops to 2.836 by epoch 10, which is a 71.6% reduction versus SupCon's 's 34.3% reduction 7.3 This convergence advantage persists throughout training, with VarCon achieving a final loss of 1.1 at epoch 50, 75.1% lower than SupCon's 's 4.578. Crucially, both components of VarCon's 's objective decrease synergistically: the KL divergence term reduces from 4.978 to 0.538 (89.2 decrease) while the negative log- posterior term decreases from 5.019 to 0.599 (88.1 % reduction), confirming that our variational formulation successfully balances distributional alignment with classification performance. Even with this abbreviated 50-epoch training schedule, significantly shorter than the standard 200-350 epochs, both methods achieve reasonable downstream classification accuracy (VarCon: 75.3%, SupCon: 74.5%). but VarCon's 's substantially lower final loss and steeper descent trajectory demonstrate more efficient optimization. This empirical evidence validates our theoretical framework: the ELBO-derrved objective provides better gradient signals, while the
34

---

SupCon Pre-training Loss Dynamics ImageNet
VarCon Pre-training Loss Dynamics ImageNet
Total Loss (Csupton)
10-
Total Loss (Evarco))
Average Loss Value
Log Posterior Term
KL Divergence Term
Average
10
Training Epochs
Training Epochs
(a) SupCon training loss
(b) VarCon training loss
Figure 7: Training loss trajectories ImageNet with ResNet-50 and batch size 4096 under full 50-epoch training regime. (a) SupCon exhibits slow convergence, decreasing from 11.142 (epoch 1) to 4.578 (epoch 50). (b) VarCon demonstrates rapid convergence with the total loss dropping from 9.997 (epoch 1) to 1. 1.138 (epoch 50), while both components decrease synergistically: the KI. divergence term reduces from 4.978 (epoch 1) to 0.538 (epoch 50) and the negative log-posterior term from 5.019 (epoch 1) to 0.599 (epoch 50). This balanced reduction validates our variational formulation' effectiveness in jointly optimizing distributional alignment and class-specific representation learning.
Table 7: Distribution of adaptive temperature 72 across ImageNet samples at key epochs during full 50-epoch training with ResNet-50, c = 0.02, TI = 0.1, and batch size 4096. The temperature 72 [0.08, 0.12] modulates gradient strength, with lower values indicating stronger gradients for difficult samples. "25th Pet" and "75th Pet" represent the first quartile (25th percentile) and third quartile (75th percentile), respectively.
Epoch
Mean
Std Dev
Min
25th Pet
Median
75th Pct
Max
10
0.09378
0.00961
0.08001
0.08550
0.09213
0.10110
0.11878
0.02
20
0.09714
0,01037
0.08000
0.08806
0.09685
0.10610
0.11873
0.02
30
0.09979
0.01078
0.08000
0.09063
0.10076
0.10935
0.11889
0.02
40
0.10326
0.01092
0.08001
0.09505
0.10589
0.11278
0.11916
0.02
50
0.10656
0.01059
0.08001
0.10040
0.11042
0.11520
0.11910
0.02
confidence-adaptiie temperature mechanism accelerates learning without sacrificing representation
quality.
D.2 Quantitative Analysis of Adaptive Temperature Dynamics
To investigate the effectiveness of our confidence- distribution of T2 across all ImageNet samples during 50-epoch training run with ResNet-50, fixed hyperparameters (c = 0.02, T = 0.1) and batch size 4096, constraining T2 E [0.08, 0.12]. Table and Figure 4 reveal systematic evolution of the temperature distribution training progresses. At epoch 10, the mean T2 = 0.09378 with the 25th percentile at 0.08550 indicates that over 25% of samples receive maximal gradient strength, while the relatively low 75th percentile of 0.100 10 shows that even easier, more confident samples remain below the midpoint of the temperature range. The distribution shifts consistently across epochs, visualized in Figure 4a: the mean increases by 13.6% (from 0.09378 to O. 10666,) while more dramatic shifts occur in the quartiles- the 25th percentile rises by 17.4% (from 0.08550 10040) and the median increases by 19.9% (from 0.09213 to 0.1104 indicating accelerating confidence gains on the majority of samples. The minimum and maximum values follow divergent trajectories: while the minimum remains fixed around 0.08001 throughout training (suggesting persistent subset of challenging samples). Figure 4b reveals that the density of low-T2 samples progressively decreases from epoch 10 to 50, demonstrating that hard samples are gradually pushed closer to their corresponding class centroids through embedding refinement. Meanwhile, the maximum stabilizes around 0. 119, and the 75th percentile increases from 0. 100110 to 0. 1520 (14.0% growth), demonstrating that high-confidence samples progressively receive more relaxed gradients. The standard deviation exhibits a subtle but meaningful pattern, initially increasing from 0.00961 to 0.01092 (epochs 10-40) before slightly decreasing to 0.01059
temperature mechanism, we analyze the
35

---

Table Distribution of adaptive temperature 72 across ImageNet samples with learnable during a full 200-epoch training with ResNet-50, 0.1, and batch size 4096. The temperature T2 modulates gradient strength, with lower values indicating stronger gradients for difficult samples. The learnable parameter evolves dynamically during training.
Epoch
Mean
Std Dev
Min
25th Pct
Median
75th Pct
Max
40
0.09365
0.00958
0,08151
0.08542
0.09205
0.10098
0.11845
0,0185
80
0.09728
0.01042
0.07950
0.08815
0.09698
0.10625
0.11885
0.0205
120
0.09995
0.01085
0.07850
0.09075
0.10090
0.10950
0.11902
0.0215
160
0.10310
0.01088
0.07921
0.09492
0.10575
0.11265
0.11895
0.0208
200
0.10642
0,01056
0.07981
0.10028
0.11030
0.11508
0.11887
0.0202
at epoch 50, suggesting initial divergence as samples differentiate followed by stabilization. This nuanced evolution, with faster increases in upper quantiles compared to lower ones, illustrates that our confidence-weigheee formulation successfully implements differential treatment: maintaining strong gradients for persistently difficult samples while progressively relaxing constraints on well-learned examples, all emerging naturally without manual scheduling. This emergent behavior represents a fundamental advance over existing approaches that either apply uniform treatment to all samples or rely on handcrafted scheduling rules. The emergence of this heterogeneous learning dynamic, where gradient strength naturally adapts to each sample's ' learning progress, demonstrates how our variational framework translates theoretical principles into practical benefits. Rather than requiring explicit sample weighting or curriculum strategies, the adaptive behavior arises from the interaction temperature mechanism, providing
between the KL divergence term and the confidence-batt principled solution rather than manual scheduling approaches.
To examine the adaptive capacity of our framework with learnable parameters, we extend our analysis full 200-epoch training schedule. Table 8 demonstrates the co-evolution of r2 distribution and the learnable parameter. The c parameter exhibits non monotonic dynamics: starting at 0,0185 (epoch 40), increasing to peak of 0.0215 (epoch 120), then gradually decreasing to 0.0202 (epoch 200). This trajectory refects the model's evolving calibration requirements across different training phases. The 72 distribution under learnable shows similar overall trends to the fixed-c setting but with notable distinctions. The mean 72 increases from 0.09365 (epoch 40) to 0.10642 (epoch 200), representing 13.6% growth comparable to the fixed case. However, the distribution exhibits greater dynamic range: minimum values decrease from 0.08 (epoch 40) to 0.07981 (epoch 200), enabling more aggressive gradient modulation for the hardest samples. The 25th percentile increases by 17.3% (0.08542 to 0. 0.10028) and the median by 19.9% (0.09205 to 0.11030), demonstrating consistent confidence development across the sample population. The interplay between e and T2 reveals sophisticated adaptation patterns. During epochs 40-120, increases from 0.0185 to 0.0215, expanding the temperature range to facilitate stronger differentiation between easy and hard samples. As training progresses (epochs 120-200), e decreases to 0.0202 while maintaining high mean 72, indicating the model has established robust class boundaries and requires less aggressive temperature modulation. The standard deviation remains stable (0.00955 0.01056) despite evolving e, demonstrating that the learnable mechanism maintains consistent relative differentiation across samples while adjusting absolute scale. This self-regulated behavior emerges without explicit scheduling, validating our variational framework's 's capacity for autonomous adaptation to training dynamics.
D.3 Memory Efficiency Analysis
Beyond training efficiency, we analyze the memory footprint of VarCon compared to SupCon during training. Figure 8 presents peak GPU memory consumption per-device (specifically Rank 0) over 50 training epochs on ImageNet with ResNet-50 and per-device batch size 512. VarCon demonstrates consistent memory efficiency advantages: after the initial epoch, VarCon maintains steady peak memory usage of 43.09 GB per GPU compared to SupCon's 's 43.25 GB, representing a 0.16 GB (0.37%) reduction per device. While this difference appears modest, it becomes significant in multi- GPU training scenarios where even small per-device savings enable larger effective batch sizes or accommodate additional model capacity within fixed memory constraints.
36

---

Peak VRAM Usage per Epoch ImageNet
Peak VRAM Usage (GB)
43.0
44.76 GB
VarCon
SupCon
44.5
44.76
44.0
43.5
43.0
30 Training Epochs
20
50
Figure 8: Per-GPU peak memory usage comparison between VarCon and SupCon during a full 50-epoch ImageNet training with ResNet-50. Experiments conducted with total batch size 4096 distributed across 8 A100 GPUs (per-GPU batch size 512), with VarCon using 0.02 and π1 0.1. VarCon consistently requires O. 16 GB less memory per device after the first epoch, maintaining 43.09 GB versus SupCon's 43.25 GB throughout training.
Table 9: Comparison of adaptive regularization methods across four key dimensions. Dynamic Target: modifies target distribution based on sample or model state; Dynamic Gradient: adjusts gradient magnitude during training; Sample-Specific: per-sample adaptive treatment; Self-Adaptive automatic confidence-based updates without manual scheduling.
Method
Dynamic Target
Dynamic Gradient Sample-Specifet Self-Adaptive
Label Smoothing [57]
X
Mixup [67]
x
x
AdaFocal [27]
X
V
Dual Focal Loss [58]
X
X
Local Adaptive LS [4]
X
X
LoT Regularization [36]
X
VarCon (Ours)
The memory savings arise from VarCon's computational structure: by computing class centroids dynamically rather than exhaustive pairwise comparisons, VarCon reduces the memory required for intermediate similarity matrices from O(B²) to O(B -C).. where B denotes the per-device batch size and Ca represents the number of unique classes device. Importantly, Ca is bounded by B (i.e., C. S B), as each sample belongs exactly one class. This bound ensures that VarCon's 's memory complexity remains strictly lower than SupCon's quadratic requirement. Furthermore, the stable memory consumption across epochs indicates that the confidence-adaptive mechanism introduces negligible overhead despite its dynamic nature. These efficiency gains, combined with VarCon's 's faster convergence and superior accuracy, establish as a more resource-efficien approach for contrastive learning at scale.
D.4 Adaptive Temperature versus Existing Regularization Methods
Traditional methods like label smoothing [57] and mixup [67] apply uniform regularization to all samples, while focal loss variants such as AdaFocal [27] and Dual Focal Loss [58] adjust gradient magnitudes based on sample difficulty. Recent approaches achieve sample-specific adaptation: Local Adaptive Label Smoothing [4] modifies smoothing based neighborhood structure, while Learning from Teaching regularization [36] combines target and gradient modifications through teacher-student dynamics. As summarized in Table 9, these methods primarily optimize either target smoothing or gradient scaling, with few achieving both through self-adaptive mechanisms.
VarCon's adaptive temperature mechanism provides bidirectional adaptation through the interaction between KL divergence and confidence-bart based temperature scaling. For samples with low confidence, the mechanism produces sharper target distributions (lower 72) and stronger gradients, while confident samples receive smoother targets (higher 72) and gentler gradients. This emerges naturally from our variational formulation without requiring manual scheduling validation set monitoring.
37

---

0.90
0.50
0.29
0.27
0.32
0.31
0.27
0.26
0.25
0.25
0.26
0.39
1.0
lyex
0.50
0.85
0.43
0.30
0.30
0.33
0.27
0.27
0.29
0.26
0.29
0.29
Labby
0.29
0.43
0.80
0.23
0.28
0.24
0.19
0.25
0.19
0.21
0.22
0.8
Maltese dog
0.27
0.84
0.27
0.23
0.24
0.28
0.23
0.30
0.26
German shepherd
0.28
0.0
Doberman
basketball
0.27
0.23
0.22
0.27
-0.4
volleyblll
bonnis ball
acoustic
0.0
saxophone-
Gemaa shepherd
Dobeman
volleyball
accussie grand Preustic gutar
zarophond
Malteze
Figure 9: Learned cosine similarity matrix between class embeddings ImageNet using VarCon with ResNet-50. The matrix visualizes pairwise similarities among 12 representative classes from four semantic groups: felines (cougar, lynx, tabby), dogs (Maltese, German shepherd, Doberman), balls (baskethall, volleyball, tennis ball), and musical instruments (acoustic guitar, grand piano, saxophone). Higher values indicate stronger learned semantic relationships between classes based solely on visual features.
The scarcity of bidirectional self-adaptive methods in the literature reflects inherent optimization chal- lenges: simultaneous target and gradient modifications can create conflicting optimization objectives or redundant regularization effects. Our approach addresses these challenges through the principled integration of temperature adaptation within the ELBO framework, where both modifications arise from unified objective rather than separate mechanisms. This integration eliminates the need for balancing multiple hyperparameters while maintaining theoretical consistency with the variational objective.
E
Additional Related Work
E.1
Multi-Modal and Single-Modal Approaches
X-CLR [55] represents recent advancement that leverages multi-modal information for contrastive learning. Table 10 summarizes the key methodological differences between SupCon, X-CLR, and VarCon.
These methodological differences highlight distinct design philosophies. SupCon extends self- supervised contrastive learning the supervised setting by treating all same-class samples positive
38

---

Table 10: Methodological comparison of supervised contrastive learnigg approaches.
Aspect
SupCon
X-CL.R
VarCon
Data Requirements
Single modality (images)
Dual modality (images + text)
Single modality (images)
Label Usage
Class labels for defining
Class names converted to text
Class labels for compating centroids
posttvehhegative pains
descriptions for similarity
and defining larget distributions
Similarit Construction
Binary (same class = 1.
Pre-comauted similarity matrix
Dynamic distribution go(r'(e)
different class 0)
from encoder cosine similarities
adjusted by confidence pe(r) [2]
Theoretical Foundation
Heuristic extension of self supervised learning
Heuristic soft labeling
Variational inference with ELBO
replacing hinary similarities
maximization over latent variables
Objective Function
InfoNCE with multiple
Modified InfoNCE with soft
KL. divergence log posterior terms derived from ELBO
positives per class
similarity targets from text
Computational Requirements
Single encoder pairwise comparisons
Dual encoder (vision + text)
Single encoder
similarity matrix compatation
class controid computation
Applicability
Any labeled visual dataset
Requires meaningful text
Any labelod visual dataset
descriptions for classes
Table 11: Core innovations and target settings of contrastive methods.
Method
Core Innovation
Target Setting
SupCon
Supervised contrastive loss
Balanced datasets
VarCon
Variational inference, ELBO objective
Balanced datasets
PaCo
Parametric centers + momentum encoder Simplified parametric centers
Long-tailed datasets
GPaCo
Balanced and long-tailed datasets
pairs, maintaining binary similarity relationships. X-CLR introduces graded similarities by leveraging semantic relationships encoded in class names through text encoders, enabling the model to learn that certain classes are more similar than others. VarCon takes a fundamentally different approach by reformulating the problem through variational inference, where the similarity construction emerges dynamically from the model's evolving confidence rather than being predetermined. This allows VarCon to adapt its supervision intensity per sample throughout training, providing stronger gradients for uncertain samples while relaxing constraints on well-learned examples. Notably, as shown in Figure 9, VarCon successfully learns semantically meaningful inter-class relationships purely from visual features, with related categories (e.g., different feline species or musical instruments) exhibiting higher similarities. While X-CLR requires auxiliary text modality to extract such semantic relationships, VarCon achieves adaptive learning using only visual data, making it applicable to any labeled dataset without requiring meaningful text descriptions. Although our variational framework could naturally extend to multi-modal settings, we first establish its effectiveness in single-modality scenarios to validate the theoretical contributions before exploring cross-modal applications.
E,2 Synergy with Eaas-lnmblldeeed Methods
VarCon is designed as general framework for supervised contrastive learning, focusing on improving representation quality through variational inference rather than addressing specific data distribution challenges. However, its theoretical foundation enables effective combination with specialized methods designed for class imbalance. PaCo [16] introduces parametric class centers maintained through exponential moving averages, specifically designed to handle long-tailed distributions where some classes have significantly fewer samples. GPaCo [17] simplifies this approach by removing the momentum encoder requirement while maintaining effectiveness on both balanced and imbalanced datasets. Table 11 illustrates how these methods address different aspects of the learning problem.
While VarCon focuses on general representation learning improvements, PaCo and GPaCo provide complementary mechanisms for handling class imbalance. Tables 12 and 13 demonstrate that when combined, VarCon's 's improved representation quality and PaCo/GPaCo's 's rebalancing strategies achieve superior results, suggesting that variational objectives and imbalance- handling are orthogonal dimensions of improvement. The performance gains from combining VarCon with PaCo/GPaCo validate that our variational framework provides a strong foundation that can be further enhanced with specialized techniques when dealing with imbalanced data distributions.
39

---

Table 12: Top-1 accuracy on ImageNet with ResNet-50.
Method
Augmentation
Top-1 Acc
SupCon
SimAugment 77.82
VarCon
SimAugment
78.23
PaCo
SimAugment
78.70
PaCo
RandAugment
79.30
GPaCo
RandAugment
79.50
VarCon+PaCo
SimAugment
79.19
VarCon+PaCo
RandAugment
79.86
VarCon+GPaCo
RandAugment
79.94
Table 13: Top-1 accuracy on CIFAR- 100 with ResNet-50.
Method
Top-1 Acc
SupCon VarCon
76.57
78.29
PaCo
79.10
GPaCo
80.30
VarCon+PaCo
80.57
VarCon+GPaCo
81.29
40


Reviews:
[{"review_id": "1s9UTbvDYU", "reviewer": "Reviewer_5QVM", "summary": "This paper introduces Variational Supervised Contrastive Learning (VarCon), a framework that casts supervised contrastive learning as variational inference over latent class variables. The method derives an ELBO-based objective, incorporating a KL divergence term between an auxiliary distribution and the model posterior, alongside a log-likelihood term. Additionally, the paper proposes a confidence-adaptive temperature scaling mechanism to modulate label sharpness based on prediction confidence. The approach is evaluated on standard benchmarks (CIFAR, ImageNet) and shows consistent performance improvements over SupCon, particularly in few-shot learning and robustness under image corruption.", "questions": "- Could you provide ablations to quantify the individual contribution of the KL term, log-likelihood, and confidence-adaptive temperature?\n\n- Could you visualize confidence-adaptive temperature values during training to confirm that they correlate with sample confidence?\n\n- Can you provide t-SNE/UMAP visualizations or inter/intra-class distance statistics to support your claim of better representation structure?\n\n- Can you provide an empirical comparison against other SupCon extensions that handle sample difficulty, such as PaCo and GPaCo?", "limitations": "The authors do not adequately discuss limitations. At minimum, it should discuss computational overhead, risk of overconfidence, or calibration issues under distribution shift. No societal concerns are discussed, though none are obvious.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths\n\n- The ELBO-based objective is grounded in variational inference theory and clearly derived.\n\n- Empirical results are broad in scope and demonstrate consistent gains across tasks including few-shot learning, robustness, and transfer.\n\n- The method is architecture-agnostic and demonstrates performance across ResNet and ViT models.\n\nWeaknesses\n\n- The paper lacks component-wise ablations to isolate the contributions of KL divergence, log-likelihood, and confidence-adaptive temperature scaling.\n\n- Key claims such as improved intra-class compactness and difficulty-aware learning are not supported by empirical evidence (e.g., visualization or metrics).\n\n- The novelty is limited as the core idea builds on prior work; SimVAE[1] already explored a similar ELBO-style objective for self-supervised learning.\n\n- Important comparisons to other advanced SupCon extensions that also address sample difficulty, such as PaCo[2] and GPaCo[3], are missing.\n\n- There is limited insight into model behavior beyond accuracy, such as representation structure.\n\n[1] Bizeul, Alice, Bernhard Schölkopf, and Carl Allen. \"A Probabilistic Model behind Self-Supervised Learning.\" Transactions on Machine Learning Research.\n\n[2] Cui, Jiequan, et al. \"Parametric contrastive learning.\" Proceedings of the IEEE/CVF international conference on computer vision. 2021.\n\n[3] Cui, Jiequan, et al. \"Generalized parametric contrastive learning.\" IEEE Transactions on Pattern Analysis and Machine Intelligence 46.12 (2023): 7463-7474.", "was_revised": true, "final_justification": "The paper presents a solid and well-motivated extension of supervised contrastive learning through a variational inference framework, supported by comprehensive experiments across multiple datasets and architectures. While the novelty is moderate, the rebuttal effectively addressed concerns regarding component contributions, representation quality evidence, and comparisons to related methods. The theoretical grounding is sound, results are consistently strong, and limitations are now better articulated. Overall, the work meets the bar for acceptance, though it remains a borderline case due to incremental conceptual advances.", "rebuttal": "We greatly thank the reviewer for recognizing our theoretical grounding in variational inference, the consistent empirical gains across diverse tasks, and the architecture-agnostic nature of our method. We appreciate your constructive feedback and will address each concern below with additional ablations, visualizations, and comparisons to further validate our contributions.\n\n> ### Component ablations (W1/Q1)\n\nWe acknowledge the reviewer's request for component-wise ablations. We note that according to our ELBO derivation in Eq. (5):\n\n$\\mathcal{L}\\_{ELBO} = D\\_{KL}(q\\_\\phi(r'|z) || p\\_\\theta(r'|z)) + \\log p(r) - \\log p\\_\\theta(r|z) - \\log p\\_\\theta(z)$\n\nAs discussed in lines 131-135, $\\log p_\\theta(z)$ would encourage high likelihood throughout the embedding space without class distinction, while $\\log p(r)$ is a fixed constant based on the dataset's class distribution, which are are uninformative for contrastive learning. Therefore, our VarCon loss (Eq. 7) focuses on the contrastive-relevant components:\n\n$\\mathcal{L}\\_{VarCon} = D\\_{KL}(q_\\phi(r'|z) || p\\_\\theta(r'|z)) - \\log p\\_\\theta(r|z)$\n\nBoth terms are mathematically necessary to maintain the validity of our variational bound, removing either would break the theoretical connection to the ELBO. In **Appendix Section C**, we provide comprehensive gradient analysis demonstrating how these components affect learning dynamics. Specifically, we show the gradient flow w.r.t. embeddings $z$ and analyze how the sample-specific confidence-adaptive temperature $\\tau_2(z) = (\\tau_1 - \\epsilon) + 2\\epsilon p_\\theta(r|z)$ modulates learning, where $\\tau_1$ is fixed and $\\epsilon$ is learnable.\n\nAlthough both components are essential to the VarCon loss, we provide ablation results to isolate their individual contributions:\n\n**Table 1:** Component ablation study on ImageNet with ResNet-50. All experiments use batch size 4096, 350 epochs, learning rate 2.0, and AutoAugment.\n\n| **Method** | **Components** | **Top-1 Acc (%)** | **Top-5 Acc (%)** |\n|------------|----------------|-------------------|-------------------|\n| NLL only | $-\\log p_\\theta(r\\|z)$ | 78.34 | 93.98 |\n| KL only | $D_{KL}(q_\\phi(r'\\|z) \\|\\| p_\\theta(r'\\|z))$ | 78.45 | 94.18 |\n| VarCon ($\\epsilon=0$) | Both terms, fixed $\\tau_2=\\tau_1=0.1$ | 78.87 | 94.33 |\n| **VarCon (Full)** | **Both terms, adaptive $\\tau_2$** | **79.36** | **94.37** |\n\nAs noted in Section 4.5 (Figure 3b), even with $\\epsilon=0$ (no adaptive temperature), VarCon achieves 78.87% Top-1 accuracy, already surpassing SupCon's 78.72%. The full model with adaptive temperature further improves performance, demonstrating the value of our confidence-based modulation mechanism.\n\n> ### Empirical evidence of superior representation quality (W2/W5/Q2/Q3)\n\nWe thank the reviewer for requesting empirical evidence on representation quality. We summarize **existing results** and provide additional experiments below.\n\n**Intra-class compactness:** Figure 3(a) demonstrates VarCon's consistently superior KNN classifier performance across all K values, which directly reflect better local neighborhood structure. In Appendix B.3, Figures 4-5 show t-SNE visualizations where VarCon achieves clearer cluster separation after 200 epochs than SupCon after 350 epochs, evidenced by more distinct cluster boundaries and reduced inter-class overlap. Additionally, Appendix B.4 (Table 6) provides hierarchical clustering analysis, which inspects embedding quality without learnable parameters that could mask poor features:\n\n**Table 2:** Hierarchical clustering evaluation on ImageNet embeddings using Ward linkage. All metrics: higher is better.\n\n| **Method** | **ARI** | **NMI** | **Homogeneity** | **Completeness** | **V-measure** | **Purity** |\n|------------|---------|---------|-----------------|------------------|---------------|------------|\n| SupCon | 0.613 | 0.888 | 0.886 | 0.890 | 0.888 | 0.755 |\n| VarCon ($\\epsilon$=0) | 0.627 | 0.892 | 0.890 | 0.892 | 0.891 | 0.766 |\n| **VarCon** | **0.634** | **0.895** | **0.893** | **0.896** | **0.895** | **0.774** |\n\nThese metrics quantify clustering quality: ARI measures agreement with ground truth, NMI captures mutual information between clusters and labels, Homogeneity/Completeness ensure class-cluster alignment, V-measure balances both, and Purity measures correct assignments. VarCon's improvements across all metrics confirm superior semantic organization.\n\n**Difficulty-aware learning:** We conducted comprehensive tracking of confidence-adaptive temperature $\\tau_2$ throughout ImageNet training (200 epochs). Due to space constraints, we refer to our detailed response to Reviewer s4KC's Q1, which includes: (1) evolution of learnable $\\epsilon$ from 0.0185 (epoch 40) to peak at 0.0215 (epoch 120) then stabilize at 0.0202 (epoch 200), (2) mean $\\tau_2$ increasing 13.6% (0.09365 to 0.10642), (3) distributional analysis showing persistent low $\\tau_2$ for hard samples (minimum stays ~0.08) while confident samples progressively receive higher $\\tau_2$ (75th percentile rises from 0.10098 to 0.11508), confirming our mechanism adapts to sample difficulty as intended.\n\n> ### Comparisons with PaCo/GPaCo (W4)\n\nWe thank the reviewer for suggesting comparisons with PaCo and GPaCo. We note that these methods address orthogonal problems and could be combined synergistically:\n\n**Table 3:** Comparisons with SupCon, VarCon, PaCo, and GPaCo.\n| **Method** | **Core Innovation** | **Target Setting** |\n|------------|---------------------|-------------------|\n| SupCon | Extends contrastive loss to supervised setting | Balanced datasets, general classification |\n| **VarCon** | **Variational inference framework, ELBO objective** | **Balanced datasets, general classification** |\n| PaCo | Parametric class centers + momentum encoder | Long-tailed/imbalanced datasets |\n| GPaCo | Simplified parametric centers (no momentum) | Both long-tailed and balanced datasets |\n\nVarCon reformulates supervised contrastive learning through variational inference to address SupCon's limitations (inadvertent semantic separation, excessive batch dependency), while PaCo/GPaCo specifically tackle class imbalance through parametric rebalancing. Therefore, VarCon and PaCo/GPaCo are orthogonal approaches rather than competing methods. Nevertheless, we provide comparative results by combining VarCon with PaCo/GPaCo's class rebalancing strategy (VarCon+PaCo/GPaCo), leveraging our variational framework's improved representation quality alongside PaCo/GPaCo's handling of imbalanced data:\n\n**Table 4:** Top-1 accuracy on full ImageNet with ResNet-50.\n| **Method** | **Augmentation** | **Top-1 Acc** |\n|------------|------------------|---------------|\n| SupCon | SimAugment | 77.82 |\n| VarCon | SimAugment | 78.23 |\n| PaCo | SimAugment | 78.70 |\n| PaCo | RandAugment | 79.30 |\n| GPaCo | RandAugment | 79.50 |\n| **VarCon+PaCo** | SimAugment | **79.19** |\n| **VarCon+PaCo** | RandAugment | **79.86** |\n| **VarCon+GPaCo** | RandAugment | **79.94** |\n\n**Table 5:** Top-1 accuracy on full CIFAR-100 with ResNet-50.\n| **Method** | **Dataset** | **Top-1 Acc** |\n|------------|-------------|---------------|\n| SupCon | CIFAR-100 | 76.57 |\n| VarCon | CIFAR-100 | 78.29 |\n| PaCo | CIFAR-100 | 79.10 |\n| GPaCo | CIFAR-100 | 80.30 |\n| **VarCon+PaCo** | CIFAR-100 | **80.57** |\n| **VarCon+GPaCo** | CIFAR-100 | **81.29** |\n\nDue to the considerable computational overhead of PaCo/GPaCo (particularly PaCo), and given time constraints, we conducted experiments for SupCon, VarCon, and VarCon+PaCo/GPaCo combinations, while using reported results from original papers for standalone PaCo/GPaCo. The results demonstrate that VarCon integrates effectively with PaCo/GPaCo, achieving new state-of-the-art performance: 79.94% on ImageNet and 81.29% on CIFAR-100. We leave comprehensive evaluation on truly long-tailed datasets (CIFAR-100-LT, ImageNet-LT) as future work and would be happy to include these results in the revised manuscript.\n\n> ### Regarding novelty (W3)\n\nWe respectfully disagree with the characterization that VarCon builds upon SimVAE. While both use variational inference tools, they address fundamentally different problems with distinct objectives and mechanisms:\n\n**Table 2:** Key differences between SimVAE and VarCon.\n\n| **Aspect** | **SimVAE** | **VarCon** |\n|------------|------------|------------|\n| **Research Goal** | Unified generative theory for SSL | High-performance supervised learning |\n| **Learning Paradigm** | Self-supervised (no labels) | Fully supervised (uses labels) |\n| **Variational Bound** | Data likelihood: $\\log p(x\\|y)$ | Embedding likelihood: $\\log p(z\\|r)$ |\n| **Loss Components** | Reconstruction + KL + Prior matching | KL divergence + Log-posterior only |\n| **Reconstruction Term** | Essential (prevents collapse) | None (representation-focused) |\n| **Core Mechanism** | Generative modeling to preserve style | Confidence-adaptive temperature $\\tau_2(z)$ |\n| **Architecture** | Encoder + Decoder | Encoder only |\n\nAs shown in the table, SimVAE is a self-supervised generative framework that models data generation, while VarCon is a novel supervised contrastive learning method that optimizes embedding distributions. The different research goals, learning paradigms, and mathematical formulations establish them as independent contributions addressing distinct problems in representation learning.\n\n\n> ### Limitation discussion\n\nWe thank the reviewer for this suggestion. VarCon reduces computational complexity from $O(B^2)$ to $O(B \\cdot C)$ and uses less GPU memory than SupCon (see response to Reviewer gzPQ Q1). Overconfidence is mitigated by our adaptive temperature mechanism (Section 3.2). Calibration under distribution shift is validated through strong transfer learning (Section 4.3, Table 4) and few-shot results (Section 4.2, Table 2). We will include the computational analysis in the revised manuscript and have already discussed broader limitations in Appendix A.1."}, {"review_id": "KP6ExJYKpy", "reviewer": "Reviewer_s4KC", "summary": "This paper introduces Variational Supervised Contrastive Learning (VarCon), a new method incorporating variational inference with (supervised) contrastive learning. \n\nThe VarCon loss is derived by extracting the contrastive-related components from the ELBO of $\\log p_\\theta(z|r)$, resulting in a KL augmented CE loss, where the classification probability is based on the representation similarity, akin to contrastive learning.\n\nThe authors then propose a confidence-adaptive temperature for the KL's target distribution, which tightens pull strength for hard samples and relaxes it for confident ones.\n\nExperiments on image classification tasks show that VarCon outperforms several existing contrastive learning baselines.", "questions": "1. As described in Weaknesses#2, can you include more ablation study regarding the adpative temperature? For example: \n    - Since $\\epsilon$ is learnable (L150), how does it evolve during training and thus affects the representations? \n    - Can adaptive temperature indeed improve the intra-class dispersion? Quantitative evidence supporting this claim would be valuable.\n    - Can similar effects by adaptive temperature be achieved with other regularization methods?\n2. Seems like it should be $\\log p_\\theta(z)$ rather than $p_\\theta$ in the LHS of Eq.(3)", "limitations": "The authors have discussed the limitations in their manuscript.", "rating": 4, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "### Strengths\n1. Theoretically Grounded and Intuitively Sound: The VarCon loss function is derived and simplified from the ELBO. This theoretical foundation leads to an intuitive combination of KL divergence and negative log-likelihood terms, which effectively regularizes embedding distributions while simultaneously maximizing classification likelihood.\n2. Demonstrated Superior Performance: VarCon consistently outperforms both existing self-supervised and supervised contrastive learning baselines across diverse datasets.\n\n### Weaknesses\n1. Inherent Supervision Requirement: \nThe VarCon method is fundamentally supervised. In contrast, a major advantage of exsiting contrastive learning methods (SimCLR, etc.) is their power in self-supervised settings.\n2. Lack of In-depth Analysis of the Adaptive Temperature: \nThe confidence-adaptive temperature, $\\tau_2$, is a central innovation, yet its behavior is not fully explored. \nThe ablation study on $\\epsilon$ only shows its effect on overall performance and fails to adequately verify the stated rationale of *\"adjusting supervision intensity\"* (L153). A deeper analysis is needed to fully understand how this adaptive mechanism functions and contributes to the method's success.", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank you for recognizing VarCon as \"theoretically grounded and intuitively sound\" with our ELBO derivation leading to \"an intuitive combination of KL divergence and negative log-likelihood terms.\" We are particularly encouraged by your acknowledgment that VarCon \"consistently outperforms both existing self-supervised and supervised contrastive learning baselines\", achieving 79.36% on ImageNet (vs. 78.72% SupCon) and 78.29% on CIFAR-100 (vs. 76.57%), while converging in 200 epochs (vs. 350) with smaller batch sizes (2048 vs. 4096). Our confidence-adaptive temperature enables sample-specific learning for both hard samples and confident ones. We will address your comments as follows.\n\n> ### Regarding supervised framework (W1)\n\nWe respectfully note that supervised learning remains an important class of problems with abundant real-world applications. Our variational framework naturally extends to self-supervised settings by replacing class centroids $w_r$ with instance centroids $w_i$ computed from augmented views (similar to SimCLR's $z_i, z_j$ but using centroids instead of pairs). The confidence-adaptive temperature $\\tau_2(z)$ would adapt based on the model's confidence in the instance centroid representation, analogous to how it currently adapts to classification confidence. We chose supervised settings first to establish theoretical soundness, with self-supervised extensions as promising future work.\n\n> ### Additional adaptive temperature analysis (W2)\n\nWe appreciate your interest in our confidence-adaptive temperature mechanism. **Section 4.5** and **Figure 3b** show that $\\epsilon=0.02$ yields optimal ImageNet performance, while even $\\epsilon=0$ (no adaptation, ELBO-only) achieves 78.87%, surpassing SupCon. This demonstrates VarCon's effectiveness stems from both ELBO-derived loss and adaptive mechanism. Since $\\epsilon$ is learnable, we provide comprehensive experiments tracking $\\epsilon$ evolution and sample-specific $\\tau_2$ distributions throughout training below.\n\n> ### Temperature evolution analysis (Q1)\n\nWe have provided comprehensive experimental analysis demonstrating our adaptive temperature mechanism's effectiveness throughout the paper. This includes superior performance across all benchmarks (Sections 4.1-4.4, 4.6; Appendix B.2), systematic ablation of $\\epsilon$'s impact (Section 4.5, Figure 3b), and quantitative embedding space evaluation via KNN classification and hierarchical clustering (Appendix B.3-B.4). To further address your insightful questions, we now provide detailed tracking of $\\epsilon$ evolution and sample-specific $\\tau_2$ distributions throughout training. We address each of your three sub-questions below.\n\n> ### Evolution of learnable $\\epsilon$\n\n**Table 1:** Distribution of adaptive temperature $\\tau_2$ across ImageNet samples at key epochs during 200-epoch training with ResNet-50, $\\tau_1 = 0.1$, and batch size 4096. The temperature $\\tau_2$ modulates gradient strength, with lower values indicating stronger gradients for difficult samples.\n\n| Epoch | Mean $\\tau_2$ | Std Dev $\\tau_2$ | Min $\\tau_2$ | 25th Pct $\\tau_2$ | Median $\\tau_2$ | 75th Pct $\\tau_2$ | Max $\\tau_2$ | $\\epsilon$ |\n|-----------|-------------------|----------------------|------------------|------------------------|---------------------|------------------------|------------------|----------------|\n| 40 | 0.09365 | 0.00958 | 0.08151 | 0.08542 | 0.09205 | 0.10098 | 0.11845 | 0.0185 |\n| 80 | 0.09728 | 0.01042 | 0.07950 | 0.08815 | 0.09698 | 0.10625 | 0.11885 | 0.0205 |\n| 120 | 0.09995 | 0.01085 | 0.07850 | 0.09075 | 0.10090 | 0.10950 | 0.11902 | 0.0215 |\n| 160 | 0.10310 | 0.01088 | 0.07921 | 0.09492 | 0.10575 | 0.11265 | 0.11895 | 0.0208 |\n| 200 | 0.10642 | 0.01056 | 0.07981 | 0.10028 | 0.11030 | 0.11508 | 0.11887 | 0.0202 |\n\nLearnable $\\epsilon$ increases from 0.0185 (epoch 40) to peak at 0.0215 (epoch 120), then stabilizes at 0.0202 (epoch 200). Mean $\\tau_2$ rises 13.6% (0.09365→0.10642), reflecting growing model confidence. Key distributional shifts: 25th percentile +17.3% (0.08542→0.10028), median +19.9% (0.09205→0.11030), 75th percentile +14.0% (0.10098→0.11508), indicating progressively confident predictions and gentler gradients. Minimum $\\tau_2$ remains ~0.08 (persistent hard samples), maximum ~0.119. Standard deviation peaks at 0.01088 (epoch 160) before decreasing to 0.01056 (epoch 200).\n\nThis evolution enables distinctive feature learning early via stronger gradients on uncertain samples, while later stabilization prevents drift. Persistent low $\\tau_2$ ensures hard samples continue improving; higher $\\tau_2$ maintains stable clusters for confident samples. This adaptive mechanism explains VarCon's superiority over fixed $\\epsilon=0$ (79.36% vs. 78.87% on ImageNet).\n\n> ### Intra-class dispersion improvement\n\nYes, our adaptive temperature mechanism demonstrably improves intra-class dispersion through multiple quantitative measures. We provide three forms of empirical evidence:\n\n**Visual analysis:** **Appendix Section B.3** shows VarCon achieves 79.11% KNN accuracy after 200 epochs versus SupCon's 78.53% after 350 epochs, demonstrating both faster convergence and superior representations. The largest gains occur during epochs 100-150 (59.57% to 71.12%) when the KL divergence term aligns posteriors. This efficiency stems from: class-centroid alignment, probabilistic inter-class relationships, and confidence-adaptive supervision.\n\n**Quantitative analysis:** We evaluate embedding quality through two complementary approaches. First, KNN classification (**Section 4.5, Figure 3a**) directly probes representation quality without learnable parameters that could mask poor features. VarCon consistently outperforms SupCon across all K values, demonstrating superior local neighborhood structure.\nSecond, while **Appendix Section B.4 and Table 6** already present hierarchical clustering results, we appreciate your suggestion to ablate the adaptive temperature. Table 2 includes additional results with VarCon fixed $\\epsilon$=0 (constant $\\tau_2=\\tau_1=0.1$):\n\n**Table 2:** Hierarchical clustering evaluation on ImageNet embeddings using Ward linkage. All metrics: higher is better.\n\n| Method | ARI | NMI | Homogeneity | Completeness | V-measure | Purity |\n|------------|---------|---------|-----------------|------------------|---------------|------------|\n| SupCon | 0.613 | 0.888 | 0.886 | 0.890 | 0.888 | 0.755 |\n| VarCon ($\\epsilon$=0) | 0.627 | 0.892 | 0.890 | 0.892 | 0.891 | 0.766 |\n| **VarCon** | **0.634** | **0.895** | **0.893** | **0.896** | **0.895** | **0.774** |\n\nThese metrics evaluate clustering quality: ARI measures agreement with ground truth correcting for chance; NMI quantifies mutual information between clusters and labels; Homogeneity ensures each cluster contains only one class; Completeness ensures each class is in one cluster; V-measure balances both; Purity measures fraction of correctly clustered samples. Progressive improvements from SupCon to VarCon ($\\epsilon$=0) to VarCon demonstrate: (1) ELBO-derived loss alone improves semantic organization even without adaptation, and (2) adaptive temperature mechanism provides additional gains, validating both components' contributions to superior embedding structure.\n\n\n> ### Comparison with other regularization methods\n\nWe appreciate this insightful question. The adaptive temperature mechanism is specifically designed for our VarCon's loss—the KL divergence and confidence-adaptive temperature work synergistically in ways standalone regularization methods cannot replicate. Table 3 compares our approach with established techniques:\n\n**Table 3:** Comparison of regularization methods (Dynamic Target: modifies target distribution based on sample/model state; Dynamic Gradient: adjusts gradient magnitude during training; Sample-Specific: per-sample treatment; Self-Adaptive: confidence-based updates without manual scheduling).\n\n| Method | Dynamic Target | Dynamic Gradient | Sample-Specific | Self-Adaptive |\n|--------|----------------|------------------|-----------------|---------------|\n| Label Smoothing [1] | × | × | × | × |\n| Mixup [2] | × | × | × | × |\n| AdaFocal [3] | × | ✓ | ✓ | ✓ |\n| Dual Focal Loss [4] | × | ✓ | ✓ | × |\n| Local Adaptive LS [5] | ✓ | × | ✓ | × |\n| LoT Regularization [6] | ✓ | ✓ | ✓ | × |\n| **VarCon (Ours)** | **✓** | **✓** | **✓** | **✓** |\n\nVarCon uniquely provides **bidirectional adaptation**: $q_\\phi(r'|z)$ and $\\tau_2(z)$ adjust simultaneously based on confidence: hard samples receive sharper targets with stronger gradients, easy samples get smoother targets with gentler gradients. Existing methods modify only one aspect. While optimized for VarCon, our adaptive temperature has broader implications for temperature-based methods including LLM training and generative models.\n\n> ### Typo correction (Q2)\n\nThank you for pointing out this typo: It should be $\\log p_\\theta(z)$ rather than $p_\\theta(z)$. We have corrected this in our revised manuscript.\n\n\n[1] Szegedy, Christian, et al. Rethinking the Inception Architecture for Computer Vision. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2016): 2818-2826. \n\n[2] Zhang, Hongyi, et al. mixup: Beyond Empirical Risk Minimization. International Conference on Learning Representations (2018). \n\n[3] Ghose, Arindam, and Balaraman Ravindran. AdaFocal: Calibration-aware Adaptive Focal Loss. Advances in Neural Information Processing Systems 35 (2022): 19998-20009. \n\n[4] Tao, Linwei, et al. Dual Focal Loss for Calibration. International Conference on Machine Learning. PMLR, 2023: 33645-33660. \n\n[5] Bahri, Dara, and Heinrich Jiang. Locally Adaptive Label Smoothing Improves Predictive Churn. International Conference on Machine Learning. PMLR, 2021: 532-541. \n\n[6] Liu, Can, et al. Learning from Teaching Regularization: Generalizable Correlations Should be Easy to Imitate. Advances in Neural Information Processing Systems 37 (2024)."}, {"review_id": "vmJB6tuteV", "reviewer": "Reviewer_gzPQ", "summary": "VarCon is a framework that reinterprets supervised contrastive learning through the lens of variational inference over latent class variables. By deriving an ELBO for class-conditional likelihoods, VarCon replaces pairwise instance comparisons with efficient class-level objectives. Its loss combines a KL-divergence term and a log-posterior term, with confidence-adaptive temperature scaling to modulate supervision strength. The method outperforms prior supervised contrastive learning counterpart, SupCon, on multiple image benchmarks.", "questions": "- How does VarCon compare against X-CLR empirically?", "limitations": "yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "## Strengths\n- The paper is well-motivated and clearly written.\n- It provides a rigorous probabilistic foundation for supervised contrastive learning by deriving the objective from a variational ELBO.\n- The authors conduct a thorough evaluation of the impact of key hyperparameters.\n\n## Weaknesses\n- Although X-CLR is mentioned, the authors do not include empirical comparisons between VarCon and X-CLR, despite both methods requiring label supervision. This omission limits the ability to contextualize VarCon’s performance gains.\n\n(minor) Tables 3 and 4 appear out of order.\n\nOverall, I find this to be a strong paper with a solid contribution. However, the lack of discussion and empirical comparison with X-CLR, another supervised contrastive method, detracts from its completeness. A more thorough evaluation against X-CLR would strengthen the paper and better contextualize its improvements.", "was_revised": true, "final_justification": "After considering the rebuttal, author response, and other reviews, I recommend Accept for this paper. The authors have adequately addressed my main concern regarding the lack of empirical comparison with X-CLR by including additional results and demonstrating that VarCon outperforms X-CLR. The paper presents a well-motivated and rigorous probabilistic framework for supervised contrastive learning, supported by strong empirical results and thorough analysis. These contributions make it a valuable addition to the NeurIPS community and relevant to a broad audience interested in contrastive methods, representation learning, and probabilistic modeling.", "rebuttal": "We deeply appreciate your high recognition our work. We thank you for highlighting our theoretical contributions and clear presentation. The rigorous probabilistic foundation enables a principled understanding of contrastive learning through variational inference. Our ELBO-derived objective with confidence-adaptive temperature enables highly efficient and robust training. We now address your main concerns below:\n\n> ### X-CLR empirical comparison (W1)\n\nWe acknowledge that we mentioned X-CLR in our paper (lines 71-73) as the most recent advancement in contrastive learning: \"methods like X-CLR [48] further improve these frameworks by analyzing optimization dynamics and introducing graded similarities.\" While both methods utilize label supervision, they leverage labels in fundamentally different ways: VarCon uses labels purely for supervised learning (identifying which samples belong to the same class), whereas X-CLR extracts semantic information from class names to construct similarity relationships between different classes. This leads to several key differences that make direct empirical comparison challenging:\n\n**Key Methodological Differences:**\n\n| **Aspect** | **SupCon** | **X-CLR** | **VarCon** |\n|-----------|------------|-----------|------------|\n| **Data Requirements** | Single modality (images only) | Dual modality (images + text) | **Single modality (images only)** |\n| **Label Usage** | Class labels for defining positive/negative pairs | Class names converted to text descriptions for similarity computation | **Class labels for computing centroids and defining target distributions in ELBO** |\n| **Similarity Construction** | Binary (same class = 1, different class = 0) | Pre-computed similarity matrix from text encoder cosine similarities | **Dynamic distribution $q_{\\phi}(r'\\|z)$ adjusted by model's classification confidence $p_{\\theta}(r\\|z)$** |\n| **Theoretical Foundation** | Heuristic extension of self-supervised contrastive learning | Heuristic soft labeling replacing binary similarities | **Variational inference with ELBO maximization over latent class variables** |\n| **Objective Function** | InfoNCE with multiple positives per class | Modified InfoNCE with soft similarity targets derived from text | **KL divergence + log-posterior terms derived from ELBO** |\n| **Computational Requirements** | Single vision encoder + pairwise comparisons | Dual encoder (vision + text) + similarity matrix computation | **Single vision encoder + class centroid computation** |\n| **Applicability** | Any labeled visual dataset | Requires meaningful text descriptions for classes | **Any labeled visual dataset** |\n\nVarCon and X-CLR address complementary scenarios: X-CLR leverages auxiliary text when available, while VarCon operates on visual data alone. As multi-modal paradigms become prevalent, VarCon provides a novel theoretical framework for understanding contrastive learning through variational inference. While naturally extensible to multi-modal settings, we first establish effectiveness in single-modality scenarios. Therefore, our comparisons focus on single-modality SOTA baselines (SupCon, SimCLR, etc.) to validate theoretical contributions before exploring multi-modal extensions.\n\nDespite extensive efforts to locate X-CLR's implementation through official repositories and author pages, the code remains unavailable. Reproducing their method without original implementation risks introducing errors leading to unfair comparisons. Thus, we provide indirect comparisons using their reported numbers. We will clarify these methodological differences in our related work section and present detailed comparisons below.\n\n\n> ### Empirical Comparison with X-CLR through additional experiments (Q1)\n\nSince X-CLR's code is not publicly available, we address the reviewer's concern by comparing against their reported results while closely matching their experimental settings. X-CLR trains for 100 epochs with batch size 1024 on ImageNet, using AutoAugment and a ResNet-50 encoder. We focus our comparison on ImageNet as it provides the most controlled setting: both methods use the same visual data and class labels, with the key difference being that X-CLR additionally leverages text-based similarities computed from class name descriptions using Sentence Transformer while VarCon operates solely on visual representations.\n\nTo ensure fair comparison, we trained VarCon with ResNet-50 (matching X-CLR) under various settings and present comprehensive results:\n\n| Method | Epochs | Batch Size | Learning Rate | Augmentation | ImageNet Top-1 | ImageNet Real |\n|------------|------------|----------------|-------------------|------------------|-------------------|-------------------|\n| X-CLR (reported) | 100 | 1024 | 0.075 | AutoAugment | 75.6% | 81.5% |\n| VarCon (matching settings) | 100 | 1024 | 0.5 | AutoAugment | 77.25% | 82.3% |\n| VarCon | 100 | 2048 | 1.0 | AutoAugment | 77.81% | 82.7% |\n| VarCon | 200 | 2048 | 1.0 | AutoAugment | 79.13% | 83.5% |\n| VarCon | 350 | 4096 | 2.0 | AutoAugment | **79.36%** | **84.12%** |\n\nThis allows direct comparison under identical conditions (row 2) while demonstrating VarCon's performance across different training configurations. Note that learning rates follow the linear scaling rule (lr $\\propto$ batch_size/256), see main paper line 167. Even under X-CLR's exact batch size, VarCon outperforms by 1.65% on ImageNet and 0.8% on ImageNet Real, with the gaps increasing to 3.76% and 2.62% respectively under our full training regime.\n\n**Computational Requirements Comparison:**\n\n| Aspect | SupCon | X-CLR | **VarCon** |\n|------------|------------|-----------|------------|\n| **Encoders** | 1 (Vision only) | 2 (Vision + Text) | **1 (Vision only)** |\n| **Memory Complexity** | $O(B^2)$ | $O(B^2) + \\text{Similarity Storage}$ | $O(B \\cdot C)$ |\n| **Similarity Matrix** | $B \\times B$ dense matrix | $B \\times B$ dense matrix + pre-computed similarities | **$B \\times C$ matrix (C$\\leq$B)** |\n| **Peak GPU Memory** | 43.25 GB | Similar to SupCon* | **43.09 GB** |\n| **Additional Requirements** | None | Pre-computed text embeddings | **None** |\n| **Inference Cost** | Vision encoder + pairwise similarities | Vision encoder + similarity lookup | **Vision encoder + class centroids** |\n\n*Peak GPU memory with ResNet-50, batch size 2048. X-CLR should have similar GPU memory usage as SupCon during training since X-CLR pre-computes text similarities offline. In the table, $B$ denotes the per-device batch size and $C$ denotes the number of unique classes present in that batch.\n\n**Key Computational Advantages of VarCon:**\n\nWe conducted detailed memory profiling experiments (to be included in the final manuscript) measuring peak GPU memory consumption over 50 training epochs on ImageNet with ResNet-50 and per-device batch size 512. VarCon demonstrates consistent memory efficiency advantages: VarCon maintains a steady peak memory usage of 43.09 GB per GPU compared to SupCon's 43.25 GB. While this difference appears modest, it becomes significant in multi-GPU training scenarios where even small per-device savings can enable larger effective batch sizes or accommodate additional model capacity within fixed memory budgets.\n\nThe memory savings stem from VarCon's architectural efficiency: by computing class centroids on-the-fly rather than exhaustive pairwise comparisons, VarCon reduces the memory required for storing intermediate similarity matrices from $O(B^2)$ to $O(B \\cdot C)$. Crucially, $C$ (the number of unique classes per device) is bounded by $B$ (the per-device batch size), with $C \\leq B$ always holding true.\n\n\n**Self-Adaptive Confidence-Based Temperature vs X-CLR's Approach:**\n\nWhile X-CLR leverages external text metadata to organize the embedding space through pre-computed semantic similarities between class descriptions, VarCon learns well-organized representations using only visual data through our ELBO-derived loss and confidence-adaptive temperature mechanism: $\\tau_2(z) = (\\tau_1 - \\epsilon) + 2\\epsilon p_\\theta(r|z)$. This dynamically adjusts supervision strength, tightening for difficult samples and relaxing for confident ones, addressing the same challenge X-CLR tackles (avoiding SupCon's rigid binary similarities) but without requiring additional modalities.\n\nAs detailed in our response to Reviewer s4KC's Q1 (Table 1 on $\\epsilon$ evolution), extensive experiments demonstrate this mechanism produces high-quality representations, including clustering quality (Section 4.5, Appendix B.4) and embedding visualization (Appendix B.3) on top of the existing experiments in the main body of our paper.\n\n**VarCon Learned Semantic Similarities (Key Examples)**\n\n| Class Pair | Similarity | Relationship |\n|----------------|----------------|------------------|\n| cougar - lynx | 0.50 | Both felines |\n| lynx - tabby | 0.43 | Both felines |\n| basketball - volleyball | 0.50 | Both sports balls |\n| German shepherd - Doberman | 0.38 | Both dogs |\n| acoustic guitar - saxophone | 0.29 | Both instruments |\n\nVarCon successfully learns semantic relationships using only visual information: similar objects (felines: 0.43-0.50, sports balls: 0.50), demonstrating that our ELBO-derived objective organizes embeddings meaningfully without text-based similarities. We greatly appreciate X-CLR's pioneering work on leveraging multi-modal information to regularize embedding spaces, addressing crucial interpretability challenges in contrastive learning. While space constraints prevent replicating their full visualization (Figure 4), our results indicate that VarCon and X-CLR are complementary: X-CLR shows how auxiliary modalities guide representation learning, while VarCon achieves similar semantic organization through principled probabilistic frameworks alone. We will ensure X-CLR's contributions are properly highlighted in our revised Related Work section.\n\n>### Minor issues\n\nThank you for noting the table ordering issue. We have corrected this in our revision."}, {"review_id": "2mN5gY2xeG", "reviewer": "Reviewer_W5fH", "summary": "The authors have derived a supervised contrastive learning solution using variational inference. Furthermore, they have introduced an adaptive temperature based on the model’s confidence.", "questions": "- Is it necessary to include the ELBO derivations in the main paper? Similarly, are the ablation studies essential to present in the main text?\n- Did you select the models for comparison in Table 1 based on architectural similarity, or because they are state-of-the-art for the task?\n- How does the confidence-adaptive temperature differ from other importance-weighted approaches for similar objectives? Are there potentially better methods for this task?\n- Have you analyzed the learned latent representations and compared them to baseline models?", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 1, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths:\n\n- Interesting idea with the confidence-adaptive temperature.\n\nWeaknesses:\n\n- Only a small improvement; the results from the second model are within the standard error.\n\n- A large portion of the article focuses on less advanced areas, such as the ELBO derivations and the number of training epochs, which are not central to the main contribution.\n\n- More focus should be placed on demonstrating the effects of the introduced solutions.", "was_revised": true, "final_justification": "After taking the responses into account, I have recognized the novelty of the method.  is \n\nThe main reason for changing my score is based on my misunderstanding of the method's behaviour. Hence, I think the presentation could be improved, but I think the novelty is important enough for acceptance.", "rebuttal": "We thank you for your feedback and for recognizing our confidence-adaptive temperature as an interesting idea. We appreciate this opportunity to address potential misunderstandings regarding our empirical results and theoretical contributions. We hope to clarify VarCon's effectiveness by addressing each of your concerns below:\n\n> ### Regarding VarCon's performance gains (W1) \n\nWe respectfully disagree with the claim that our improvements are \"within standard error.\" This appears to be a **factual misunderstanding** of our results.\n\n**Table 1: Statistical significance of VarCon's improvements over SupCon.** All Top-1 accuracy values are directly from Table 1 in our main paper. We add statistical significance analysis showing the number of standard errors ($\\sigma$) calculated as: $\\sigma = \\frac{\\text{Gain}}{\\sqrt{\\text{SE}\\_{\\text{VarCon}}^2 + \\text{SE}\\_{\\text{SupCon}}^2}}$, where SE denotes standard error. Even our smallest improvement ($4.7\\sigma$) is highly significant.\n\n| Dataset | VarCon | SupCon | Absolute Gain | # of SEs ($\\sigma$) |\n|-------------|------------|-------------|-------------------|-------------------------|\n| CIFAR-10 | $95.94 \\pm 0.07$ | $95.51 \\pm 0.06$ | $+0.43\\%$ | $4.7\\sigma$ |\n| CIFAR-100 | $78.29 \\pm 0.08$ | $76.57 \\pm 0.10$ | $+1.72\\%$ | $13.4\\sigma$ |\n| ImageNet-100 | $86.34 \\pm 0.09$ | $85.06 \\pm 0.07$ | $+1.28\\%$ | $11.1\\sigma$ |\n| ImageNet | $79.36 \\pm 0.05$ | $78.72 \\pm 0.06$ | $+0.64\\%$ | $8.1\\sigma$ |\n\nAll improvements over SOTA SupCon are highly statistically significant, ranging from $4.7\\sigma$ to $13.4\\sigma$. \nAdditionally, we evaluate few-shot learning performance where $N$ denotes the number of training samples per class from ImageNet's 1000 classes. For each setting, we randomly sample $N$ images per class for training (if a class has fewer than $N$ samples, all samples are retained). With $N=50$, both methods must learn to distinguish 1000 classes using merely 50,000 total samples, which is an extremely challenging scenario resulting in \\~2.5% accuracy for both approaches. As data increases to N=100, 200, and 500, VarCon demonstrates clear advantages over SupCon (37.81% vs 36.57%, 51.10% vs 50.25%, and 65.83% vs 64.91% respectively), with improvements ranging from $3.3\\sigma$ to $4.2\\sigma$ significance. At $N=1000$, both methods reach similar performance (~73%) as they approach the ceiling achievable with this limited per-class data. Importantly, when trained on the full ImageNet dataset where no such ceiling exists, VarCon's advantage expands to $8.1\\sigma$, demonstrating that our variational framework's benefits scale with data availability. We faithfully report all results across the entire data spectrum to provide a complete picture of our method's performance characteristics.\n\n> ### Significance of ELBO derivation (our core contribution) (W2, Q1)\n\nWe respectfully disagree with the characterization of ELBO derivations and the number of training epochs as \"less advanced areas\" and \"not central to main contribution\". We believe these comments stem from a major misapprehension of our technical contribution and would like to further elaborate on that. \nFirstly, Section 3.1's derivation of the ELBO of Class-Conditional Likelihood is the **key theoretical foundation** of our approach and is **a novel and non-trivial contribution** - establishing connection between contrastive learning and variational inference for the first time. This derivation lay the foundation of the formulation of our VarCon loss that explicitly regulates embedding distributions and enforces appropriate semantic relationships between samples.  It also motivate our confidence-adaptive temperature mechanism, which addresses the fundamental challenge of hard samples being difficult to learn while easy samples overfit. Moving this to the appendix would obscure our core theoretical contribution.\n**We were encouraged that other reviewers (gzPQ, s4KC, 5QVM) unanimously recognized the significance of this theoretical contribution.** To prevent any future misinterpretation, we will revise the manuscript to more explicitly highlight the centrality and novelty of this framework from the outset.\n    \nSecondly, we argue that interperting an established subject from a novel variational angle is fundamental to modern deep learning and comprises enough novelty and significance, as evidenced by:\n\n| Method/Paper | Year | Citations | Key ELBO Innovation | Impact on Deep Learning |\n|------------------|----------|---------------|-------------------------|----------------------------|\n| Auto-Encoding Variational Bayes (VAE) [1] | 2014 | 47,000+ | Introduced ELBO as unified objective: $\\mathcal{L} = \\mathbb{E}\\_q[\\log p(x\\|z)] - D\\_{KL}(q\\|\\|p)$ | Foundation of deep generative models |\n| Understanding Diffusion as ELBO [2] | 2023 | 100+ | All diffusion objectives as weighted ELBOs | Theoretical unification |\n\n[1] Kingma, Diederik P., and Max Welling. Auto-Encoding Variational Bayes. International Conference on Learning Representations (2014).\n\n[2] Diederik P. Kingma, and Ruiqi Gao. Understanding Diffusion Objectives as the ELBO with Simple Data Augmentation. Advances in Neural Information Processing Systems 36 (2023).\n\n**Regarding the importance of training epoch analysis**, this analysis directly validates our method's efficiency. While SimCLR requires 1000 epochs, MoCo needs 200-800 epochs, and SupCon needs 350 epochs for peak performance, VarCon achieves superior results in just 200 epochs: a 1.75-5× speedup with significant practical implications for computational resources and research accessibility.\n    \n**Regarding content placement in main paper**, the ELBO derivations must remain in the main text as they constitute our core theoretical contribution detailed above. This derivation directly yields our loss formulation and confidence-adaptive temperature mechanism. Relocating this material to the appendix would obscure the theoretical foundation of our work. Similarly, ablation studies warrant inclusion in the main text as they demonstrate that our improvements arise from principled design choices rather than parameter tuning, establishing robustness and reproducibility—essential criteria for methodological contributions.\n\n> ### Comprehensiveness of our experimental verification (W3, Q2-4)\n\nOur empirical evaluation comprehensively demonstrates VarCon's advantages: Section 4.1 presents SOTA classification results on 4 datasets with 4 architectures; Section 4.2 shows consistent improvements across all few-shot learning regimes; Section 4.3 validates superior transfer learning on 12 diverse tasks; Section 4.4 tests robustness across different augmentations and architectures; Section 4.5 provides comprehensive ablation studies; Section 4.6 demonstrates superior corruption robustness on ImageNet-C; Appendix B.3 visualizes clear semantic organization evolution; and Appendix B.4 quantifies representation quality through clustering metrics. To our best knowledge, our paper provides one of the most comprehensive experimental validations in recent contrastive learning literature. We would appreciate specific suggestions on what additional demonstrations would be needed should this reviewer found the current results insufficient.\n\n\n> ### Regarding baseline selection criteria (Q2)\n\nWe selected models based on both criteria. Table 1 includes the most established SOTA methods in single-modality contrastive learning: SimCLR, MoCo V2, BYOL, SwAV, VicReg, and Barlow Twins. For supervised methods, we included Cross-Entropy (standard baseline) and SupCon (current SOTA).\nWe tested across ResNet-50/101/200 and ViT-Base, the most common architectures in visual contrastive learning. VarCon outperforms all baselines across every architecture, demonstrating our improvements stem from the method, not architectural advantages.\nWe focus on single-modality (where paired multi-modal data is often unavailable) as it represents the fundamental challenge. Establishing superiority here is essential before extending to multi-modal settings.\n\n> ### Regarding comparison with importance weighting (Q3)\n\nWhile specific importance-weighted approaches were not mentioned, we can clarify the fundamental differences from our method. Importance weighting scales loss values ($w_i \\cdot L_i$) while keeping targets fixed. Our confidence-adaptive temperature $\\tau_2(z) = (\\tau_1 - \\epsilon) + 2\\epsilon p_\\theta(r|z)$ instead modifies the target distribution itself, creating sharper distributions for hard samples (low $p_\\theta(r|z)$) and smoother ones for confident samples (high $p_\\theta(r|z)$). This represents a fundamentally different principle: dynamically shaping learning objectives rather than merely reweighting losses. If the reviewer has specific alternative methods in mind, we would be happy to discuss comparisons.\n\n> ### Regarding representation analysis (Q4)\n\nOur paper provides extensive representation analysis across multiple sections. In Section 4.5, KNN classification (Figure 3a) demonstrates VarCon's superior class boundary separation through higher accuracy. Appendix B.3's t-SNE visualization (Figure 4) reveals clearer semantic clusters with reduced overlap compared to SupCon, while Appendix B.4's hierarchical clustering analysis (Table 6) quantifies this improved organization through enhanced ARI/NMI/purity metrics. These complementary analyses comprehensively validate VarCon's superior representation quality."}]

Author rebuttal:

Meta-review: VarCon presents a theoretically grounded and empirically validated framework that reformulates supervised contrastive learning as variational inference over latent class variables. The method introduces a ELBO-based objective and a confidence-adaptive temperature mechanism, yielding consistent improvements over SupCon across classification, few-shot learning, and robustness benchmarks. Reviewers appreciated the formulation, strong empirical results, and comprehensive rebuttal, with multiple scores raised during discussion. While some concerns were raised about novelty and presentation, the authors convincingly clarified distinctions from prior work and demonstrated the method’s scalability, efficiency, and representation quality. This is a technical sound and potentailly impactful contribution to contrastive learning and representation learning. I therefore recommend acceptance with the expectation that the additional results and clarifications that emerged during the discussion are integrated into the final version.
Decision: Accept (poster)