Probabilistic Contrastive Learning with Explicit Concentration on the Hypersphere

Authors: Hongwei Bran Li, Cheng Ouyang, Tamaz Amiranashvili, Matthew Rosen, bjoern menze, Juan Eugenio Iglesias
Primary area: unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords: representation learning, von Mises–Fisher distribution, uncertainty, probabilistic contrastive learning

Abstract:
Contrastive learning is predominantly deterministic, limiting its effectiveness in noisy and uncertain environments. We propose a probabilistic approach inspired by the von Mises-Fisher (vMF) distribution, embedding representations on a hyperspherical space. To address numerical instability, we introduce an unnormalized and regularized vMF distribution, preserving essential properties with theoretical guarantees. 
The concentration parameter, $\kappa$, serves as an interpretable measure of aleatoric uncertainty.
Empirical evaluations show a strong correlation between estimated $\kappa$ and unseen data corruption severity, enabling effective failure analysis and enhancing out-of-distribution detection without modeling epistemic uncertainty. From a fresh perspective, our approach introduces a flexible alignment mechanism for improved uncertainty estimation in high-dimensional spaces while remaining compatible with existing contrastive learning frameworks.

Decision: reject

Reviews and discussion:

Metareview: