Before revision: Contrastive approaches to self-supervised learning (SSL) learn representations by minimizing the distance between two augmented views of the same data point (positive pairs) and maximizing the same from different data points (negative pairs) . However, recent approaches like  BYOL and SimSiam , show remarkable performance {\it without} negative pairs, raising a fundamental theoretical question : how can SSL with only positive pairs avoid representational collapse ?
After revision: Contrastive approaches to self-supervised learning (SSL) learn representations by minimizing the distance between two augmented views of the same data point (positive pairs) and maximizing the same from different data points (negative pairs) , recent BYOL and SimSiam , show remarkable performance {\it without} negative pairs, raising a fundamental theoretical question : how can SSL with only positive pairs avoid representational collapse ?