# Combining Contrastive Learning with Auto-Encoder for Out-of-Distribution Detection
## 2 Related Work
### Contrastive Learning  
One of the fundamental concepts in contrastive learning is the idea of learning representations by contrasting positive pairs (similar samples) and negative pairs (dissimilar samples) [39; 40; 41]. Early work in this area often utilized siamese networks and triplet loss functions to optimize these embeddings. However, the approach had scalability issues with large datasets and required the careful selection of triplets. The introduction of InfoNCE loss by the momentum contrast (MoCo) framework by [PERSON] et al. [39] marked a significant advancement. It simplified the training process by using a large memory bank to store representations, making it efficient for large-scale datasets like ImageNet. This method also contributed to the success of SSL. Up to now, contrastive learning has gained significant attention in the field of OOD detection. SimCLR, by [PERSON] et al. [42; 43], further refined the contrastive learning paradigm. It emphasized data augmentation and introduced an augmented view of the same image, leading to SOTA results in various benchmarks. The framework also highlighted the importance of batch normalization in improving performance. In the context of OOD detection, contrastive learning has demonstrated its versatility. Researchers have explored using contrastive representations as the basis for more accurate and efficient OOD detection techniques [44]. Moreover, multi-scale representations have recently gained traction in contrastive learning. Models like MoCo and SwAV [45] explicitly consider multiple scales or views of data, enhancing the robustness of learned representations. Contrastive learning has not only significantly improved self-supervised learning but has also become a powerful tool for OOD detection and feature representation. However, contrastive learning may overfit the features in in-distribution data, as the majority of the training samples usually belong to the normal class. This can result in higher similarity scores for OOD samples that deviate significantly from the learned in-distribution patterns. [PERSON] et al. [46] found that existing models trained by off-the-shelf cross-entropy or contrastive losses are incompetent to capture valuable local representations for multi-scale OOD detection. Therefore, they proposed to leverage both the global visual information and local region details of images to benefit OOD detection. In addition, the performance of contrastive learning models for OOD detection may vary across different datasets and domains. Models trained on one dataset may not generalize well to unseen datasets with different characteristics. Often, addressing these challenges involves a combination of strategies, such as incorporating labeled OOD data when available, utilizing more advanced contrastive learning architectures, introducing regularization techniques, and carefully selecting and fine-tuning model hyperparameters. Additionally, considering ensemble methods or combining contrastive learning with other OOD detection approaches may enhance the overall performance.