# Combining Contrastive Learning with Auto-Encoder for Out-of-Distribution Detection
## 2 Related Work
### Auto-Encoders  
Auto-encoders have made significant strides in machine learning in recent years and have played a crucial role in OOD detection. An auto-encoder is a neural network designed to map input data to a low-dimensional representation and then map it back to the original data [47; 48; 49]. It consists of an encoder and a decoder and learns how to compress and reconstruct data to extract essential features. Auto-encoders are highly regarded in OOD detection due to their ability to capture critical data features [50; 51; 52]. [PERSON] et al. [53] is the first to employ an auto-encoder to measure the outlyingness of data records. Researchers used to train auto-encoders to reconstruct only normal samples and then use the reconstruction error to measure the degree of abnormality in samples. This approach effectively identifies samples that do not conform to the normal data distribution. However, auto-encoder-based models tend to overestimate the reconstruction error for normal points, whereas they underestimate the error for outliers. [PERSON] and [PERSON] [54] proposed to compute the reconstruction error only with normal points, where they exploited the novel clustering approach to filter the probable outliers based on density and distance to the higher-density points. Adversarial auto-encoders [55] introduce adversarial networks to improve the quality of generated data. These models excel in generating OOD data and hold potential in modeling model uncertainty [56; 57; 58]. [PERSON] and [PERSON] [59] designed a VAE to identify data that does not conform to the patterns learned during the VAE's training on in-distribution data. Auto-encoders are also widely used in transfer learning by transferring auto-encoders trained in a known domain to an unknown domain to improve OOD detection performance [60; 61; 62; 63; 64]. However, auto-encoders might struggle to capture the diverse and novel patterns present in OOD samples, especially if these samples exhibit characteristics not well-represented in the training data. Further, auto-encoders are primarily designed for data reconstruction and may not inherently capture the dis criminative features useful for distinguishing between in-distribution and OOD samples. [PERSON] et al. [65] argued that the performance of OOD detection suffers from overfitting and instability during training.  
Addressing the above challenges may involve exploring more sophisticated auto-encoder architectures, incorporating additional supervision or labeled OOD data, experimenting with different loss functions, and carefully optimizing hyperparameters. Ensemble methods or combining auto-encoders with other OOD detection techniques may also be explored to enhance the overall performance.  
Our approach strategically combines contrastive learning with auto-encoders for effective OOD detection. The synergy between these two components is instrumental in enhancing the overall model performance. Contrastive learning plays a pivotal role in reinforcing the backbone's representation learning capabilities. By encouraging in-distribution samples to be pulled closer together and pushing OOD samples farther apart during training, contrastive learning contributes to a more discriminative feature space. This learned feature space becomes adept at capturing the nuances of the data distribution, thus improving the model's ability to distinguish between in-distribution and OOD samples. On the other hand, the auto-encoder component contributes to the refinement of the feature space. It achieves this by minimizing the reconstruction error, effectively encouraging the preservation of crucial information during the encoding and decoding process. The auto-encoder serves as a regularization mechanism, ensuring that the learned features are not only discriminative but also conducive to accurate reconstruction. Aside from that, the combination of the two methods improves the model's capability to capture both global and local features for OOD detection.