## 2 Related Work
### Vision Transformer-Based Approaches
The model is a 3D UNET, with the ViT structure being employed as an encoder. Then, they used a skip-connection structure to connect the input pictures to the decoder after converting them into sequence representations. They noted that self-supervised learning is crucial to the study of medical OOD identification, as certain anomalous images of uncommon diseases are extremely challenging to produce. In the construction area, [PERSON] et al. [PERSON] et al. (2017) rebuilt unlabeled pavement pictures using the ViT-S self-supervised learning model. They suggested a pavement anomaly detection technique based on encoding, retrieval, and matching to deal with the classification retraining issue. The authors of [PERSON] et al. (2018) introduced a unique model for anomaly detection called \"ViV-Ano\" that is based on an encoder-decoder design, where the encoder comprises a variational autoencoder (VAE) and ViT to extract both local and global characteristics from the images for anomaly identification in industrial processes. Work for low-resolution image-based anomaly detection and localization in industrial fields was proposed by [PERSON] et al. [PERSON] et al. (2018). The work investigated the use of ViT in the classification and localization of surface defects in leather. Furthermore, [PERSON] et al. [PERSON] et al. (2018) presented an approach to identifying logical flaws under sophisticated semantic conditions in the industry area, particularly for defect inspection industrial tasks involving printed circuit boards (PCBs). The provided method used a pre-trained network for multi-scale prior embeddings, followed by a vision transformer with dual attention mechanisms for global-local two-level reconstruction. A masked multi-head self-attention method was used by [PERSON] et al. [PERSON] et al. (2018) to enable the model to understand the link between various input picture patches for anomaly detection in the industrial quality control field. To accomplish high-precision picture anomaly detection, they added a new masking component to the ViT architecture and adjusted the attention between patches of varying forms. A hybrid transformer model called ViTALnet was introduced by [PERSON] et al. [PERSON] et al. (2018). It was constructed based on fine-grained feature reconstruction. To leverage the global semantic capturing capacity, ViTALnet used the vision transformer (ViT) to derive the local discriminating characteristics as feature representation. Then, a pyramidal design and global attention mechanisms were combined to create an anomaly estimation module that improves contextual data for the localization of fine-grained industrial anomalies. By using a ViT to encode the input image that has been split into fragments, VT-ADL [PERSON] et al. (2018) was able to obtain the features that represent the normal image and perform the image abnormality identification task for industrial textured surfaces from various fine-grained levels in a reconstruction-based manner. In addition, a Gaussian mixture density function was also introduced for pixel-level picture anomaly localization. Hence, the distribution of ViT-encoded features is modeled using the Gaussian mixture density function to model the distribution of normal data in the potential space.