# Fine-Tuned Visual Transformer Masked Autoencoder Applied for Anomaly Detection in Satellite Images
## 1 Introduction
### Contribution  
Our contribution addresses the first application and evaluation of the ViTMAE model in a manner similar to that used in Model Genesis [26]. The alignment of the model within the framework is presented in Figure 1. Furthermore, the ViTMAE model was fine-tuned on a custom dataset following pre-training on a well-known dataset. This approach tests the possibility of using well-recognized state-of-the-art architectures within the framework used for the anomaly detection task on a custom dataset and in a new domain.  
Considering the generalization, we come to the origin of applying a deep learning classifier to the anomaly detection problem. Here, first of all, a complex task of image analysis and classification is undertaken, which requires a classifier capable of extracting features. In addition, deep learning methods are well recognized for their ability to construct self-representations of inner characteristics when it comes to image analysis performed in an autoencoder manner. However, deep learning methods can be characterized as valuable in terms of application to complex and challenging tasks such as classification, object detection, and segmentation of images; there is always a trade-off between costs and benefits. The extensive and broad architecture of deep learning models in terms of the data structures needed to store and transfer internal data representations also requires computational resources and often specialized hardware. Furthermore, almost all deep  
Figure 1: Proposed framework, consisting of training and validation pipelines.  
learning models tend to have a structure so complex and broad that it cannot be fully analyzed or controlled by a single individual. This black-box feature of many machine learning methods has led to the development of techniques and concepts that enhance interpretability, explainability, and traceability, thereby improving our understanding of deep learning models. Such methods include, for example, activation maps, heatmaps, AI testing, and coverage.  
The proposed system was designed, implemented, tested, and evaluated according to the framework shown in Figure 1. The source of images was the mapbox public database. The images were then pre-processed and split into normal and anomaly data, which were used for the ViTMAE fine-tuning phase. After the first model was fine-tuned, anomaly scores were calculated, providing input data for the second classification. For this step, an automated machine learning task was performed. After choosing the final classifier, we evaluated the performance metrics for the whole model.  
The alignment of the model's components differs during the training and testing phases. Firstly, the model is trained with the goal of reconstructing partially masked images with the highest possible quality. Using the data preprocessing steps, training and validation sets are built. After training the model on normal samples only, reconstruction error anomaly scores are calculated for both normal and abnormal samples. The training phase is carried out using only the normal dataset. The idea behind this concept is to enable the model to learn an internal representation of the features of normal images only. We assume that the reconstruction error will be lower for well-known (from the perspective of the model) samples, while it is expected to be higher for anomalous samples. The whole process is organized as an automated machine learning task to extract performance indicators for later assessment and analysis.