# Optimizing Mobile Vision Transformers for Land Cover Classification
## 4 Results  
For the training and evaluation stage of the completed model architectures, a group of four experiments were devised, with each experiment corresponding to one of the four datasets that we planned to train and evaluate the models on. This was followed by the application of each representative architecture, which included the two benchmarks ViT and MViT along with the three MViT variants, to all of the datasets. Multiple train-test splits for each model architecture were also introduced to ascertain if the models could achieve a high level of performance with lesser amounts of training data. The train-test splits used were 20-80%, 40-60%, 50-50%, and 60-40%. Hence, each experimental group corresponding to one of the four datasets had 28 models used for training, bringing the number of trained models to 112. All experiments were conducted at the Blugold Center for High-Performance Computing using an NVIDIA Tesla V100 GPU (Nvidia, Santa Clara, CA, USA) with 32 GB memory and an AMD EPYC CPU at 2.35 GHz. Data augmentation for the training images started with resizing all of the images to \(72\times 72\) followed by randomly flipping those images along a horizontal axis. The images were randomly rotated before being randomly zoomed in on during the training process. The hyperparameters were configured to control the training process. The number of epochs was kept constant at 500, and the batch size was kept constant at 64. Also, all of the models used an Adam optimizer [45] to help change the weights and loss rates. For this optimizer, the learning rate was set to 0.001, which allowed for slow and precise learning. The weight decay was set to 0.0001 to regularize the neural network by adding a penalty to the loss function. The only hyperparameter that we varied across the models was the number of classes or labels because each dataset has a different number of class labels. This hyperparameter played a role in influencing the total number of training parameters because it modified the size of the output dense layer in the ViT models. The training accuracies for all model variants are shown in Table 1 in Appendix A. The testing accuracy serves as a vital metric for evaluating the model's performance on unseen data by providing insights into its generalization capabilities. A high testing accuracy indicates that the model has successfully learned meaningful patterns from the training data and can effectively make predictions on new, unseen samples. Moreover, monitoring testing accuracy helps with detecting overfitting, wherein the model performs well on the training data but fails to generalize to new data. By continuously assessing and improving testing accuracy throughout the training process, developers can fine-tune their models, enhance their predictive capabilities, and build robust solutions that perform reliably in real-world scenarios. The testing accuracy results can be seen in Table 1. The mean testing accuracy of all MViT-Combined models is shown in Figure 6. These models were able to converge to an optimal value at the end of 500 epochs, showcasing their robustness. Throughout the research setup, we monitored the testing accuracy from the splits and observed that all the proposed MViT variants seemed to outperform traditional versions. For the AID experiment group, the MViT variants outperformed the ViT model across all of the splits. A consistent trend was noted among all architectures, where the 60-40 split was typically the highest-performing model compared to other splits, with the MViT-Combined model achieving a test accuracy of 89.42%, which outperformed ViT's best validation accuracy by 17.92% and MViT's best test accuracy by 2.84%. The MViT-Combined model, with only 1,016,846 parameters, extended this performance, achieving the highest accuracy among MViT's variants while using significantly fewer parameters than the ViT model, which stands at 21,687,269 parameters.