# Optimizing Mobile Vision Transformers for Land Cover Classification
## 2 Previous Work
These changes increased the training characteristics on multi-spectral data. A prior work that attempted to modify MViT models to boost performance was also reported in [35]; the authors replaced a \(3\times 3\) convolution layer in the fusion block with a \(1\times 1\) convolution layer and replaced a \(3\times 3\) convolution layer in the global representation block with a depthwise convolution layer. Also, the model fuses the input features, combines local and global features, and increases the number of channels of the layers. Moreover, the model, which is called MViTv3, can outperform MViT variants such as MViTv1-XS and MViTv2-0.75 while maintaining a similar but slightly higher number of parameters. Another attempt to modify MViT for better performance and lower latency is demonstrated in [36]; the authors replaced the MHSA in the MViTv1_Block's transformer block with a separable self-attention method and did not use MViTv1_Block's skip connection and \begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline
**Paper** & **Datasets** & **Model** & \
\begin{tabular}{c} **Parameter** \\ **Size** \\ \end{tabular} & **Paper** & **Datasets** & **Model** &
\begin{tabular}{c} **Parameter** \\ **Size** \\ \end{tabular} \\ \hline \multirow{6}{*}{[34]} & \multirow{6}{*}{AID, UC-Merced, EuroSAT and WHURIS19} & MobileNetV2 & 2.5 m & & & & & & \\ \cline{3-8}  & & DenseNet201 & 18.7 m & & & & & & \\ \cline{3-8}  & & Xception & 21.8 m & & & & & & \\ \cline{3-8}  & & & InceptionResNetV2 & 54.8 m & & & & & \\ \cline{3-8}  & & ResNet152V2 & 58.8 m & & & & & & \\ \cline{3-8}  & & NASNetLarge & 87.3 m & & & & & & \\ \hline \multirow{6}{*}{[29]} & \multirow{6}{*}{AID, UC-Merced} & ShuffleNet v2 & 1.3 m & & & & & & \\ \cline{3-8}  & & SqueezeNet & 1.3 m & & & & & & \\ \cline{3-8}  & & MobileNet v2 & 2.3 m & & & & & \\ \cline{3-8}  & & DenseNet-121 & 7 m & & & & & \\ \cline{3-8}  & & ResNet-50 & 25.6 m & & & & & \\ \cline{3-8}  & & VGG-16 & 102 m & & & & & \\ \hline \multirow{6}{*}{[33]} & \multirow{6}{*}{UC-Merced, WHU-RS19,} & VGG16 Healthy Fine-tuned & 9.47 m & & & & & \\ \cline{3-8}  & & VGG16 Healthy Fine-tuned & 14.7 m & & & & & \\ \cline{3-8}  & & Fine-tuned & & & & & & \\ \cline{3-8}  & & ResNet152V2 Lightly Fine-tuned & 53.97 m & & & & & \\ \cline{3-8}  & & ResNet152V2 Heavily Fine-tuned & 58.32 m & [12] & & & & \\ \hline \multirow{6}{*}{[31]} & \multirow{6}{*}{UC-Merced, AID} & AMB-CNN & 5.6 m & & & & & \\ \cline{3-8}  & & HABFNet & 6.2 m & & & & & \\ \cline{3-8}  & & LCNN-BFF method & 6.2 m & & & & & \\ \cline{3-8}  & & Inceptionv3 + CapsNet & 22 m & & & & & \\ \cline{3-8}  & & ResNet+WSPM-CRC & 23 m & & & & & \\ \cline{3-8}  & & Proposed method & 28 m & & & & & \\ \hline \multirow{6}{*}{[26]} & EuroSAT, AID, WHU-RS19} & ResNet-50 & 38 m & & & & & & \\ \cline{3-8}  & & Vit-Base-32 & 87 m & & & & & \\ \cline{3-8}  & & Vit-Large-14 & 304 m & & & & & \\ \cline{3-8}  & & Slim Network & 2.5 m & & & & & \\ \cline{3-8}  & & Slim Network & 2.5 m & & & & & \\ \cline{3-8}  & & Slim Network & 2.5 m & & & & & \\ \cline{3-8}  & & SlugezeNet & 4.8 m & & & & & \\ \cline{3-8}  & & Plump Network & 9 m & & & & & \\ \cline{3-8}  & & AlexNet & 60 m & & & & & \\ \cline{3-8}  & & VGG-VD-16 & 140 m & [28] & EuroSAT, UC-Merced, AID & & & & \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Notable work in geospatial image classification to reduce model complexity. Parameter size is in millions. fusion block. As a result, MViTv2 maintains a similar or smaller number of parameters and outperforms the MViTv1 model by about 0.9% on the ImageNet dataset. While these works can modify MViTs in a way that minimizes costs and boosts performance, the number of parameters used is still too high at around more than 1.25 million (MViTv3-XXS), and the increase in accuracy (0.9% by MViTv2) from the original MViT architecture is negligible. This is a drawback to implementing plug-and-play AI models in the real world with resource-constrained environments.