# Image-Level Anti-Personnel Landmine Detection Using Deep Learning in Long-Wave Infrared Images
## 3 Methods
### Transfer Learning
Specifically, the standard convolutions are replaced by depthwise separable convolutions having a large kernel size (\(7\times 7\)) to improve computational efficiency and performance. Inverted bottleneck blocks, a design borrowed from MobileNetV2 [43] are used to enhance feature transformation. Various modifications to micro-design such as replacing ReLU with GeLU and substituting batch normalization with layer normalization are also introduced. Simulation results of ConvNeXt pre-trained on a large dataset show that the modernized CNN models are not inferior compared to ViT models [38]. The third pre-trained model is ViT [39] trained on the ImageNet21K [25] dataset. Unlike CNN models that rely on localized receptive fields and hierarchical feature extraction, ViT treats an image as a sequence of fixed-size non-overlapping patches and processes them using a transformer encoder. In ViT, the non-overlapping patches is linearly embedded into a fixed-dimensional vector. These patch embeddings are combined with positional encodings to retain spatial information and then passed through multiple transformer layers consisting of multi-head self-attention and feedforward sub-layers. One of the key advantages of ViT is its ability to model long-range dependencies across the entire image using self-attention mechanisms, enabling more global feature representations [44]. However, its performance heavily depends on large-scale pre-training, as it lacks the inductive biases inherent in CNN models [39]. OpenCLIP [40] designed to facilitate scalable multi-modal (text-image pair) learning using publicly available datasets such as LAION-400M and LAION-2B [45] is also inspected for our task of interest. OpenCLIP adopts a dual-encoder architecture consisting of a vision encoder and a transformer-based text encoder that jointly learns via contrastive loss to align image and text representations in a shared embedding space. OpenCLIP supports large-batch contrastive learning, mixed-precision learning, and an efficient distributed \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Model** & **Backbone** & **\# of Param.(M)** & **Dataset** & **Sup./Self.** & **Modality** \\ \hline ResNet [16] & ResNet50 & 23.5 & IN1K & sup. & image \\ \hline ConvNeXt [38] & ConvNext-Base & 87.6 & IN21K & sup. & image \\ \hline ViT [39] & ViT-Base & 102.6 & IN21K & sup. & image \\ \hline OpenCLIP [40] & ViT-Base & 86.6 & LAION400M & self. & image–text pair \\ \hline InfMAE [41] & ConvViT-Base & 89.1 & Inf30 & self. & infrared image \\ \hline \hline \end{tabular}
\end{table} 
Table 1: A summary of models employed in this work and some key features of the dataset used to perform pre-training. learning framework that allows OpenCLIP to scale to billions of image-text pairs [45]. The pre-trained model demonstrates strong zero-shot and few-shot performance across a range of vision tasks, including image classification, retrieval, and transfer learning. This makes OpenCLIP a versatile foundation model for vision-language applications. In this work, only the pre-trained vision encoder of OpenCLIP is utilized. The final model employed in our study is InfMAE [41], a foundation model specifically designed for the infrared image modality. Inspired by the well-known self-supervised framework, Masked Autoencoder (MAE) [33], InfMAE introduces several key modifications tailored to the characteristics of infrared imagery. Notably, the original random masking strategy in MAE is replaced with an information-aware masking mechanism, which selectively masks less informative regions to better focus the learning process on semantically rich content. Additionally, the transformer-only encoder architecture of MAE is modified in InfMAE to include two convolutional stages followed by a transformer stage (ConvViT), enabling multi-scale feature extraction. A further significant contribution is the release of Inf30, a newly curated large-scale unannotated infrared dataset comprising 305,241 images that enables the pre-training of the model.