# Adaptive Frame Sampling and Feature Alignment for Multi-Frame Infrared Small Target Detection
###### Abstract  
In recent years, infrared images have attracted widespread attention, due to their extensive application in low-visibility search and rescue, forest fire monitoring, ground target monitoring, and other fields. Infrared small target detection technology plays a vital role in these applications. Although there has been significant research over the years, accurately detecting infrared small targets in complex backgrounds remains a significant challenge. Multi-frame detection methods can significantly improve detection performance in these cases. However, current multi-frame methods face difficulties in balancing the number of input frames and detection speed, and cannot effectively handle the background motion caused by movement of the infrared camera. To address these issues, we propose an adaptive frame sampling method and a detection network aligned at the feature level. Our adaptive frame sampling method uses mutual information to measure motion changes between adjacent frames, construct a motion distribution, and sample frames with uniform motion based on the averaged motion distribution. Our detection network handles background motion by predicting a homography flow matrix that aligns features at the feature level. Extensive evaluation of all components showed that the proposed method can more effectively perform multi-frame infrared small target detection. infrared small target detection; multi-frame; adaptive frame sampling; feature alignment Article Adaptive Frame Sampling and Feature Multi-Frame Infrared Small Target Detection  
16 June 202412 July 202412  
information helps build more robust infrared small target detection models [11]. This paper primarily discusses the difficulties and challenges faced by multi-frame-based infrared small target detection methods. Multi-frame-based methods usually take the current frame \(F_{t}\) and the previous \(k\) consecutive frames, i.e., \([F_{t-k},\ldots,F_{t-1},F_{t}]\), as input. While the detection accuracy of the model is proportional to the number of input frames within a certain range, increasing the number of input frames significantly increases the computational workload, resulting in longer detection times. This poses challenges in scenarios with strict real-time detection requirements and limited computing resources, such as high-speed airborne platforms, unmanned aerial vehicle (UAV) systems, and handheld embedded devices. These situations present significant constraints and limitations in terms of the computational efficiency of infrared small target detection. As shown in Figure 1, the information contained between adjacent frames is often redundant. Enhancing detection performance and computational efficiency without increasing the computational burden can be achieved by extracting \(N\) frames from a sequence of \(T\) consecutive frames. This approach allows for the incorporation of more motion and scene details, leading to improved detection performance, while maintaining a favorable computational cost ratio. Most of the currently used methods sample frames from consecutive dense frames at random or fixed intervals. However, this fixed strategy sampling method has the following issues. First, the motion patterns of different dense frame sets vary, and the targets do not move at a uniform speed. A fixed strategy sampling method may lose important information in some sets. Second, this method may sample redundant frames in other sets. Another issue with multi-frame infrared small target detection is the interference caused by background displacement due to the motion of the infrared detection sensor. Recently, this problem has been addressed by pre-processing the input frames to align them with the perspective of the detection frame \(F_{t}\)[12; 13; 14]. However, this approach heavily relies on the quality of image registration preprocessing. Improper registration can introduce significant interference into the detection process, leading to poor robustness.