# Convolutional Sparse Modular Fusion Algorithm for Non-Rigid Registration of Visible-Infrared Images
## 6 Conclusions  
This study presents a comprehensive framework for convolutional sparse modular fusion of non-rigidly registered visible-infrared images. Building upon the attention-based convolutional sparse encoder and the feature filtering network, we have introduced feature-guided loss and information entropy loss to enhance the efficiency of feature encoding and extraction. Through the development of self-supervised registration loss and fully supervised prediction loss, we achieved accurate image registration and effective image fusion.  
The proposed algorithm was tested across multiple datasets to validate its generality and robustness. Visualization experiments of feature filtering and extraction demonstrated that the feature extraction module can extract homologous feature images with a consistency correlation coefficient of 0.961 and isolated feature images with a coefficient of 0.137. In pre-registered image fusion experiments and practical image fusion experiments, the fusion images generated by the proposed algorithm achieved the highest MI scores of 3.51 and 3.11, respectively, outperforming mainstream alternative algorithms by up to 30.6% and 8.3% in this metric. In downstream application experiments, the fusion images generated by the proposed algorithm improved the target detection accuracy of YOLOv8 by a maximum of 20.1% compared to the original images and outperformed mainstream alternative algorithms by 1%. In performance comparison experiments, the proposed algorithm demonstrated the best overall fusion performance and computational efficiency among mainstream alternative algorithms and could process 1800 \(\times\) 1600 images at a speed of 10 FPS when deployed on Raspberry Pi 5.  
Although both the image fusion and image registration tasks of the proposed algorithm rely on feature extraction, exhibiting a certain degree of coupling, this coupling is not entirely interdependent. Instead, it is achieved through a shared feature extraction module that facilitates partial information exchange. However, this also results in a weaker overall anti-interference capability of the algorithm. Future research should focus on developing multimodal image fusion models with strong anti-interference capabilities.  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline \multirow{2}{*}{**Content**} & **MI** & **RMSE** & **GFLOPs** & \(\boldsymbol{N_{p}\big{(}10^{6}\big{)}}\) \\ \cline{2-5}  & **+** & **–** & **–** & **–** \\ \hline Change Dictionary & 2.17 & 58.77 & 3.286 & 406.6 \\ Change FFN & 2.27 & 57.68 & 5.725 & 461.5 \\ Change IFM & 2.59 & 53.63 & 2.647 & 362.9 \\ Change input of RM & 2.29 & 59.28 & 3.553 & 407.7 \\ Change \(L_{FG}\) & 2.19 & 54.36 & 3.527 & 416.7 \\ Remove \(L_{EN}\) & 2.28 & 56.33 & 3.527 & 416.7 \\ Remove \(L_{SR}\) & 2.75 & 60.85 & 3.527 & 416.7 \\ ICSF & 3.41 & 50.57 & 3.527 & 416.7 \\ \hline \hline \end{tabular}
\end{table}
Table 5: Ablation study results in the RoadScene dataset. Here, a “\(+\)” indicates that larger metric values are better, while a “\(-\)” denotes the opposite.  
**Author Contributions:** Software, T.L.; formal analysis, H.Y.; data curation, X.Z.; writing--original draft preparation, T.L.; writing--review and editing, N.C.; visualization, W.D.; project administration, N.C. All authors have read and agreed to the published version of the manuscript.  
**Funding:** This research received no funding.  
**Institutional Review Board Statement:** Not applicable.  
**Informed Consent Statement:** Not applicable.  
**Data Availability Statement:** The original data presented in the study are openly available in [[https://github.com/logic1734/ICSF](https://github.com/logic1734/ICSF)]([https://github.com/logic1734/ICSF](https://github.com/logic1734/ICSF)), accessed on 13 February 2025.  
**Conflicts of Interest:** The authors declare that they have no conflicts of interest.