# Integrated Fusion Network for Hyperspectral, Multispectral and Panchromatic Data Fusion
## 1 Introduction  
The emergence and application of Hyperspectral Remote Sensing dates back to over 20 years ago. It was first developed on the basis of Imaging Spectroscopy. Compared with traditional remote sensing, high spectral resolution imaging spectrometers provide a narrow (usually less than 10 nm) imaging band for each imaging pixel, continuously distributed in a certain spectral interval. Nevertheless, the constraints imposed by the physical characteristics of optical sensor design necessitate a compromise between spatial resolution and spectral resolution in sensor imaging. There are two types of remote sensing images: multispectral remote sensing images (MSI) and panchromatic remote sensing images (PAN) have higher spatial resolution but lower spectral resolution, while hyperspectral remote sensing images (HSI) have higher spectral resolution but lower spatial resolution. As a result, HSI sharpening has garnered significant interest. HSI sharpening typically entails integrating HSI with PAN (or MSI) data acquired simultaneously and in the same area to produce high-resolution hyperspectral remote sensing images. Currently,this technology is extensively utilized in several domains including geography, agriculture, land surveying, and environmental monitoring [1,2]. Over the past ten years, many researchers have developed different techniques for enhancing HSI. These algorithms can be broadly categorized into four groups: component substitution (CS) [3], multi-resolution analysis (MRA) [4], variant optimization (VO) [5], and deep learning (DL) methods [6]. Researchers are currently primarily focusing on HSI fusion approaches based on convolutional neural networks (CNN) due to its advantages [7,8]. Currently, the primary focus of CNN-based HSI sharpening is on the combination of HSI and MSI (FHM) [9,10]. While FHM approaches effectively retain spectral information, the spatial resolution of HSI has not experienced significant enhancement due to the restricted spatial resolution of MSI. Some researchers have combined HSI with PAN (FHP) [11,12] to get a higher spatial resolution. Nevertheless, there exists a notable disparity in the spatial and spectral resolution of HSI and PAN, leading to substantial distortion in the FHP approach. This article introduces a novel CNN network named \"IFNPMH\" that can effectively combine HSI, MSI, and PAN imageries concurrently (FHMP), among them, the PAN image provides clear spatial features for the fused image, the HSI image provides accurate spectral features for the fused image, and the MSI image compensates for the huge gap in spatial and spectral resolution between HSI and PAN, preventing the fused image from having obvious distortion. The primary objective of the IFNMPH network is to provide hyperspectral imagery (HSI) with a high level of spatial resolution, while ensuring precise spectral information and distinct spatial features. First, it utilizes the initial dense module (DM) block to extract spatial characteristics of PAN and MSI, then it employs the second DM block to extract spectral attributes of HSI. Furthermore, a block called Convolutional Block Attention Module (CBAM) has been incorporated into the network following the extraction module in order to enhance the network's capability to extract features and retain spectral information. Ultimately, the extracted feature layer and attention mechanism module are utilized in the reconstruction module to produce a high spatial resolution Hyperspectral Image (HHSI) via the Re-block algorithm. This article utilizes a loss function in the training phase that combines spectral and spatial information loss to facilitate network training. The contribution of this article is summarized as follows:  
(1) A network that integrates FHMP is proposed to address the issue of HSI sharpening. To our knowledge, there is relatively scarce on the fusion of three types of data, namely HSI, MSI, and PAN. (2) IFNMPH is an end-to-end network that can autonomously learn the internal correlations between HSI, MSI, and PAN.