# Panchromatic and Hyperspectral Image Fusion Using Ratio Residual Attention Networks
## 3 Methods
### Ratio Transformation Method  
The hyperspectral image and panchromatic image captured simultaneously exhibit a strong correlation, which forms the foundational basis for the ratio transformation fusion method [26]. The ratio transformation fusion method is defined as the ratio of the high-resolution panchromatic image to the degraded panchromatic image being equal to the ratio of the fused image to the hyperspectral upsampled image. The formula for the ratio transformation is as follows:  
\[\frac{P(i,j)}{D(i,j)}=\frac{F_{k}(i,j)}{H_{k}(i,j)},\quad k=1,2,\ldots,n, \tag{1}\]  
where \((i,j)\) represents the pixel coordinates in the corresponding images, \(P\) represents the panchromatic image, \(D\) represents the degraded panchromatic image, \(F\) represents the fused image, \(H\) is the hyperspectral upsampled image, \(k\) represents the spectral band index of the hyperspectral image, and \(n\) represents the number of spectral bands in the hyperspectral image. The physical meaning of the ratio transformation is as follows: when the panchromatic image is divided by the degraded panchromatic image, the grayscale information in the panchromatic image is canceled out, and the ratio image retains the high-frequency information, i.e., the spatial details of the ground objects. The hyperspectral image provides rich spectral information, and by multiplying the ratio image with the low-resolution hyperspectral image, the spatial details of the ground objects are injected into the hyperspectral image, resulting in a high-resolution fused image. The fusion process of the ratio transformation method is expressed by the following formula:  
\[F_{k}(i,j)=\frac{P(i,j)}{D(i,j)}\times H_{k}(i,j),\quad k=1,2,\ldots,n. \tag{2}\]  
Since the ratio transformation fusion method treats the process of dividing the panchromatic image by the degraded panchromatic image as the removal of grayscale information from the image, the key challenge of this method lies in obtaining a \"reasonable\" degraded panchromatic image [27]. Generally, a \"reasonable\" degraded panchromatic image should meet the following two criteria: (1) the difference in the regional mean at the corresponding pixel points must be sufficiently small compared to the panchromatic image; (2) the loss of spatial details in the degraded panchromatic image, relative to the panchromatic image, should be comparable to the loss of spatial details in the hyperspectral upsampled image, relative to the fused image. Therefore, understanding how to reasonably construct the degraded panchromatic image is critical to the ratio transformation fusion method.  
The proposed method uses mean filtering to construct a low-resolution panchromatic image as the initial degraded image, as shown in the following formula:  
\[L(i,j)=(P*M)(i,j). \tag{3}\]  
In this equation, \(M\) represents the mean filter, and \(L\) denotes the initial degraded panchromatic image. By dividing the panchromatic image by the initial degraded image, the initial ratio image is obtained, as expressed by the following formula:  
\[R_{e}(i,j)=\frac{P(i,j)}{L(i,j)}. \tag{4}\]Subsequently, the initial ratio image \(R_{e}\) is input into the deep residual attention network for fine-tuning, resulting in a multi-band ratio image \(R\). The ratio image \(R\) obtained through fine-tuning with the residual attention network has the same dimensions as the hyperspectral image. This process can be expressed by the following formula:  
\[R(i,j)=f(R_{e}(i,j);\theta), \tag{5}\]  
where \(f(\cdot)\) represents the residual attention network, the architecture of which will be detailed in the next subsection. The symbol \(\theta\) denotes the learnable parameters of the network. After training the network, the new ratio image \(R\) is obtained. We then inject this new ratio image \(R\) into the hyperspectral upsampled image, resulting in a high spatial resolution fused image, which can be expressed by the following formula:  
\[F_{k}(i,j)=R_{k}(i,j)\times H_{k}(i,j),\quad k=1,2,\dots,n. \tag{6}\]