# Artificial Intelligence-Based Deep Fusion Model for Pan-Sharpening of Remote Sensing Images

Authors: Ahmed I. Iskanderani; Ibrahim M. Mehedi; Abdulah Jeza Aljohani; Mohammad Shorfuzzaman; Farzana Akhter; Thangam Palaniswamy; Shaikh Abdul Latif; Abdul Latif; Rahtul Jannat
Journal: Computational Intelligence and Neuroscience
 (2021)

---

## Abstract

During the past two decades, many remote sensing image fusion techniques have been designed to improve the spatial resolution of the low-spatial-resolution multispectral bands. The main objective is fuse the low-resolution multispectral (MS) image and the high-spatial-resolution panchromatic (PAN) image to obtain a fused image having high spatial and spectral information. Recently, many artificial intelligence-based deep learning models have been designed to fuse the remote sensing images. But these models do not consider the inherent image distribution difference between MS and PAN images. Therefore, the obtained fused images may suffer from gradient and color distortion problems. To overcome these problems, in this paper, an efficient artificial intelligence-based deep transfer learning model is proposed. Inception-ResNet-v2 model is improved by using a color-aware perceptual loss (CPL). The obtained fused images are further improved by using gradient channel prior as a postprocessing step. Gradient channel prior is used to preserve the color and gradient information. Extensive experiments are carried out by considering the benchmark datasets. Performance analysis shows that the proposed model can efficiently preserve color and gradient information in the fused remote sensing images than the existing models.


---

## Body

## 1. Introduction

Fusion of multispectral (MS) and panchromatic (PAN) images has attracted researchers’ interest, since it results in a fused image with better spatial resolution and spectral information [1]. The spatial resolution of a MS image is significantly better as compared to a PAN image. But a MS image only has a single band. Thus, to obtain an image with significant spectral information and better spatial resolution, efficient pan-sharpening approaches are required [2].Many pan-sharpening techniques have been implemented so far. The traditional methods suffer from blurring effect and color distortion [1, 3]. Sparse representation theory-based fusion methods can easily overcome the problems of color distortion by enhancing the spatial resolution of MS images [4]. The intensity-hue-saturation (IHS) method was also used to fuse the images. These models were quite simple and efficient and can produce high-spatial-quality images [5, 6]. However, they experience spectral distortion. The spectral fidelity can be enforced using an edge-adaptive IHS method [7]. Compressed sensing (CS) theory is also used for pan-sharpening of multispectral images. It can recover the sparse signal from a small number of linear measurements [8]. Optimized pan-sharpening techniques were also developed to preserve the spectral and geometry constraints [9, 10]. The Bayesian theory-based fusion model solved the problems of linear model and attained superior spatial and spectral fusion [11].Recently, various deep learning models have been used to implement pan-sharpening techniques to produce HR PAN images. These techniques can effectively model complex relationships between variables via the composition of several levels of non-linearity [12]. In the deep pan-sharpening model, the correlation between the LR/HR MS image patches is the same as the LR/HR PAN image patches. Thereafter, this assumption is used to learn the mapping using convolutional neural network (CNN) [13]. Different types of CNN were used to fuse the images. CNN contains three convolutional layers such input, hidden, and output layers. Activation functions are contained in each layer. Input and hidden layers contain non-linear activation layer while output layer comprises linear activation function. For every layer, there are I input bands, J output bands, filters, parameters needed to be learned, tensors, weights, and biases. In case of fusion, PAN bands are given as input to the CNN. The components of MS are upsampled and then radiometric indices are extracted. Lastly, non-linear combinations of MS bands are made to improve the performance [14]. However, most of these methods suffer from inadequate spatial texture improvement and spectral distortion issues. To overcome these issues, many techniques were developed. A dual-path fusion network (DPFN) enhanced spatial texture and spectral distortion [15]. Shallow-deep convolutional network (SDCN) can produce fused images with minimal spectral distortion [16]. Dynamic deep learning models were proposed to build sensitive models towards input images [15]. Coupled multiscale convolutional neural network considered the PAN and MS images at different resolutions for better feature extraction [17]. A four-layer CNN and a loss function were designed which can extract spatial and spectral characteristics efficiently from original image. It did not require any refence fused image and hence did not need simulation data for training [18]. Generative adversarial learning (GAN) was also utilized to implement the fusion of PAN and MS images. It has an ability to produce high-fidelity fused images [19].From the existing literature, it has been found that the deep learning and deep transfer learning models can efficiently fuse the remote sensing images. However, these models do not consider the inherent image distribution difference between MS and PAN images. Therefore, the obtained fused images may suffer from gradient and color distortion problems. To overcome these problems, in this paper, an efficient deep transfer learning model is proposed. The main contributions of this paper are as follows:(1)
An efficient Inception-ResNet-v2 model is improved by using a CPL.(2)
The obtained fused images are further improved by using gradient channel prior as a postprocessing step.(3)
Extensive experiments are carried out by considering the benchmark datasets.This paper is organized as follows. Section2 discusses the literature review. Section 3 presents the proposed model. Comparative analysis is discussed in Section 4. Section 5 concludes the paper.


## 2. Literature Review

Wang et al. [20] proposed a pan-sharpening technique based on the channel-spatial attention model (CSA). In this, residual attention module was designed to produce high-resolution images. Xu et al. [21] implemented the soil prediction model using the pan-sharpened remote sensing indices. In this, images of Landsat 8, GeoEye-1, and WorldView-2 were fused. A prediction model was designed using random forest. Ma et al. [22] used the generative adversarial network to implement the pan-sharpening technique. For network training, it did not require ground truth. Akula et al. [23] implemented a pan-sharpening technique using adaptive principal component analysis and local variation contourlet transform. Wang et al. [24] utilized area-to-point regression kriging (ATPRK) for pan-sharpening. Wang et al. [25] presented a pan-sharpening technique based on compressed sensing. The joint sparsity model was used to recover the high-resolution multispectral images.Wu et al. [26] utilized multiobjective decision for improving the fused multiband images. The information injection model was used to improve texture and gradient details of MS image. Spectral fidelity fusion was designed using injected information and spectral modulation to fused image. Zhuang et al. [27] designed a probabilistic model to fuse MS and PAN images. Gradient domain-guided image filtering was also used to refine the results. The maximum a posteriori model was also implemented on the difference between PAN and MS images and in respective gradient domains too. Sibiya et al. [28] combined image texture obtained from a fused image with partial least squares discriminant analysis to monitor and map commercial forest species. This model proved that the image texture can discriminate commercial forest species.Fang et al. [29] designed a framelet-based fusion model by using a variational model. The split Bregman iteration was also used to obtain better results. The Bregman method solves the convex optimization problems using regularization. It is best suited for those optimization problems where constraints are well specified. Due to error cancellation effect, it converges very fast. Wang et al. [30] proposed sparse tensor neighbor embedding for fusion of PAN and MS images using N-way block pursuit. A sparse tensor was concatenated with neighbor embedding to obtain a new high-dimensional sparse tensor embedding for fusion of PAN and MS images in an efficient manner. Saeedi and Faez [31] utilized shiftable contourlet transform and multiobjective particle swarm optimization (MPSO) to fuse PAN and MS images. PAN and MS images were histogram matched prior to fusion process. Fang et al. [32] designed a pan-sharpening technique using variational approach. In this, three assumptions were made to construct the energy function. The minimized solution was obtained using the Bregman algorithm. Zhang et al. [33] implemented a variational energy function to preserve the spectrum, geometry, and correlation information of the original images while pan-sharpening.Ye et al. [34] proposed gradient-based deep network prior to fuse PAN and MS images. Convolutional neural network (CNN) was trained in gradient domain using the problem-specific recursive block. Xing et al. [35] implemented the pan-sharpening technique using deep metric learning (DML). The deep metric learning was used to train refined geometric multimanifold neighbor embedding. The hierarchical characteristics of masks were used by considering various non-linear deep learning models. Gogineni and Chaturvedi [36] used multiscale learned dictionary (MSLD) to design a pan-sharpening technique. It could obtain the underlying features of images, in which the characteristics of both learned dictionaries and multiscale were possessed. Huang et al. [37] developed a fusion model using multiple deep learning models (MDLMs). The non-subsampled contourlet transform (NSCT) used to decompose the PAN images into frequency bands. The characteristics of high-frequency bands were learned by the deep learning model.From the literature, it is found that the deep learning model should be improved by using a better loss function and some preprocessing techniques [38–40].


## 3. Proposed Model

An efficient artificial intelligence-based deep transfer learning model is proposed. Inception-ResNet-v2 model is improved by using a CPL. The obtained fused images are further improved by using gradient channel prior as a postprocessing step.


### 3.1. Inception-ResNet-v2

Inception-ResNet-v2 is a well-known model which is improved InceptionNet with residual connections. It is achieved by replacing the filter concatenation stage of the InceptionNet (see [41]). Figure 1 shows the architecture of Inception-ResNet-v2.Figure 1
Architecture of Inception-ResNet-v2.


### 3.2. Color-Aware Perceptual Loss

CPL [42] is utilized to assign small coefficients to feature channels which are more sensitive to colors for every L Inception-ResNet-v2 layer during the computation of perceptual loss.The difference among the respective color and a grayscale-inverted MS image (Mδ−1) is utilized to compute coefficients of features. Higher difference indicates that the features are more sensitive to colors. CPL is also more sensitive to gradient information. The average feature difference is then assigned to the exponential function with a variable γ (see [42]). It can be represented as(1)Mδ−1=1−MγR+MγG+MγB3,where MγR, MγG, and MγB represent color channels of MS image. The CPL coefficients for color channels of every layer l, Wcpll, can be computed as(2)Wcpll=e−γ1/N∑nNϕlMδ−1−ϕlMγ,where γ is used to neglect features which are sensitive to colors. For PAN image (Mpc) and a CNN-based fused image (Mps), CPL can be computed as(3)lcpl=∑l=1LWcpll⊙ϕmllMpc−ϕmllMps1,where ml=7,5,3 shows the size of max-pool implemented on lth layer feature which is used to achieve average of shift invariance to CPL. It can efficiently manage the misalignment problem.Although CPL assigns high-frequency information toMps, additional loss is also required for color fidelity. Therefore, the perceptual and l1 losses are used. The fidelity loss can be defined as(4)lf=αcpllcplMpc,Mps+αγlpMγ,Mps↓+αl1l1Mγ,Mps↓,where Mps↓ shows a downscaled PS image to the MS resolution, l1 loss is an average absolute difference, i.e., 1/N∑nNMγ−Mps↓, and αcpl, αγ, and αl1 are set to 0.85, 0.02, and 0.95, respectively.


### 3.3. Gradient Channel Prior

GCP is utilized to restore any kind of degradation from the images. It has an ability to preserve the gradient and texture information of restored images [43]. GCP can be defined as(5)∇Im,n=ψmψn=∂I/∂m∂I/∂n.An amplitude ofI can be computed as(6)magI=ψm2+ψn2.An orientation angle of∇I can be computed as(7)∇Om,n=arctanψnψm.ForIm,n, ψm and ψn can be computed using various masks (see [44]).


## 3.1. Inception-ResNet-v2

Inception-ResNet-v2 is a well-known model which is improved InceptionNet with residual connections. It is achieved by replacing the filter concatenation stage of the InceptionNet (see [41]). Figure 1 shows the architecture of Inception-ResNet-v2.Figure 1
Architecture of Inception-ResNet-v2.


## 3.2. Color-Aware Perceptual Loss

CPL [42] is utilized to assign small coefficients to feature channels which are more sensitive to colors for every L Inception-ResNet-v2 layer during the computation of perceptual loss.The difference among the respective color and a grayscale-inverted MS image (Mδ−1) is utilized to compute coefficients of features. Higher difference indicates that the features are more sensitive to colors. CPL is also more sensitive to gradient information. The average feature difference is then assigned to the exponential function with a variable γ (see [42]). It can be represented as(1)Mδ−1=1−MγR+MγG+MγB3,where MγR, MγG, and MγB represent color channels of MS image. The CPL coefficients for color channels of every layer l, Wcpll, can be computed as(2)Wcpll=e−γ1/N∑nNϕlMδ−1−ϕlMγ,where γ is used to neglect features which are sensitive to colors. For PAN image (Mpc) and a CNN-based fused image (Mps), CPL can be computed as(3)lcpl=∑l=1LWcpll⊙ϕmllMpc−ϕmllMps1,where ml=7,5,3 shows the size of max-pool implemented on lth layer feature which is used to achieve average of shift invariance to CPL. It can efficiently manage the misalignment problem.Although CPL assigns high-frequency information toMps, additional loss is also required for color fidelity. Therefore, the perceptual and l1 losses are used. The fidelity loss can be defined as(4)lf=αcpllcplMpc,Mps+αγlpMγ,Mps↓+αl1l1Mγ,Mps↓,where Mps↓ shows a downscaled PS image to the MS resolution, l1 loss is an average absolute difference, i.e., 1/N∑nNMγ−Mps↓, and αcpl, αγ, and αl1 are set to 0.85, 0.02, and 0.95, respectively.


## 3.3. Gradient Channel Prior

GCP is utilized to restore any kind of degradation from the images. It has an ability to preserve the gradient and texture information of restored images [43]. GCP can be defined as(5)∇Im,n=ψmψn=∂I/∂m∂I/∂n.An amplitude ofI can be computed as(6)magI=ψm2+ψn2.An orientation angle of∇I can be computed as(7)∇Om,n=arctanψnψm.ForIm,n, ψm and ψn can be computed using various masks (see [44]).


## 4. Performance Analysis

The proposed model is trained on using 100 epochs with a mini-batch size of 10 using Adam optimizer [45]. The learning rate is used as 5×10−5. All experiments are performed on MATLAB2021a software. Experiments are performed on Pleiades, QuickBird, IKONOS, and WorldView-2 images. Comparisons are performed with six well-known competitive techniques.


### 4.1. Visual Analysis

Figures2 and 3 show the visual analysis of the proposed model. It is found that the proposed model has better visibility as compared to the existing techniques. Red rectangles show the specific region in the obtained fused images. The selected region reflects the spatial and spectral information along with any kind of artifacts which are present in the obtained fused images. Also, the proposed model shows better gradient and color preservation as compared to the existing techniques. The results obtained from the proposed model show better spatial and spectral information. It clearly shows that the existing models are able to fuse the images by improving the spatial and spectral information of fused images. But whenever there is redundant information in both PAN and MS images, then the existing method fails to fuse the content efficiently. Also, the texture and gradient preservation is significantly more in the fused image obtained using the proposed model.Figure 2
Visual analysis of pan-sharpening techniques. (a) Low-resolution multispectral (MS) image. (b) High-spatial-resolution panchromatic (PAN) image.
(a)(b)(c)(d)(e)(f)(g)(h)(i)Figure 3
Visual analysis of pan-sharpening techniques. (a) Low-resolution multispectral (MS) image. (b) High-spatial-resolution panchromatic (PAN) image.
(a)(b)(c)(d)(e)(f)(g)(h)(i)


### 4.2. Quantitative Analysis

Five well-known quality metrics, i.e., root mean square error (RMSE) [46], universal image quality index (UIQI) [47], correlation coefficient (CC) [46], spectral angle mapper (SAM) [46], and Erreur relative globale adimensionnelle de synthese (ERGAS) [48], are used for comparative analysis.Table1 shows CC analysis of the proposed deep pan-sharpening model. CC is desirable to be maximum. It is found that the proposed model outperforms the competitive pan-sharpening models by 1.7824%.Table 1
Analysis of correlation coefficient (maximum is desirable).
ImagesCSACNNMSLDMDLMDMLMPSOProposedPleiades 10.94820.95850.95250.93560.94950.95310.9612Pleiades 20.93840.95450.95880.96360.95950.95940.9668QuickBird 10.95430.95250.95170.95940.93530.94060.9626QuickBird 20.93880.93870.96270.94350.94510.93880.9659QuickBird 30.94220.94510.93860.94570.94810.94160.9513IKONOS 10.96010.94930.95890.93660.95470.96450.9677IKONOS 20.93910.96170.94420.94520.94050.95780.9649IKONOS 30.94220.95620.93850.95630.96020.95290.9634WorldView-2 10.93990.95720.96090.95330.94010.95490.9641WorldView-2 20.94460.94180.93930.93860.95280.95370.9569Table2 depicts UIQI analysis of the proposed deep pan-sharpening model. UIQI is desirable to be maximum. It is found that the proposed model outperforms the competitive pan-sharpening models by 1.2498%.Table 2
Analysis of UIQI (maximum is desirable).
ImagesCSACNNMSLDMDLMDMLMPSOProposedPleiades 10.82270.82530.82520.82560.82070.81630.8288Pleiades 20.82860.82220.81680.82770.83330.83030.8365QuickBird 10.82760.82920.81610.82470.82190.82380.8324QuickBird 20.83240.82350.81820.82350.81880.82760.8356QuickBird 30.81760.83020.82160.83100.83220.81570.8354IKONOS 10.82530.82050.81520.82660.81550.82530.8298IKONOS 20.83240.83350.83250.83370.83050.82540.8369IKONOS 30.82090.83380.82920.81540.82490.82370.8375WorldView-2 10.82360.83470.81650.82940.83040.82530.8379WorldView-2 20.81850.81830.82750.82360.82160.83230.8355Table3 shows SAM analysis of the proposed deep pan-sharpening model. SAM is desirable to be minimum. It is found that the proposed model outperforms the competitive pan-sharpening models by showing an average reduction of 1.3457%.Table 3
Analysis of SAM (minimum is desirable).
ImagesCSACNNMSLDMDLMDMLMPSOProposedPleiades 16.12335.24974.85045.23145.61405.78764.8472Pleiades 25.03225.00385.23205.60895.01425.0275.0006QuickBird 15.58755.74935.65285.59095.59255.10865.1054QuickBird 25.31674.98015.22755.45225.07704.84954.8463QuickBird 34.86645.46975.55655.45245.63934.98154.8632IKONOS 15.63744.98824.84935.35914.93815.46984.8461IKONOS 25.59235.49015.54784.89915.75025.72764.8959IKONOS 35.17434.88635.67825.02555.17095.01384.8831WorldView-2 15.41055.16625.68275.69855.80125.33165.1635WorldView-2 25.05454.88305.38215.33355.08765.54234.8798The quality of pan-sharpened images can be assessed using ERGAS. It determines the transition between spectral and spatial information [49]. Table 4 demonstrates ERGAS analysis of the proposed deep pan-sharpening model. ERGAS is desirable to be minimum. It is found that the proposed model outperforms the competitive pan-sharpening models by showing an average reduction of 1.0985%.Table 4
Analysis of ERGAS (minimum is desirable).
ImagesCSACNNMSLDMDLMDMLMPSOProposedPleiades 16.46515.70226.03665.53567.01936.50083.0324Pleiades 27.07336.79865.52585.84207.58527.19213.7944QuickBird 15.16406.29837.11745.60547.01575.87083.2941QuickBird 26.21496.37347.72597.42696.08976.69303.0855QuickBird 35.32916.44816.49947.09566.74666.52063.4439IKONOS 15.04437.55065.34785.69937.66627.97032.9661IKONOS 25.49287.22826.82156.24755.43545.04933.2433IKONOS 37.33335.29855.97167.33136.44976.44983.4455WorldView-2 17.36295.97187.62876.04757.19615.46704.0428WorldView-2 25.93066.95766.24516.69747.12925.89113.2409Table5 demonstrates RMSE analysis of the proposed deep pan-sharpening model. RMSE is desirable to be minimum. It is found that the proposed model outperforms the competitive pan-sharpening models by showing an average reduction of 1.5486%.Table 5
Analysis of RMSE (minimum is desirable).
ImagesCSACNNMSLDMDLMDMLMPSOProposedPleiades 117.330117.581217.043518.640717.507518.809416.9898Pleiades 217.484017.439917.88318.632417.213217.738317.1600QuickBird 116.943717.204417.344318.127818.497918.718316.8905QuickBird 218.704417.996817.207518.69818.341218.395617.1543QuickBird 318.226018.759917.585816.943916.992118.109816.8907IKONOS 118.741117.687917.946217.433517.996617.184217.1314IKONOS 217.932218.27518.260218.160717.286316.953716.9005IKONOS 317.079617.884118.538717.960617.746117.840817.0264WorldView-2 117.951917.507417.392918.510518.465317.128617.0754WorldView-2 217.082718.563117.570118.694217.173118.168517.0295


## 4.1. Visual Analysis

Figures2 and 3 show the visual analysis of the proposed model. It is found that the proposed model has better visibility as compared to the existing techniques. Red rectangles show the specific region in the obtained fused images. The selected region reflects the spatial and spectral information along with any kind of artifacts which are present in the obtained fused images. Also, the proposed model shows better gradient and color preservation as compared to the existing techniques. The results obtained from the proposed model show better spatial and spectral information. It clearly shows that the existing models are able to fuse the images by improving the spatial and spectral information of fused images. But whenever there is redundant information in both PAN and MS images, then the existing method fails to fuse the content efficiently. Also, the texture and gradient preservation is significantly more in the fused image obtained using the proposed model.Figure 2
Visual analysis of pan-sharpening techniques. (a) Low-resolution multispectral (MS) image. (b) High-spatial-resolution panchromatic (PAN) image.
(a)(b)(c)(d)(e)(f)(g)(h)(i)Figure 3
Visual analysis of pan-sharpening techniques. (a) Low-resolution multispectral (MS) image. (b) High-spatial-resolution panchromatic (PAN) image.
(a)(b)(c)(d)(e)(f)(g)(h)(i)


## 4.2. Quantitative Analysis

Five well-known quality metrics, i.e., root mean square error (RMSE) [46], universal image quality index (UIQI) [47], correlation coefficient (CC) [46], spectral angle mapper (SAM) [46], and Erreur relative globale adimensionnelle de synthese (ERGAS) [48], are used for comparative analysis.Table1 shows CC analysis of the proposed deep pan-sharpening model. CC is desirable to be maximum. It is found that the proposed model outperforms the competitive pan-sharpening models by 1.7824%.Table 1
Analysis of correlation coefficient (maximum is desirable).
ImagesCSACNNMSLDMDLMDMLMPSOProposedPleiades 10.94820.95850.95250.93560.94950.95310.9612Pleiades 20.93840.95450.95880.96360.95950.95940.9668QuickBird 10.95430.95250.95170.95940.93530.94060.9626QuickBird 20.93880.93870.96270.94350.94510.93880.9659QuickBird 30.94220.94510.93860.94570.94810.94160.9513IKONOS 10.96010.94930.95890.93660.95470.96450.9677IKONOS 20.93910.96170.94420.94520.94050.95780.9649IKONOS 30.94220.95620.93850.95630.96020.95290.9634WorldView-2 10.93990.95720.96090.95330.94010.95490.9641WorldView-2 20.94460.94180.93930.93860.95280.95370.9569Table2 depicts UIQI analysis of the proposed deep pan-sharpening model. UIQI is desirable to be maximum. It is found that the proposed model outperforms the competitive pan-sharpening models by 1.2498%.Table 2
Analysis of UIQI (maximum is desirable).
ImagesCSACNNMSLDMDLMDMLMPSOProposedPleiades 10.82270.82530.82520.82560.82070.81630.8288Pleiades 20.82860.82220.81680.82770.83330.83030.8365QuickBird 10.82760.82920.81610.82470.82190.82380.8324QuickBird 20.83240.82350.81820.82350.81880.82760.8356QuickBird 30.81760.83020.82160.83100.83220.81570.8354IKONOS 10.82530.82050.81520.82660.81550.82530.8298IKONOS 20.83240.83350.83250.83370.83050.82540.8369IKONOS 30.82090.83380.82920.81540.82490.82370.8375WorldView-2 10.82360.83470.81650.82940.83040.82530.8379WorldView-2 20.81850.81830.82750.82360.82160.83230.8355Table3 shows SAM analysis of the proposed deep pan-sharpening model. SAM is desirable to be minimum. It is found that the proposed model outperforms the competitive pan-sharpening models by showing an average reduction of 1.3457%.Table 3
Analysis of SAM (minimum is desirable).
ImagesCSACNNMSLDMDLMDMLMPSOProposedPleiades 16.12335.24974.85045.23145.61405.78764.8472Pleiades 25.03225.00385.23205.60895.01425.0275.0006QuickBird 15.58755.74935.65285.59095.59255.10865.1054QuickBird 25.31674.98015.22755.45225.07704.84954.8463QuickBird 34.86645.46975.55655.45245.63934.98154.8632IKONOS 15.63744.98824.84935.35914.93815.46984.8461IKONOS 25.59235.49015.54784.89915.75025.72764.8959IKONOS 35.17434.88635.67825.02555.17095.01384.8831WorldView-2 15.41055.16625.68275.69855.80125.33165.1635WorldView-2 25.05454.88305.38215.33355.08765.54234.8798The quality of pan-sharpened images can be assessed using ERGAS. It determines the transition between spectral and spatial information [49]. Table 4 demonstrates ERGAS analysis of the proposed deep pan-sharpening model. ERGAS is desirable to be minimum. It is found that the proposed model outperforms the competitive pan-sharpening models by showing an average reduction of 1.0985%.Table 4
Analysis of ERGAS (minimum is desirable).
ImagesCSACNNMSLDMDLMDMLMPSOProposedPleiades 16.46515.70226.03665.53567.01936.50083.0324Pleiades 27.07336.79865.52585.84207.58527.19213.7944QuickBird 15.16406.29837.11745.60547.01575.87083.2941QuickBird 26.21496.37347.72597.42696.08976.69303.0855QuickBird 35.32916.44816.49947.09566.74666.52063.4439IKONOS 15.04437.55065.34785.69937.66627.97032.9661IKONOS 25.49287.22826.82156.24755.43545.04933.2433IKONOS 37.33335.29855.97167.33136.44976.44983.4455WorldView-2 17.36295.97187.62876.04757.19615.46704.0428WorldView-2 25.93066.95766.24516.69747.12925.89113.2409Table5 demonstrates RMSE analysis of the proposed deep pan-sharpening model. RMSE is desirable to be minimum. It is found that the proposed model outperforms the competitive pan-sharpening models by showing an average reduction of 1.5486%.Table 5
Analysis of RMSE (minimum is desirable).
ImagesCSACNNMSLDMDLMDMLMPSOProposedPleiades 117.330117.581217.043518.640717.507518.809416.9898Pleiades 217.484017.439917.88318.632417.213217.738317.1600QuickBird 116.943717.204417.344318.127818.497918.718316.8905QuickBird 218.704417.996817.207518.69818.341218.395617.1543QuickBird 318.226018.759917.585816.943916.992118.109816.8907IKONOS 118.741117.687917.946217.433517.996617.184217.1314IKONOS 217.932218.27518.260218.160717.286316.953716.9005IKONOS 317.079617.884118.538717.960617.746117.840817.0264WorldView-2 117.951917.507417.392918.510518.465317.128617.0754WorldView-2 217.082718.563117.570118.694217.173118.168517.0295


## 5. Conclusion

To obtain a remote sensing image with better spatial and spectral information, efficient image fusion techniques are desirable. However, it has been found that the existing models do not consider the inherent image distribution difference between MS and PAN images. Therefore, the obtained fused images suffer from gradient and color distortion problems. To overcome these problems, in this paper, an efficient deep transfer learning model has been proposed. Inception-ResNet-v2 model was improved by using a color-aware perceptual loss (CPL). The obtained fused images were further improved by using gradient channel prior as a postprocessing step. Gradient channel prior was utilized to preserve the color and gradient information. Extensive experiments were carried out by considering the benchmark datasets. Performance analysis has shown that the proposed model can efficiently preserve color and gradient information in the fused remote sensing images than the existing models. The proposed model outperformed the competitive pan-sharpening models in terms of CC and UIQI by1.7824% and 1.2498%, respectively. Also, compared to the existing models, the proposed model has achieved an average reduction in SAM, ERGAS, and RMSE by 1.3457%, 1.2847%, and 1.5486%, respectively.



---

*Source: 7615106-2021-12-23.xml*