# A Spectral-Aware Convolutional Neural Network for Pansharpening
## 1 Introduction  
Multiband optical (MBO) images, including multispectral (MS) and hyperspectral (HS) images, can provide higher spectral resolution than red, green and blue (RGB) images and panchromatic (PAN) images, which expands the difference of the target objects and increases their identifiability. Such characteristics can be used to improve the effectiveness of various image tasks such as change detection [1], classification [2], object recognition [3], and scene interpretation [4]. Among almost all of these tasks, MS or HS images with high spatial resolution are desired. However, physical constraints make it difficult to acquire high spatial resolution MBO (HR-MBO) images with a single sensor, which are with high spectral resolution and high spatial resolution simultaneously. One way to address this problem is to generate HR-MBO images by fusing low-spatial-resolution MBO (LR-MBO) images and the associated high-resolution PAN images which is usually called pansharpening. Over the last decades, a variety of pansharpening methods have been proposed in the literature [5; 6; 7; 8]. Often-used methods can be categorized into two main groups: component substitution (CS) methods and multiresolution analysis (MRA) methods. CS methods seek to replace a component of the LR-MBO image with the PAN image, usually in a suitable transformed domain. This class includes intensity-hue-saturation (IHS) [9; 10], principal component analysis (PCA) [11; 12; 13], Band-DependentSpatial-Detail (BDSD) [14], Gram-Schmidt (GS) [15], Gram-Schmidt adaptive (GSA) [16], Brovey transform (BT) [17; 18], and partial replacement adaptive component substitution (PRACS) [19]. Although these methods preserve accurate spatial information, they are often characterized by high spectral distortion, as the spectral ranges of the PAN and MBO images overlap only partially. MRA approaches regard pansharpening as a detailed extraction and injection process under multiresolution decomposition. This group consists of decimated wavelet transform (DWT) [20; 21], undecimated wavelet transform (UDWT) [22], _a trous_ wavelet transform (ATWT) [23], and Laplacian pyramid (LP) [24; 25; 26] methods. Compared with CS-based approaches, methods belonging to the MRA class can achieve higher spectral fidelity, but unfortunately, they may exhibit spatial distortion. Besides the abovementioned, other categorizations of pansharpening methods can also be identified. Matrix factorization (MF) methods refer to inferring a basis or dictionary of the spectral subspace by factorizing the LR-MBO image and the associated coefficients by involving the PAN image, which are then used to reconstruct the HR-MBO image. This type of method includes the nonnegative matrix factorization (NMF) [27] and coupled nonnegative matrix factorization (CNMF) [28]. Owing to simple factorization rules, MF methods are easy to implement and efficient. Bayesian approaches, following the Bayesian inference framework, involve prior knowledge about the expected characteristics of the pansharpened image to determine its posterior probability distribution and to thus draw the estimation. Different types of prior knowledge have been used to assign prior distributions for Bayesian designing [29; 30; 31]. This type of method attempts to include the fusion process within an interpretable framework and achieves a trade-off between spatial and spectral fidelity. Model-based optimization methods mainly rely on observation models between the ideal fused image and the observed data, which is generally posed as a problem of minimizing cost function through iterative optimization algorithms. Examples include the sparse representation-based (SR) approach [32; 33], the compressive sensing-based fusion (CSF) technique [34], and the online coupled dictionary learning (OCDL) [35]. Compared with the aforementioned methods, the model-based optimization methods can achieve higher spatial and spectral precision in the fusion result. However, it is relatively complicated and time-consuming due to the iterative optimization process.