# An Overview on Linear Unmixing of Hyperspectral Data

Authors: Jiaojiao Wei; Xiaofei Wang
Journal: Mathematical Problems in Engineering
 (2020)

---

## Abstract

Hyperspectral remote sensing technology has a strong capability for ground object detection due to the low spatial resolution of hyperspectral imaging spectrometers. A single pixel that leads to a hyperspectral remote sensing image usually contains more than one feature coverage type, resulting in a mixed pixel. The existence of a mixed pixel affects the accuracy of the ground object identification and classification and hinders the application and development of hyperspectral technology. For the problem of unmixing of mixed pixels in hyperspectral images (HSIs), the linear mixing model can model the mixed pixels well. Through the collation of nearly five years of the literature, this paper introduces the development status and problems of linear unmixing models from four aspects: geometric method, nonnegative matrix factorization (NMF), Bayesian method, and sparse unmixing.


---

## Body

## 1. Introduction

Hyperspectral imaging has a pivotal role in the field of remote sensing, which collects and processes information across the entire electromagnetic spectrum. We have witnessed a huge development in hyperspectral techniques during the past decade, which has reshaped the way we learn [1]. Hyperspectral images (HSIs) integrate spectral information which reflects radiation information of substances and image information which reflects the two-dimensional space information of matter. If the influence of atmospheric factors is ignored, each pixel of HSIs can be regarded as the comprehensive performance in the scene in tens or even hundreds of continuous waves of electromagnetic wave signals. HSIs are the images captured with hundreds of bands of electromagnetic spectrum generally in the range of 400 nm to 2500 nm, and its spectral resolution is generally less than 10 nm. HSIs have many spectral bands, narrow spectral range, and continuous spectral bands, which greatly enhance their ability to detect material attribute information, making hyperspectral technology widely used in earth observation at the early stage of development.In recent years, hyperspectral remote sensing imaging technology is used in many applications ranging from environmental monitoring to city planning and many other fields [2–4]. However, due to low spatial resolution of hyperspectral imaging spectrometers, each pixel may be a mixture of several spectra materials in a scene. The mixed pixel problem is not only a serious obstacle to the quantitative development of remote sensing technology but also seriously affects the application of computer technology in remote sensing field. Therefore, it is a key preprocessing to identify the component spectra from the mixed pixels and calculate the proportion of each component spectra [5] in the mixed pixels. This is called hyperspectral unmixing. Spectral unmixing technology is the most effective method to deal with mixed pixels. It can break for the limitation of spatial resolution of hyperspectral imaging spectrometers and express the real attributes of mixed pixels, thus improving the classification accuracy of hyperspectral images and applying them to the field of remote sensing. In addition, each mixed pixel into a collection of pure spectral signatures is called “endmembers,” and the corresponding percentages are called “abundances.” The algorithm of hyperspectral unmixing (HU) depends on the establishment of scene mixing model.The mixing model is used to describe how the materials in the scene interact with each other to form a composite spectrum within the pixel. Moreover, most mixing models can be characterized as linear spectral mixing model (LSMM) or nonlinear spectral mixing model (NLSMM) [6]. However, most of the existing HU methods are based on the LMM. When the spatial resolution of HSIs is below the meter level, the LSMM can better describe the actual spectral mixing phenomenon.Linear mixing assumes that photons arriving at the sensor interact with only one substance. However, when the size of the mixed element is small and the incident photon has multiple reflections and refraction, it will interact with a variety of substances, leading to nonlinear mixing. Spectral mixture model is shown in Figure1.Figure 1
Spectral mixture model. (a) Linear spectral mixture model. (b) Unlinear spectral mixture model.
(a)(b)There are many hyperspectral datasets used in the hyperspectral unmixing study. Urban is one of the most widely used hyperspectral dataset used in the hyperspectral unmixing study. Take the urban dataset as an example to introduce the hyperspectral dataset, and there are 307 × 307 pixels, each of which corresponds to a 2 × 2 square meter area. In this image, there are 210 wavelengths ranging from 400 nm to 2500 nm, resulting in a spectral resolution of 10 nm. After the channels 1–4, 76, 87, 101–111, 136–153, and 198–210 are removed, 162 channels remain. As shown in Figure2, it is the urban hyperspectral data cube and the corresponding endmembers reflection spectrum and abundance maps. The four endmembers are “asphalt road,” “grass,” “tree,” and “roof” respectively.Figure 2
The urban hyperspectral data cube and the corresponding endmembers reflection spectrum and abundance maps.The study of spectral unmixing has been carried out for more than 30 years. There are many literatures about spectral unmixing, and more than 70 references have been collected from conferences and journals, which cover the relevant contents of linear spectral unmixing model in the past five years. It can be seen that the linear spectral mixing model is still a hot topic at home and abroad, and it is also the most widely used model. The linear spectral mixing model has the advantages of simplicity, high efficiency, and clear physical meaning, which is scientific in theory, and the linear spectral mixing model can better describe the actual spectral mixing phenomenon for hyperspectral images with spatial resolution below the meter level.


## 2. Linear Spectral Mixing Model

In linear spectral mixing model, each mixed pixel can be expressed as a linear combination of endmembers weighted by its corresponding abundance.As shown in Figure3, Y=y1,y2,…,yk∈RL×K is the spectral image of K pixels and L bands, an endmember spectral library θ=θ1,θ2,…,θN∈RL×N containing N endmembers, where θii=1,2,…,N is the spectrum of L bands of the nth endmember, and the process of linear mixing can be described as follows:(1)Y=θW+E,where E is Gaussian noise, Y represents the observed spectral matrix, θ is the endmembers spectral matrix, and W is the abundance matrix. The hyperspectral unmixing usually consists of three main steps: estimation of the number of endmembers, extraction the spectral signatures of these endmembers, and estimation of the endmember abundances in each pixel.Figure 3
Linear hyperspectral unmixing.In this paper, linear spectral unmixing methods are divided into four categories: geometric method, nonnegative matrix factorization (NMF), Bayesian method, and sparse regression. Geometric methods treat the vertices of a single body as endmembers. Therefore, they are mainly used for endmembers extraction. The NMF algorithm has obvious nonconvexity and is easy to achieve local minimization. To solve this problem, the endmembers constraint and abundance constraint are briefly summarized. The goal of the Bayesian method is to achieve spectral unmixing by constructing endmembers and the maximum posterior probability of abundance. The likelihood function and prior information are sorted out, respectively. Bayesian method can realize both extraction and abundance estimation of endmembers and also can only realize abundance estimation when the endmembers are known. The goal of sparse regression is to estimate the abundance with the regression technique when the endmembers spectrum signatures are known. This part will be sorted out from two aspects of fitting error and sparse regression algorithm. Linear spectrum mixing method also includes archetypal analysis method, and the archetypal analysis unmixing algorithm is used to improve the objective function of convexity and false is the case, but the method of how to adjust the relaxation factor and scale parameter to accurately extract endmembers is still a problem. Therefore, this paper will not introduce in detail the research status of this work and the recent popular deep learning methods and the use of deep learning network models combined with linear spectral unmixing models for hyperspectral image unmixing. This paper only summarizes four kinds of linear hyperspectral unmixing methods: geometric method, nonnegative matrix factorization (NMF), Bayesian method, and sparse regression. The overview of this paper is shown in Figure4.Figure 4
Context diagram of the full-text summary.


## 3. Spectral Unmixing Based on the Geometric Method

The geometric unmixing method has a premise: every substance in the image data has a pure pixel. The geometric method is to extract the endmembers according to the distribution characteristics of the pixels in the geometric space and then unmix the images. Ideally, the spatial distribution of all pixels in the hyperspectral dataset is considered being located in a convex simplex. A convex simplex contains all data points. In the convex simplex space formed by data points, the vertex of the convex simplex is the endmembers. Therefore, the endmembers extraction is to obtain the endmembers by finding the vertex of the corresponding convex simplex. In the two-dimensional spatial data, the convex simplex is regarded as a triangle, while, in the three-dimensional spatial data, the convex simplex is a pyramid, as shown in Figure5, respectively. A convex simplex is a polyhedron in multidimensional space.Figure 5
Schematic diagram of a single body in space. (a) Schematic diagram of a single body in a two-dimensional space. (b) Schematic diagram of a single body in a three-dimensional space.
(a)(b)Endmembers extraction is one of the important steps of hyperspectral linear unmixing. Endmembers extraction methods are based on geometry, and the conventional methods include pixel purity index (PPI) [7], N-FINDR [8], successive projection algorithm (SPA) [9], and vertex component analysis (VCA) [10], and so on. Li-Guo [11] proposes a new geometric unmixing method, named geometric estimation method of spectral unmixing, which was constructed to completely meet the full constrained least squares (FCLS) requirements. Geometric methods are all based on the assumption that the pure pixels of different places in HSIs are contained. Mixed pixels make up a single row. With the sum to one constraint, the vector of the vertex is the pure endmembers. Although these methods are simple and fast to implement, they need light to satisfy the assumption of pure pixel, which rarely exists in practical applications. Xiao-Fei [12] stated that hyperspectral image data were classified into two parts: inner class and outer class. They proposed a hyperspectral image unmixing algorithm based on support vector data description. Li et al. [13] proposed a spectral unmixing resolution using extended support vector machines. Wang et al. [14] proposed a spectral unmixing model based on least squares support vector machine with unmixing residue constraints. Kocakusaklar et al. [15] used the total variation of spatiospectral to remove noise and spatial preprocessing to process spatial information in geometric algorithms. Moreover, these preprocessing steps can lead to closer endmembers signature estimates.


## 4. Spectral Unmixing Based on NMF

### 4.1. Nonnegative Matrix Factorization

NMF was first proposed by Lee and Seung [16] in the famous Nature magazine. Compared with some traditional algorithms, it is a convenient and fast implementation method with strong theoretical significance and many advantages, such as saving storage space. It provides a new idea for large-scale data processing.The mathematical model of NMF is as follows: find two nonnegative matrices;W∈RP×N approximately expresses the known nonnegative matrix V∈RL×N, which can be expressed as(2)V=θW.The study of hyperspectral unmixing based on NMF is based on blind source separation theory. From the perspective of unsupervised linear hyperspectral unmixing, nonnegative matrices before and after factorization are used in the process of NMF, which can meet the nonnegative requirements of endmembers matrix and abundance matrix of hyperspectral data [17–21]. For example, Lu et al. [22] proposed manifold regularized sparse NMF for hyperspectral unmixing. Wang et al. [23] proposed endmember dissimilarity constrained nonnegative matrix factorization method for hyperspectral unmixing. In a real hyperspectral solution of a mixed question, only nonnegative constraints are not enough, and the NMF algorithm has obvious nonconvexity and easily reaches the local minimum. So, the solution of such an algorithm had mixed results affected by the initial value matrix θ and the components belonging to degree matrix W had a greater influence on the constraint conditions and generally led to neglecting the important features of HSIs, such as spatial information characteristics.


### 4.2. Nonnegative Matrix Factorization Based on Constraints

Nonnegative matrix factorization has three constraints, including those on endmembers, abundance, and both endmembers and abundance.(1)
The objective function of the constrained NMF for endmembers matrix is(3)minR−θWF2+αJM,whereJ is the penalty function for M, and α is the regularized parameter.This kind of constrained NMF with minimum volume constrained (MVCNMF) [24] constrains the single-body volume of endmembers in the feature space. Iterative constrained endmembers (ICE) [25] constrains the cumulative distance between endmembers. Weighted endmember constrained nonnegative matrix factorization (WECNMF) [26] constrains the weighted distance of each vertex of a single form to the data center point. This kind of restriction makes the endmembers distribution conform to the single form structure of LSMM, but the abundance inversion results obtained by it cannot satisfy the abundance sparse. Andersen Man Shun Ang [27] considered the nonnegative matrix factorization (NMF) with a regularization that promotes small volume of the convex hull spanned by the basis matrix. He wrote the article algorithms and comparisons of nonnegative matrix factorizations with volume regularization for hyperspectral unmixing.(2)
Considering the distribution law of ground objects, sparsity can be considered the inherent property of abundance matrix. Hyperspectral unmixing based on sparse NMF is based on nonnegative matrix model. Taking the sum of the nonnegative constraint and abundant constraint as a constraint matrix, the model becomes a new form of NMF and a group of sparse constraints. Among them, the most typical is thel0 norm sparse constraints.The objective function is(4)fA,S=12X−AS22+λS0,s.t.A≥0,S≥0,1PTS=1NT,whereX∈RL×N is the hyperspectral image data matrix, A∈RL×P is the matrix containing P endmembers, S∈RP×N is the abundance coefficient matrix, λS0 is the sparse additional constraint function of l0, and λ∈R is the regularization parameter.Althoughl0 norm has good sparsity, its hyperspectral unmixing is an NP problem, which is not suitable for practical problems. Therefore, many scholars have made improvements on this basis. Candes et al. [28] proved that l0 norm can be replaced by l0 norm in RIP condition. l0 Norm can be solved faster than l0 norm. Qian et al. [29] applied l1/2 norm to nonnegative matrix unmixing algorithm. The result is much stable and sparse. Although l1/2 sparse regularization constraint is convenient to solve, its robustness and sparsity are still poor. Therefore, Gao [30] proposed a nonnegative matrix unmixing model algorithm based on approximate l0 constraint to be closer to the image unmixing result of l0 sparse constraint. However, the l0 norm will lead to the nondifferentiability of the objective function. Liu et al. [31] introduced the abundance separation and smoothness into NMF and obtained a new NMF method called abundance separation and smoothness constraint (ASSNMF). The abundance separation minimizes the mutual information between the abundance distributions of different endmembers, and the abundance smoothness constraint is based on the fact that the surrounding objects usually change slowly and rarely suddenly. However, the disadvantages of ASSNMF are its dependence on parameters and high computational complexity.Lu et al. [22] proposed a sparse NMF algorithm based on manifold regularization, which integrated manifold regularization into sparse constrained NMF hyperspectral unmixing algorithm, thus getting better unmixing effect. Jiang et al. [32] proposed a semisupervised NMF method based on sparse constraints and graph regularization, which could maintain the geometric structure of data when performing low-dimensional nonnegative factorization. However, this algorithm was an unsupervised algorithm, requiring a large amount of prior knowledge, which was not conducive to its application in practice. Xu et al. [33] proposed a multitask joint sparse decommissioning method based on spatial characteristics and jointed sparse decommissioning ideas of HSIs such as neighborhood and nonlocal similarity in images. However, this method does not fully use the typical correlation characteristics of hyperspectral images in space and spectrum. Wang et al. [34] proposed spatial group sparsity regularized NMF for hyperspectral unmixing, introducing group structure is prior information of HSIs optimization of NMF, and the data is organized into a space group. The structure and abundance sparsity of space group are taken as the improved mixed mode regularization, and the shared sparsity mode is used to avoid the loss of space details in space group.(3)
The current NMF series algorithms are studied, including constraints on endmembers, constraints on abundance, and constraints on both endmembers and abundance. The NMF algorithm only imposes unilateral restrictions on endmembers or abundances and does not adequately reflect the characteristics of hyperspectral remote sensing data. In terms of endmembers and abundances, the results are inconsistent with the actual situation.Constrained NMF methods based on endmembers matrix and abundance matrix are sparsity promoting iterative constrained endmembers (SPICE) [35], collaborative NMF (CoNMF) [36], and NMF with piecewise smoothness constraint (PSCNMF) [37]. SPICE is based on ICE, and it adds the sparse constraint term cooperative NMF. The CoNMF method uses cooperative regular prior constraint abundance and the cumulative distance constraint endmembers of the data cloud center. PSCNMF introduced piecewise smoothness of spectral data and sparsity of abundance fraction of each substance into NMF. Gayathri and Renjith [38] take the denoising algorithm as THE preprocessing step produces high spatial resolution HSIs, by the endmembers spectra of low spatial resolution HSI and the abundance fractions of high spatial resolution multispectral image. to enhance the spatial resolution of HSIs.


### 4.3. Nonnegative Matrix Factorization Based on Spatial Information

HSIs have spatial feature correlation and spectral feature correlation. Some scholars have studied hyperspectral unmixing from this aspect. Liu et al. [39] proposed a nonnegative matrix factorization method with the constraints of abundance separability and smoothness, which considered the relationship between endmembers and the spatial information of each endmember, and constrained the results of NMF from the frequency domain and the spatial domain, respectively.Zhang [40] designed a hyperspectral data unmixing method based on local low-rank constrained NMF (LLrNMF) to address the shortcoming of existing NMF linear unmixing model that the spatial structure information is not fully utilized. First, HSIs are segmented into superpixels, and low-rank constraints are added to the segmented superpixels and the NFM model. This method can keep spatial structure information when the constraints are sparse. The SS-NMF algorithm proposed by Zhu et al. [41] introduced the spatial constraint term to improve the sparse algorithm. Fang et al. [42] proposed a hyperspectral unmixing algorithm based on the sparsity of abundance and the local invariance of images, which combined constrained NMF and improved spatial spectral preprocessing to perform the mixed pixel unmixing of HSIs.


## 4.1. Nonnegative Matrix Factorization

NMF was first proposed by Lee and Seung [16] in the famous Nature magazine. Compared with some traditional algorithms, it is a convenient and fast implementation method with strong theoretical significance and many advantages, such as saving storage space. It provides a new idea for large-scale data processing.The mathematical model of NMF is as follows: find two nonnegative matrices;W∈RP×N approximately expresses the known nonnegative matrix V∈RL×N, which can be expressed as(2)V=θW.The study of hyperspectral unmixing based on NMF is based on blind source separation theory. From the perspective of unsupervised linear hyperspectral unmixing, nonnegative matrices before and after factorization are used in the process of NMF, which can meet the nonnegative requirements of endmembers matrix and abundance matrix of hyperspectral data [17–21]. For example, Lu et al. [22] proposed manifold regularized sparse NMF for hyperspectral unmixing. Wang et al. [23] proposed endmember dissimilarity constrained nonnegative matrix factorization method for hyperspectral unmixing. In a real hyperspectral solution of a mixed question, only nonnegative constraints are not enough, and the NMF algorithm has obvious nonconvexity and easily reaches the local minimum. So, the solution of such an algorithm had mixed results affected by the initial value matrix θ and the components belonging to degree matrix W had a greater influence on the constraint conditions and generally led to neglecting the important features of HSIs, such as spatial information characteristics.


## 4.2. Nonnegative Matrix Factorization Based on Constraints

Nonnegative matrix factorization has three constraints, including those on endmembers, abundance, and both endmembers and abundance.(1)
The objective function of the constrained NMF for endmembers matrix is(3)minR−θWF2+αJM,whereJ is the penalty function for M, and α is the regularized parameter.This kind of constrained NMF with minimum volume constrained (MVCNMF) [24] constrains the single-body volume of endmembers in the feature space. Iterative constrained endmembers (ICE) [25] constrains the cumulative distance between endmembers. Weighted endmember constrained nonnegative matrix factorization (WECNMF) [26] constrains the weighted distance of each vertex of a single form to the data center point. This kind of restriction makes the endmembers distribution conform to the single form structure of LSMM, but the abundance inversion results obtained by it cannot satisfy the abundance sparse. Andersen Man Shun Ang [27] considered the nonnegative matrix factorization (NMF) with a regularization that promotes small volume of the convex hull spanned by the basis matrix. He wrote the article algorithms and comparisons of nonnegative matrix factorizations with volume regularization for hyperspectral unmixing.(2)
Considering the distribution law of ground objects, sparsity can be considered the inherent property of abundance matrix. Hyperspectral unmixing based on sparse NMF is based on nonnegative matrix model. Taking the sum of the nonnegative constraint and abundant constraint as a constraint matrix, the model becomes a new form of NMF and a group of sparse constraints. Among them, the most typical is thel0 norm sparse constraints.The objective function is(4)fA,S=12X−AS22+λS0,s.t.A≥0,S≥0,1PTS=1NT,whereX∈RL×N is the hyperspectral image data matrix, A∈RL×P is the matrix containing P endmembers, S∈RP×N is the abundance coefficient matrix, λS0 is the sparse additional constraint function of l0, and λ∈R is the regularization parameter.Althoughl0 norm has good sparsity, its hyperspectral unmixing is an NP problem, which is not suitable for practical problems. Therefore, many scholars have made improvements on this basis. Candes et al. [28] proved that l0 norm can be replaced by l0 norm in RIP condition. l0 Norm can be solved faster than l0 norm. Qian et al. [29] applied l1/2 norm to nonnegative matrix unmixing algorithm. The result is much stable and sparse. Although l1/2 sparse regularization constraint is convenient to solve, its robustness and sparsity are still poor. Therefore, Gao [30] proposed a nonnegative matrix unmixing model algorithm based on approximate l0 constraint to be closer to the image unmixing result of l0 sparse constraint. However, the l0 norm will lead to the nondifferentiability of the objective function. Liu et al. [31] introduced the abundance separation and smoothness into NMF and obtained a new NMF method called abundance separation and smoothness constraint (ASSNMF). The abundance separation minimizes the mutual information between the abundance distributions of different endmembers, and the abundance smoothness constraint is based on the fact that the surrounding objects usually change slowly and rarely suddenly. However, the disadvantages of ASSNMF are its dependence on parameters and high computational complexity.Lu et al. [22] proposed a sparse NMF algorithm based on manifold regularization, which integrated manifold regularization into sparse constrained NMF hyperspectral unmixing algorithm, thus getting better unmixing effect. Jiang et al. [32] proposed a semisupervised NMF method based on sparse constraints and graph regularization, which could maintain the geometric structure of data when performing low-dimensional nonnegative factorization. However, this algorithm was an unsupervised algorithm, requiring a large amount of prior knowledge, which was not conducive to its application in practice. Xu et al. [33] proposed a multitask joint sparse decommissioning method based on spatial characteristics and jointed sparse decommissioning ideas of HSIs such as neighborhood and nonlocal similarity in images. However, this method does not fully use the typical correlation characteristics of hyperspectral images in space and spectrum. Wang et al. [34] proposed spatial group sparsity regularized NMF for hyperspectral unmixing, introducing group structure is prior information of HSIs optimization of NMF, and the data is organized into a space group. The structure and abundance sparsity of space group are taken as the improved mixed mode regularization, and the shared sparsity mode is used to avoid the loss of space details in space group.(3)
The current NMF series algorithms are studied, including constraints on endmembers, constraints on abundance, and constraints on both endmembers and abundance. The NMF algorithm only imposes unilateral restrictions on endmembers or abundances and does not adequately reflect the characteristics of hyperspectral remote sensing data. In terms of endmembers and abundances, the results are inconsistent with the actual situation.Constrained NMF methods based on endmembers matrix and abundance matrix are sparsity promoting iterative constrained endmembers (SPICE) [35], collaborative NMF (CoNMF) [36], and NMF with piecewise smoothness constraint (PSCNMF) [37]. SPICE is based on ICE, and it adds the sparse constraint term cooperative NMF. The CoNMF method uses cooperative regular prior constraint abundance and the cumulative distance constraint endmembers of the data cloud center. PSCNMF introduced piecewise smoothness of spectral data and sparsity of abundance fraction of each substance into NMF. Gayathri and Renjith [38] take the denoising algorithm as THE preprocessing step produces high spatial resolution HSIs, by the endmembers spectra of low spatial resolution HSI and the abundance fractions of high spatial resolution multispectral image. to enhance the spatial resolution of HSIs.


## 4.3. Nonnegative Matrix Factorization Based on Spatial Information

HSIs have spatial feature correlation and spectral feature correlation. Some scholars have studied hyperspectral unmixing from this aspect. Liu et al. [39] proposed a nonnegative matrix factorization method with the constraints of abundance separability and smoothness, which considered the relationship between endmembers and the spatial information of each endmember, and constrained the results of NMF from the frequency domain and the spatial domain, respectively.Zhang [40] designed a hyperspectral data unmixing method based on local low-rank constrained NMF (LLrNMF) to address the shortcoming of existing NMF linear unmixing model that the spatial structure information is not fully utilized. First, HSIs are segmented into superpixels, and low-rank constraints are added to the segmented superpixels and the NFM model. This method can keep spatial structure information when the constraints are sparse. The SS-NMF algorithm proposed by Zhu et al. [41] introduced the spatial constraint term to improve the sparse algorithm. Fang et al. [42] proposed a hyperspectral unmixing algorithm based on the sparsity of abundance and the local invariance of images, which combined constrained NMF and improved spatial spectral preprocessing to perform the mixed pixel unmixing of HSIs.


## 5. Spectral Unmixing Based on the Bayesian Method

The unmixing of mixed pixels is an ill-conditioned inverse process, so the result is not unique. However, if one can use some prior knowledge and some additional assumptions, a better solution can be obtained. Bayesian method can incorporate meaningful prior information into the modeling process through good statistical means [43] to model the variability and uncertainty existing in spectral data, abundance, and endmembers. The main idea of Bayesian method is to deduce the posterior probability density through the prior distribution and likelihood probability:(5)pθ,WY,φ=pYθ,W,σ2pθφpWφpσ2φpY,where σ2, θ, and W obey some probability distribution, and the parameters of these distributions functions make up the super-parameter φ. pYθ,W,σ2 is the likelihood function, pθφ is the prior distribution of endmembers, pWφ is the prior distribution of abundance, and pσ2φ is the prior distribution of noise.There are some spectral unmixing algorithms based on Bayesian Method. Dobigeon et al. [44] proposed a hierarchical Bayesian model that can be used for semisupervised hyperspectral image unmixing. The posterior distribution of the unknown model parameters is deduced by constraint conditions. Chen et al. [45] used a sparse Bayesian model to hyperspectral image unmixing. They proposed to solve the spectral unmixing problem by using sparse Bayesian learning (SBL) framework. Fan-Qiang et al. [46] presented a compound regularized multiple sparse Bayesian learning algorithm for sparse unmixing, in which sparse Bayesian learning model is integrated in the linear hyperspectral pixel unmixing.In addition, when using spectral unmixing based on Bayesian method, focus should be on the likelihood function, abundance prior distribution, endmembers prior distribution, noise prior distribution, etc.


### 5.1. Likelihood Function

Let the mixed model of the observed spectrum be written as a vector:y=θw+e, where e is noise. At present, the expression forms of likelihood function are mainly divided into two types. We can express the first kind of likelihood function model as a multivariate Gaussian model [47]. Its expression is(6)pyw,β=Nywθ,β−1I=2π−N/2β−N/2exp−β2y−θw22.Most literature studies adopt this expression method of likelihood function. The second likelihood function model is proposed by Wu et al. [48], who believe that the previously mentioned method cannot deal with the situation of outliers and point out that y is not always Gaussian, and occasionally there may be outliers, at which time y can be expressed as(7)pyw,β=1−αNyw,β−1I+αhy,where α represents the probability that y is the abnormal point, and h. represents the probability density function of the abnormal point, whose probability density function is uniformly distributed. This method uses stochastic maximum likelihood algorithm to solve the parameters of the solution model.


### 5.2. Prior Distribution

It summarizes the prior distribution of endmembers from four aspects. We can regard uniform distribution terminal element as the nonnegative real number information distribution, the uniform distribution between zero and infinity. The calculation of uniform distribution is simple; however, the information of uniform distribution is insufficient. Gaussian distribution is modeled with Gaussian distribution as the endmembers. On the premise that the endmembers obey the Gaussian distribution, maximum expectation algorithm [49] is used to solve the mean value and variance, but they cannot measure the variability of endmembers spectrum. The beta distribution of endmembers can express the skewness of distribution and spectral variability. The distribution treats each terminal element of each band as a inferred beta distribution. The previously mentioned single distribution cannot express the complex scene information, and the Gaussian mixture distribution usually assumes that the endmembers in each pixel obey a single Gaussian distribution or beta distribution. By constructing a Gaussian mixture model of the endmembers, it approximates more complex scenes, achieving good results, but also increasing the computational complexity [50].The mathematical models of abundance prior include the uniform distribution, Gaussian distribution, and Dirichlet distribution. Uniform distribution is selected as the prior probability distribution of the abundance vector according to the constraint that the abundance must satisfy the nonnegative sum and sum to be 1. Gaussian prior distribution can divide spectral data into several homogeneous regions and calculate the variance and mean of homogeneous regions. Dirichlet distribution can automatically be satisfied with the abundance sum to one constraint (ASC) and abundance nonnegative constraints (ANC) and can deal with the usual statistical dependence problems in hyperspectral data. Dirichlet distribution can overcome the problem of the inadequate description of prior information by a uniform distribution.


## 5.1. Likelihood Function

Let the mixed model of the observed spectrum be written as a vector:y=θw+e, where e is noise. At present, the expression forms of likelihood function are mainly divided into two types. We can express the first kind of likelihood function model as a multivariate Gaussian model [47]. Its expression is(6)pyw,β=Nywθ,β−1I=2π−N/2β−N/2exp−β2y−θw22.Most literature studies adopt this expression method of likelihood function. The second likelihood function model is proposed by Wu et al. [48], who believe that the previously mentioned method cannot deal with the situation of outliers and point out that y is not always Gaussian, and occasionally there may be outliers, at which time y can be expressed as(7)pyw,β=1−αNyw,β−1I+αhy,where α represents the probability that y is the abnormal point, and h. represents the probability density function of the abnormal point, whose probability density function is uniformly distributed. This method uses stochastic maximum likelihood algorithm to solve the parameters of the solution model.


## 5.2. Prior Distribution

It summarizes the prior distribution of endmembers from four aspects. We can regard uniform distribution terminal element as the nonnegative real number information distribution, the uniform distribution between zero and infinity. The calculation of uniform distribution is simple; however, the information of uniform distribution is insufficient. Gaussian distribution is modeled with Gaussian distribution as the endmembers. On the premise that the endmembers obey the Gaussian distribution, maximum expectation algorithm [49] is used to solve the mean value and variance, but they cannot measure the variability of endmembers spectrum. The beta distribution of endmembers can express the skewness of distribution and spectral variability. The distribution treats each terminal element of each band as a inferred beta distribution. The previously mentioned single distribution cannot express the complex scene information, and the Gaussian mixture distribution usually assumes that the endmembers in each pixel obey a single Gaussian distribution or beta distribution. By constructing a Gaussian mixture model of the endmembers, it approximates more complex scenes, achieving good results, but also increasing the computational complexity [50].The mathematical models of abundance prior include the uniform distribution, Gaussian distribution, and Dirichlet distribution. Uniform distribution is selected as the prior probability distribution of the abundance vector according to the constraint that the abundance must satisfy the nonnegative sum and sum to be 1. Gaussian prior distribution can divide spectral data into several homogeneous regions and calculate the variance and mean of homogeneous regions. Dirichlet distribution can automatically be satisfied with the abundance sum to one constraint (ASC) and abundance nonnegative constraints (ANC) and can deal with the usual statistical dependence problems in hyperspectral data. Dirichlet distribution can overcome the problem of the inadequate description of prior information by a uniform distribution.


## 6. Spectral Unmixing Based on the Sparse Method

### 6.1. Sparse Unmixing Method

With the appearance of the spectral library of the USGS, sparse unmixing has attracted more and more attention. The aim is to estimate the abundance of endmembers in spectral images based on the known endmembers spectral library. It assumes that each observed feature linearly combines only a few spectra from a known spectral library. The known endmembers spectrum library contains a large amount of spectral information of pure endmembers, from which several endmembers spectra are selected to approximate the spectra of mixed pixels, which will definitely lead to sparsity of abundance. Therefore, this method is called sparse unmixing or abundance estimation. The basic model of sparse unmixing is(8)minw12Y−θWF2+λW1,1,W≥0,1TW=1.The model and the method to solve the model are called the robustness of variable separation and sum, but it complicates the calculation process.


### 6.2. Sparse Unmixing Algorithm

There are five types of commonly used sparse unmixing algorithms: greedy algorithm, convex relaxation algorithm, sparse Bayesian algorithm, nonconvex optimization method, and brute force method. The most famous greedy algorithm is Mallat’s matching pursuit algorithm [51]. Then, Pati proposed OMP, an orthogonal matching pursuit algorithm based on the MP algorithm, which can converge faster than the MP algorithm.Then, simultaneous orthogonal matching pursuit (SOMP) [52] algorithm is proposed. SOMP algorithm is also a typical simultaneous greedy algorithm for sparse unmixing, which involves finding the optimal subset of signatures for the observed data from a spectral library. However, the numbers of endmembers selected by SOMP are still large, so Kong et al. [53] present a variant of SOMP, termed backtracking-based SOMP (BSOMP), for sparse unmixing of hyperspectral data. Tang et al. [54] proposed regularized simultaneous forward-backward greedy algorithm for sparse unmixing of hyperspectral data. Shi et al. [55] proposed subspace matching pursuit for sparse unmixing of hyperspectral data.Sparse Bayesian method (SBL) uses the parameterized Gaussian distribution as the solution to get the prior distribution. Wipf and Rao [56] theoretically proved that the sparse Bayesian algorithm can get the sparsest solution. It has a series of obvious advantages. When the correlation between the columns of the perceptive matrix is very strong, performing the convex relaxation and greedy algorithm will be poor, but the SBL method still has a good effect.The nonconvex optimization method tries to reducel0 problem to a related nonconvex problem, but its computational complexity is relatively high. The convex relaxation method usually replaces the l0 norm with l1 norm or the other norm, but the l1 norm cannot guarantee that the obtained abundance vector is sparse enough.There are many new and common sparse unmixing methods. Sigurdsson et al. [57] developed a distributed sparse hyperspectral unmixing algorithm using the alternating direction method of multipliers algorithm and l1 sparse regularization. Tang et al. [58] presented a new algorithm, which is termed sparse unmixing using spectral a priori information, to tackle the noise in observed spectral vectors and high mutual coherence of spectral libraries. Seyyedsalehi et al. [59] proposed a probabilistic sparse regression method for linear hyperspectral unmixing, which utilizes the implicit relations of neighboring pixels. For HSIs, low spatial and spectral resolution will cause inaccurate unmixing. Yang et al. [60] proposed a novel jointly spatial-spectral resolution enhancement algorithm. Yidan et al. [61] put forward a local adaptive sparse unmixing based fusion (LASUF) algorithm, in which the sparsity of the abundance matrices is appended as the constraint to the optimization fusion, considering the limited categories of ground objects in a specific range and the local correlation of their distribution. Aggarwal and Majumdar [62] took into account both the Gaussian noise and sparse noise. The joint-sparsity of abundance maps is used to solve the unmixing problem. The sparse unmixing method suffers from sensitivity of the sparse coefficients. Zhang et al. [63] introduced weighting factors to penalize the nonzero coefficients in the solution. At the same time, hyperspectral unmixing is carried out by using spatial information and spectral information, and a new spectral-spatial weighted sparse unmixing framework is proposed to further imposing sparsity.To better extract the different levels of spatial details, Ruyi et al. [64] proposed rolling guidance based scale-aware spatial sparse unmixing (RGSU) to extract and recover the different levels important structures and details in the hyperspectral remote sensing image unmixing procedure. Wang et al. [65] introduced redundant spectrum to represent the spectral variation caused by ion substitution and develop a sparse redundant unmixing model by adding the redundant regularization into the classical sparse regression formulation.Wang et al. [66] proposed some ideas of future work on structure sparsity in the unmixing of hyperspectral image, such as spatial smoothness, collaborative sparsity, and group structure of spectral library. Sigurdsson et al. [67] developed an algorithm based on l1 sparse and low rank hyperspectral unmixing, to remove outliers and structured noise and improve the accuracy of unmixing.Prior work for sparse unmixing usually utilizesl1 norm, but the l1 norm is not differentiable, which may lead to unstable results. Ruyi et al. [68] adopt Bregman divergence for sparse unmixing, which is a differentiable, smoother prior. Iordache et al. [69] put forward the SUNSAL-TV algorithm after taking into account the similarity or similar characteristics of local pixels. Solving the sparse mixing problem directly is difficult for NP. Lin et al. [70] proposed a new sparse unmixing algorithm named.Spectral library pruning method in hyperspectral sparse unmixing aims at finding the optimal subset of signatures from a spectral library to best model each pixel in HSIs and estimating their corresponding abundance. However, the sparsity and accuracy of the results are not well-guaranteed. In this case, Zhenwei et al. [71] proposed a new mixed algorithm named collaborative sparse hyperspectral unmixing using l0 norm, designed to solve the problem of l0. Xia Xu et al. [72] proposed a new sparse unmixing method for HSIs via integrating the pruning operation into the optimization process. They developed a new multiobjective-based method where reconstruction error, sparsity error, and the projection function are considered three parallel objectives that could be optimized simultaneously. This method has some advantages in high-noise conditions. Huang et al. [73] added joint sparse block and low rank said to hyperspectral solution mixing and put forward the joint-sparse-blocks and low-rank representation for hyperspectral unmixing.Using spatial information to improve spectral unmixing results has been a trend in recent years. These methods usually assume that the abundance of pixels is piecewise smooth. However, in reality, the abundance may vary dramatically depending on the pixel. Therefore, Zhang [74] proposed a new strategy to preserve the spatial details in the abundance maps via a spatial discontinuity weight. To make the structure clearer, Table 1 of this paper briefly introduces the pros and cons of some algorithms mentioned in this section.Table 1
The pros and cons of some sparse unmixing methods.
AuthorYearProsConsSigurdsson et al.2016This algorithm can be applied on very large hyperspectral image where a standard algorithm could not be used due to memory limitations. And each subproblem does not need to have access to the whole hyperspectral image.Endmembers in the scene are assumed to be the same with its corresponding sample in the spectral library; this assumption is too ideal in the real environment.Wang et al.2017To solve the existing spectral variability between the measured endmembers in the real environment and corresponding ones in the spectral library.The method only uses spectral information but ignores the spatial-contextual information.Sigurdsson et al.2017A novel sparse outlier component is used to remove outliers and structured noise from the unmixing algorithm.The algorithm is easy to make the estimated data appear over smooth.Ruyi et al.2017This paper adopts Bregman divergence for sparse unmixing, which is a differentiable, smoother prior. And based on the maximum a posterior estimation, the proposed method has achieved sparse, stable, and precise fractional abundances.Endmembers in the scene are assumed to be the same with its corresponding sample in the spectral library, and this assumption is too ideal in the real environment.Zhenwei et al.2018l0 norm was used to solvel 0 problem directly. The antinoise ability of this algorithm also gets improved.This algorithm is difficult to integrate into a larger learning architecture.Huang et al.2019This paper developed a two-level reweighting strategy to enhance the sparsity along the rows within each block and considered spatial correlation among nearby pixels.If this algorithm will be applied on very large hyperspectral image, the computation time will be too long.The sparse unmixing algorithm has obvious disadvantages. Before using the sparse unmixing algorithm based on spectral library, spectral libraries composed of different matter spectra must be obtained. However, for the influence of many factors on the natural features, there will be some differences between the spectral library and the actual features. There is a strong correlation between the spectral curves of the features in the spectral reservoir, so it is difficult to ensure the stability of the unmixing results. However, the sparse unmixing algorithm based on spectral library does not make full use of the spatial information of images.


## 6.1. Sparse Unmixing Method

With the appearance of the spectral library of the USGS, sparse unmixing has attracted more and more attention. The aim is to estimate the abundance of endmembers in spectral images based on the known endmembers spectral library. It assumes that each observed feature linearly combines only a few spectra from a known spectral library. The known endmembers spectrum library contains a large amount of spectral information of pure endmembers, from which several endmembers spectra are selected to approximate the spectra of mixed pixels, which will definitely lead to sparsity of abundance. Therefore, this method is called sparse unmixing or abundance estimation. The basic model of sparse unmixing is(8)minw12Y−θWF2+λW1,1,W≥0,1TW=1.The model and the method to solve the model are called the robustness of variable separation and sum, but it complicates the calculation process.


## 6.2. Sparse Unmixing Algorithm

There are five types of commonly used sparse unmixing algorithms: greedy algorithm, convex relaxation algorithm, sparse Bayesian algorithm, nonconvex optimization method, and brute force method. The most famous greedy algorithm is Mallat’s matching pursuit algorithm [51]. Then, Pati proposed OMP, an orthogonal matching pursuit algorithm based on the MP algorithm, which can converge faster than the MP algorithm.Then, simultaneous orthogonal matching pursuit (SOMP) [52] algorithm is proposed. SOMP algorithm is also a typical simultaneous greedy algorithm for sparse unmixing, which involves finding the optimal subset of signatures for the observed data from a spectral library. However, the numbers of endmembers selected by SOMP are still large, so Kong et al. [53] present a variant of SOMP, termed backtracking-based SOMP (BSOMP), for sparse unmixing of hyperspectral data. Tang et al. [54] proposed regularized simultaneous forward-backward greedy algorithm for sparse unmixing of hyperspectral data. Shi et al. [55] proposed subspace matching pursuit for sparse unmixing of hyperspectral data.Sparse Bayesian method (SBL) uses the parameterized Gaussian distribution as the solution to get the prior distribution. Wipf and Rao [56] theoretically proved that the sparse Bayesian algorithm can get the sparsest solution. It has a series of obvious advantages. When the correlation between the columns of the perceptive matrix is very strong, performing the convex relaxation and greedy algorithm will be poor, but the SBL method still has a good effect.The nonconvex optimization method tries to reducel0 problem to a related nonconvex problem, but its computational complexity is relatively high. The convex relaxation method usually replaces the l0 norm with l1 norm or the other norm, but the l1 norm cannot guarantee that the obtained abundance vector is sparse enough.There are many new and common sparse unmixing methods. Sigurdsson et al. [57] developed a distributed sparse hyperspectral unmixing algorithm using the alternating direction method of multipliers algorithm and l1 sparse regularization. Tang et al. [58] presented a new algorithm, which is termed sparse unmixing using spectral a priori information, to tackle the noise in observed spectral vectors and high mutual coherence of spectral libraries. Seyyedsalehi et al. [59] proposed a probabilistic sparse regression method for linear hyperspectral unmixing, which utilizes the implicit relations of neighboring pixels. For HSIs, low spatial and spectral resolution will cause inaccurate unmixing. Yang et al. [60] proposed a novel jointly spatial-spectral resolution enhancement algorithm. Yidan et al. [61] put forward a local adaptive sparse unmixing based fusion (LASUF) algorithm, in which the sparsity of the abundance matrices is appended as the constraint to the optimization fusion, considering the limited categories of ground objects in a specific range and the local correlation of their distribution. Aggarwal and Majumdar [62] took into account both the Gaussian noise and sparse noise. The joint-sparsity of abundance maps is used to solve the unmixing problem. The sparse unmixing method suffers from sensitivity of the sparse coefficients. Zhang et al. [63] introduced weighting factors to penalize the nonzero coefficients in the solution. At the same time, hyperspectral unmixing is carried out by using spatial information and spectral information, and a new spectral-spatial weighted sparse unmixing framework is proposed to further imposing sparsity.To better extract the different levels of spatial details, Ruyi et al. [64] proposed rolling guidance based scale-aware spatial sparse unmixing (RGSU) to extract and recover the different levels important structures and details in the hyperspectral remote sensing image unmixing procedure. Wang et al. [65] introduced redundant spectrum to represent the spectral variation caused by ion substitution and develop a sparse redundant unmixing model by adding the redundant regularization into the classical sparse regression formulation.Wang et al. [66] proposed some ideas of future work on structure sparsity in the unmixing of hyperspectral image, such as spatial smoothness, collaborative sparsity, and group structure of spectral library. Sigurdsson et al. [67] developed an algorithm based on l1 sparse and low rank hyperspectral unmixing, to remove outliers and structured noise and improve the accuracy of unmixing.Prior work for sparse unmixing usually utilizesl1 norm, but the l1 norm is not differentiable, which may lead to unstable results. Ruyi et al. [68] adopt Bregman divergence for sparse unmixing, which is a differentiable, smoother prior. Iordache et al. [69] put forward the SUNSAL-TV algorithm after taking into account the similarity or similar characteristics of local pixels. Solving the sparse mixing problem directly is difficult for NP. Lin et al. [70] proposed a new sparse unmixing algorithm named.Spectral library pruning method in hyperspectral sparse unmixing aims at finding the optimal subset of signatures from a spectral library to best model each pixel in HSIs and estimating their corresponding abundance. However, the sparsity and accuracy of the results are not well-guaranteed. In this case, Zhenwei et al. [71] proposed a new mixed algorithm named collaborative sparse hyperspectral unmixing using l0 norm, designed to solve the problem of l0. Xia Xu et al. [72] proposed a new sparse unmixing method for HSIs via integrating the pruning operation into the optimization process. They developed a new multiobjective-based method where reconstruction error, sparsity error, and the projection function are considered three parallel objectives that could be optimized simultaneously. This method has some advantages in high-noise conditions. Huang et al. [73] added joint sparse block and low rank said to hyperspectral solution mixing and put forward the joint-sparse-blocks and low-rank representation for hyperspectral unmixing.Using spatial information to improve spectral unmixing results has been a trend in recent years. These methods usually assume that the abundance of pixels is piecewise smooth. However, in reality, the abundance may vary dramatically depending on the pixel. Therefore, Zhang [74] proposed a new strategy to preserve the spatial details in the abundance maps via a spatial discontinuity weight. To make the structure clearer, Table 1 of this paper briefly introduces the pros and cons of some algorithms mentioned in this section.Table 1
The pros and cons of some sparse unmixing methods.
AuthorYearProsConsSigurdsson et al.2016This algorithm can be applied on very large hyperspectral image where a standard algorithm could not be used due to memory limitations. And each subproblem does not need to have access to the whole hyperspectral image.Endmembers in the scene are assumed to be the same with its corresponding sample in the spectral library; this assumption is too ideal in the real environment.Wang et al.2017To solve the existing spectral variability between the measured endmembers in the real environment and corresponding ones in the spectral library.The method only uses spectral information but ignores the spatial-contextual information.Sigurdsson et al.2017A novel sparse outlier component is used to remove outliers and structured noise from the unmixing algorithm.The algorithm is easy to make the estimated data appear over smooth.Ruyi et al.2017This paper adopts Bregman divergence for sparse unmixing, which is a differentiable, smoother prior. And based on the maximum a posterior estimation, the proposed method has achieved sparse, stable, and precise fractional abundances.Endmembers in the scene are assumed to be the same with its corresponding sample in the spectral library, and this assumption is too ideal in the real environment.Zhenwei et al.2018l0 norm was used to solvel 0 problem directly. The antinoise ability of this algorithm also gets improved.This algorithm is difficult to integrate into a larger learning architecture.Huang et al.2019This paper developed a two-level reweighting strategy to enhance the sparsity along the rows within each block and considered spatial correlation among nearby pixels.If this algorithm will be applied on very large hyperspectral image, the computation time will be too long.The sparse unmixing algorithm has obvious disadvantages. Before using the sparse unmixing algorithm based on spectral library, spectral libraries composed of different matter spectra must be obtained. However, for the influence of many factors on the natural features, there will be some differences between the spectral library and the actual features. There is a strong correlation between the spectral curves of the features in the spectral reservoir, so it is difficult to ensure the stability of the unmixing results. However, the sparse unmixing algorithm based on spectral library does not make full use of the spatial information of images.


## 7. Conclusion

In this paper, the linear spectral mixing model is reviewed, and it describes the related contents of the linear spectral mixing model from four aspects: geometric method, NMF, Bayesian, and sparse unmixing. The unmixing method based on NMF has a clear physical meaning, overcomes the defect of geometric method, and can deal with the case of high mixed pixels. However, the problem is that the objective function is nonconvex and easy to generate false endmembers. However, the method based on Bayesian theory can effectively integrate the variation and uncertainty of spectra, endmembers, and abundance into the unmixing model and improve the unmixing effect by adding reasonably prior information. The problem in the previously mentioned methods is that the unreliable endmembers extraction greatly reduces the accuracy of abundance estimation. To avoid the negative effects of unreliable endmembers extraction, sparse unmixing has attracted more and more attention in the industry. The problem is that the mobility of the established endmembers spectrum library is poor, which makes it impossible to avoid the error of the real environment when the endmember spectrum library is used to approximate the real spectral image.It reviews the latest relevant theories of domestic and foreign research achievements. Although many scholars have studied linear spectral unmixing, there are still some problems.(1)
The key to the success of hyperspectral sparse unmixing is to have an appropriate spectral library. However, the acquisition of spectral libraries is often a time-consuming and expensive process. Due to the complexity of the ground object and the change of the environment, there are certain differences between the actual matter and the matter in the spectral library. Therefore, the accuracy and stability of the results should be improved.(2)
Spectral unmixing based on matrix factorization is easy to fall into the local extremum solution. Increasing prior information can solve this problem and get an appropriate solution. However, it is difficult to get prior information, and adding too much prior information in the model will increase the computational complexity of the model. Most of the unmixing methods based on nonnegative matrix factorization do not fully use the spatial information and spectral information. Therefore, it is necessary to deeply explore and fully use all kinds of information features in hyperspectral remote sensing images, reduce the dependence on prior knowledge, reduce the difficulty of unmixing, and improve the accuracy of unmixing.(3)
Bayesian estimation theory encodes the available information by assigning the prior distribution to pure pixels or high-abundance pixels, but it still requires additional constraints and assumptions on pure pixels and high-abundance pixels to get unique solutions. A conclusion section is not required. Although a conclusion may review the main points of the paper, one should not replicate the abstract as the conclusion. A conclusion might elaborate on the importance of the work or suggest applications and extensions.



---

*Source: 3735403-2020-08-25.xml*