# Spectral-Spatial Hyperspectral Image Semisupervised Classification by Fusing Maximum Noise Fraction and Adaptive Random Multigraphs

Authors: Eryang Chen; Ruichun Chang; Kaibo Shi; Ansheng Ye; Fang Miao; Jianghong Yuan; Ke Guo; Youhua Wei; Yiping Li
Journal: Discrete Dynamics in Nature and Society
 (2021)

---

## Abstract

Hyperspectral images (HSIs) contain large amounts of spectral and spatial information, and this provides the possibility for ground object classification. However, when using the traditional method, achieving a satisfactory classification result is difficult because of the insufficient labeling of samples in the training set. In addition, parameter adjustment during HSI classification is time-consuming. This paper proposes a novel fusion method based on the maximum noise fraction (MNF) and adaptive random multigraphs for HSI classification. Considering the overall spectrum of the object and the correlation of adjacent bands, the MNF was utilized to reduce the spectral dimension. Next, a multiscale local binary pattern (LBP) analysis was performed on the MNF dimension-reduced data to extract the spatial features of different scales. The obtained multiscale spatial features were then stacked with the MNF dimension-reduced spectral features to form multiscale spectral-spatial features (SSFs), which were sent into the RMG for HSI classification. Optimal performance was obtained by fusion. For all three real datasets, our method achieved competitive results with only 10 training samples. More importantly, the classification parameters corresponding to different hyperspectral data can be automatically optimized using our method.


---

## Body

## 1. Introduction

Due to the advancements in remote sensing technology, hyperspectral images (HSIs) are containing increasing spectral and spatial information (SSI), resulting in their extensive use in domains, such as forest inventory [1], urban area monitoring [2], road extraction [3], geological surveys [4], precision agriculture [5], environmental protection [6], military applications [7], hydrocarbon detection [8], oil reservoir exploration [9], and lake sediment analysis [10]. HSI classification is a crucial research topic related to these applications. In contrast to those of Synthetic Aperture Radar (SAR) [11] or RGB images [12], the two main challenges associated with HSI classification are the high dimensionality of the dataset and the redundancy of spectral information.Many approaches for HSI dimension reduction have been proposed. The most frequently used methods are principal component analysis (PCA) [13, 14], independent component analysis (ICA) [15], maximum noise fraction (MNF) [16], linear discriminant analysis [17], and the deep learning approach [18]. Uddin et al. [13] proposed an information-theoretic normalized-based PCA for HSI classification. Fu et al. [14] proposed a segmented PCA-based algorithm for HSI classification.However, because the noise in hyperspectral data can easily mask subtle hyperspectral features, careful noise removal is required to extract useful information. This phenomenon is problematic for PCA, which maximizes the variance of the orthogonal set of the projection sample and vector. Unlike PCA, MNF aims to maximize the signal-to-noise ratio (SNR) rather than the variance. MNF is worth considering in HSI classification because removing the noise is effective during dimensionality reduction.HSI classification can be significantly improved by employing an appropriate object-based [19], spectral-spatial fusion-based approach [20], decision fusion-based method [21], or deep learning-based method [22]. However, two challenges are unavoidable for most researchers. On the one hand, the traditional depth learning-based approaches rely heavily on a large amount of labeled data to achieve competitive results, whereas HSI annotation is expensive and time-consuming because it requires expert knowledge and skills. On the other hand, many methods require the use of a large number of parameter settings during the experiment, which involves expert experience [23]. HSI classification has been widely studied, and various classification methods have been adopted, namely, support vector machine (SVM) [24], random forests [25], neural networks [26], low-rank representation [27], sparse representation [28], deep learning methods [29, 30], and meta-learning methods [31]. Xu et al. [29] directly used the random patches extracted from the HSI image as the convolution kernel, without any training, to improve the classification efficiency. Yin et al. [30] proposed a CapsNet-based alternative data-driven HSI classification model. The aforementioned results demonstrate that the proposed method can achieve ideal results when the training samples are sufficient.To integrate the spatial information, researchers have proposed many spectral-spatial feature extraction (FE) techniques, such as the extinction profile (EP) [32] and local binary patterns (LBP) [33]. In contrast with the EP features, LBP features facilitate the mining of the HSI texture information, such as global contrast information and texture depth [34]. In this study, we adopted the LBP features as spatial features.However, when the labeled samples in the training set are insufficient, the classification accuracy achieved by the traditional methods significantly reduces [35] because of the so-called Hughes effect or the curse of dimensionality [36]. Moreover, many methods require a series of manual parameter settings. For instance, to extract spatial features, researchers must select an appropriate window size for the neighborhood. The selection is time-consuming [37] and requires expertise [38]. Since 2010, ensemble learning methods for HSI classification have received significant attention because of their dependence on limited training samples [39]. Many methods have been developed, such as support vector machines [40], boosting [41], segmentation-based methods [19], unsupervised methods [42], and semisupervised methods [43]. However, the graph-based semisupervised method is rarely considered in HSI classification [44, 45].Motivated by the aforementioned discussions, by combining the SSI, we proposed a new spectral-spatial HSI semisupervised classifier based on MNF and adaptive random multigraphs (SS-MNF-ARMG). The primary contributions of this study are as follows:(1)
A novel spectral-spatial HSI semisupervised classification framework was developed. Because of the adaptive properties, the optimal parameters can be determined without artificial auxiliaries.(2)
By introducing the MNF, the noise in the HSI can be removed more efficiently during dimensionality reduction. On the basis of dimension-reduced HSI, the SSI is combined, which can degrade the curse of dimensionality.(3)
In contrast with several studies, SS-MNF-ARMG can achieve competitive performance for three real HSI datasets while leveraging tiny labeled samples, which is further improved by introducing RMG in a new mode.


## 2. Relevant Work

### 2.1. MNF

LetX be the HSI data, and S and N are the signal and noise part of X, respectively. The goal of MNF is to seek out a linear transformation matrix W to maximize the SNR of the transformed data. Assume that S and N are uncorrelated; then X can be represented as [43](1)X=S+N.And, the covariance matrix (CM) ofX can be obtained by(2)covX=∑X=∑S+∑N,where ∑S and ∑N are the CMs of S and N, respectively. The MNF transform can be expressed as(3)Y=WTX,where Y is the MNF result of X, W are the eigenvectors associated with the L largest eigenvalues of ∑N−1∑X, and L is the number of MNF principal components.Then the SNR of eachyi∈Y can be described as(4)covwiTScovwiTN=wiT∑SwiwiT∑Xwi,where cov⋅ computes the variance and wi is the ith component in W. Then we can obtain W by solving the following problem:(5)argmaxWWT∑SWWT∑NW=argmaxWWT∑XWWT∑NW−1.


### 2.2. LBP

For each pixel in HSI, aP-bit binary code and a new matrix with the new value (binary to decimal value) are generated by thresholding adjacent pixel values. The LBP operator can be calculated by(6)LBPi,rNc=∑i=0P−1gNi−Nc2i,where P is the number of neighbors represented on a circle with the radius r, and Nc and Ni, respectively, represent the gray level intensity values of the center and the ith neighbour. The binary threshold function gx is described as(7)gx=0,x<0,1,x≥0.Take the 10th band of Indian Pines HSI as an example; the procedure of LBP is shown in Figure1. As shown in Figure 1, for a given center pixel in a 3 × 3 window, binary labels (“0” or “1”) are assigned to adjacent pixels according to whether the gray value of the center pixel is large. Starting at the top left corner, all binary codes are joined clockwise to produce an 8-bit binary number. The resulting binary number is called LBP code. The results show that LBP algorithm has significant rotation invariance and gray invariance [46], and it can be effectively applied to HSI classification [47].Figure 1
Implementation of LBP FE.


### 2.3. RMG

Given the HSI datasetD=Xlab∪Xulab is comprised of l labeled data Xlab=xii=1l and u unlabeled data Xulab=xii=l+1l+u, then a weighted graph can be obtained. The vertices in the graph consist of Xlab and Xulab. Weighted edges, which can be defined as a matrix W=ℝl+u×l+u, represent the similarities between associated nodes. For a c-class classification problem, it can be defined as a quadratic optimization problem [43]:(8)minFtrF−YTCF−Y+trFTLF,where tr⋅is the trace function and F∈ℝl+u×c is the prediction matrix to be learned. The indicator vector yi∈Y=y1,y2,…,yl,0,…,0T∈ℝl+u×c is the label vector corresponding to xi and 0 is the 0 vector. In addition, we can obtain yij by the following equation:(9)yij=1,ifxibelongs to thejth class,0,otherwise.Then eachxi can be classified to the jth class if Fij is the largest one in the ith row of F which can be described as yi=argmaxiFij,j=1,...,c. C∈ℝl+u×l+u is a diagonal matrix, and its element ci can be calculated as(10)ci=αl,1≤i≤l,αl>0,αu,l+1≤i≤l+u,αu≥0,where αl and αu are two parameters.The popular choice forLis the graph Laplacian [48], which is defined as(11)L=D−W,where W is the weight matrix of the graph, which can be formulated by Gaussian kernel as(12)wij=e−xi−yi2/2σ2where σ is the kernel width parameter to be adjusted. And the diagonal matrix D is the row sum of W.However, it is difficult to discover the neighborhood structure inherent in the graph and learn the proper compact representation automatically. To solve this problem, researchers have proposed the anchor graph algorithm (AGA) [49] and multiview anchor graph algorithm [50]. The AGA extrapolates the Laplacian eigenvector of the graph to the eigenfunction, allowing constant time hashing of new data points. Then the hierarchical threshold learning method is used to make each feature function generate more than 1 bit to improve the search accuracy. And the label prediction function in the AGA can be expressed as(13)fxi=∑j=1mPijfaj,wherePijis the data-adaptive weight, and each aj in A=ajj=1m is an anchor point. By this formula, the solution space of the unknown labels can be reduced from a larger space to a smaller one. The centers of the K-means cluster are selected as anchor points because these centers have a powerful representation that covers the entire dataset. In this paper, we use Local Anchor Embedding (LAE) [49] to calculate the anchor points. Figure 2 shows the flowchart of RMG.Figure 2
Flowchart of the RMG.


## 2.1. MNF

LetX be the HSI data, and S and N are the signal and noise part of X, respectively. The goal of MNF is to seek out a linear transformation matrix W to maximize the SNR of the transformed data. Assume that S and N are uncorrelated; then X can be represented as [43](1)X=S+N.And, the covariance matrix (CM) ofX can be obtained by(2)covX=∑X=∑S+∑N,where ∑S and ∑N are the CMs of S and N, respectively. The MNF transform can be expressed as(3)Y=WTX,where Y is the MNF result of X, W are the eigenvectors associated with the L largest eigenvalues of ∑N−1∑X, and L is the number of MNF principal components.Then the SNR of eachyi∈Y can be described as(4)covwiTScovwiTN=wiT∑SwiwiT∑Xwi,where cov⋅ computes the variance and wi is the ith component in W. Then we can obtain W by solving the following problem:(5)argmaxWWT∑SWWT∑NW=argmaxWWT∑XWWT∑NW−1.


## 2.2. LBP

For each pixel in HSI, aP-bit binary code and a new matrix with the new value (binary to decimal value) are generated by thresholding adjacent pixel values. The LBP operator can be calculated by(6)LBPi,rNc=∑i=0P−1gNi−Nc2i,where P is the number of neighbors represented on a circle with the radius r, and Nc and Ni, respectively, represent the gray level intensity values of the center and the ith neighbour. The binary threshold function gx is described as(7)gx=0,x<0,1,x≥0.Take the 10th band of Indian Pines HSI as an example; the procedure of LBP is shown in Figure1. As shown in Figure 1, for a given center pixel in a 3 × 3 window, binary labels (“0” or “1”) are assigned to adjacent pixels according to whether the gray value of the center pixel is large. Starting at the top left corner, all binary codes are joined clockwise to produce an 8-bit binary number. The resulting binary number is called LBP code. The results show that LBP algorithm has significant rotation invariance and gray invariance [46], and it can be effectively applied to HSI classification [47].Figure 1
Implementation of LBP FE.


## 2.3. RMG

Given the HSI datasetD=Xlab∪Xulab is comprised of l labeled data Xlab=xii=1l and u unlabeled data Xulab=xii=l+1l+u, then a weighted graph can be obtained. The vertices in the graph consist of Xlab and Xulab. Weighted edges, which can be defined as a matrix W=ℝl+u×l+u, represent the similarities between associated nodes. For a c-class classification problem, it can be defined as a quadratic optimization problem [43]:(8)minFtrF−YTCF−Y+trFTLF,where tr⋅is the trace function and F∈ℝl+u×c is the prediction matrix to be learned. The indicator vector yi∈Y=y1,y2,…,yl,0,…,0T∈ℝl+u×c is the label vector corresponding to xi and 0 is the 0 vector. In addition, we can obtain yij by the following equation:(9)yij=1,ifxibelongs to thejth class,0,otherwise.Then eachxi can be classified to the jth class if Fij is the largest one in the ith row of F which can be described as yi=argmaxiFij,j=1,...,c. C∈ℝl+u×l+u is a diagonal matrix, and its element ci can be calculated as(10)ci=αl,1≤i≤l,αl>0,αu,l+1≤i≤l+u,αu≥0,where αl and αu are two parameters.The popular choice forLis the graph Laplacian [48], which is defined as(11)L=D−W,where W is the weight matrix of the graph, which can be formulated by Gaussian kernel as(12)wij=e−xi−yi2/2σ2where σ is the kernel width parameter to be adjusted. And the diagonal matrix D is the row sum of W.However, it is difficult to discover the neighborhood structure inherent in the graph and learn the proper compact representation automatically. To solve this problem, researchers have proposed the anchor graph algorithm (AGA) [49] and multiview anchor graph algorithm [50]. The AGA extrapolates the Laplacian eigenvector of the graph to the eigenfunction, allowing constant time hashing of new data points. Then the hierarchical threshold learning method is used to make each feature function generate more than 1 bit to improve the search accuracy. And the label prediction function in the AGA can be expressed as(13)fxi=∑j=1mPijfaj,wherePijis the data-adaptive weight, and each aj in A=ajj=1m is an anchor point. By this formula, the solution space of the unknown labels can be reduced from a larger space to a smaller one. The centers of the K-means cluster are selected as anchor points because these centers have a powerful representation that covers the entire dataset. In this paper, we use Local Anchor Embedding (LAE) [49] to calculate the anchor points. Figure 2 shows the flowchart of RMG.Figure 2
Flowchart of the RMG.


## 3. Proposed Method

The SS-MNF-ARMG framework is shown in Figure3. Because of their adaptive properties, the optimal parameters can be determined without artificial auxiliaries. The framework comprises three main modules:(1)
Preprocessing the HSI image by applying MNF, the noise in the HSI can be removed effectively during dimension reduction. This result avoids the problem of the dimension.(2)
Through LBP, spatial vectors corresponding to different neighborhood regions are obtained. These spatial vectors are stacked with the spectral vector, respectively, and a series of spectral-spatial stacked feature information can be obtained.(3)
For classification and decision fusion, a set of necessary parameters for the RMG is established and the spectral-spatial feature information is integrated into the RMG. Next, a set of classification results with different accuracies are obtained, based on which the optimal classification results are obtained through decision fusion. In addition, by injecting randomness into the graph in the RMG, overfitting due to the limited training sample can be avoided.Figure 3
Schematic of the SS-MNF-ARMG algorithm.The proposed SS-MNF-ARMG algorithm is summarized in Algorithm1.Algorithm 1: SS-MNF-ARMG.
Input: the original HSI set is D∈ℝx×y×h; the training set is tra_s; the test set is tes_s; the number of spectra after dimension reduction is bandNum=h1′,h2′,…,hk′; the patch size for FE is W=w1,w2,…,wt, wherewii=1tis odd and wi≥3;Output: the best classification result of all test samples;(1)
for each hi′∈bandNumdo(2)
Obtain the dimension-reduced HSIDi′∈ℝx×y×hi′ by using MNF;(3)
Extract the spectral vectorSpe_vec of Di′;(4)
for each wj∈Wdo(5)
Calculate the spatial vectorSpa_vecj by using LBP;(6)
Obtain the spectral-spatial vector by stacking theSpe_vec and the Spa_vecj;(7)
Obtain the bestOAi,j (overall accuracy) by voting, and the corresponding confusion matrix ConfMati,j;(8)
end for(9)
Obtain the bestOA′ from OAi,j by decision fusion, and the corresponding confusion matrix ConfMat′;(10)
end for


## 4. Experimental Results and Analysis

### 4.1. Experimental Datasets

Three hyperspectral datasets were employed to evaluate the performance of the SS-MNF-ARMG.Indian Pines: this scene was gathered by the AVIRIS sensor over the Indian Pines test site in northwestern Indiana and consists of 145 × 145 pixels and 224 spectral reflectance bands in the wavelength range 0.4 to 2.5μm. This scene, which includes 16 different ground truths, contains two-thirds of agriculture and one-third of forest or other natural perennial vegetation. The number of bands was reduced to 200 by removing the 24 water absorption bands.Pavia University: this scene was acquired by the ROSIS sensor during a flight campaign over Pavia, northern Italy. The number of spectral bands was 103 at Pavia University. It is a 610 × 340 pixel image containing nine different ground objects with a geometric resolution of 1.3 meters.Salinas: this scene was collected by the 224-band AVIRIS sensor over the agricultural area of Salinas Valley, California, and is characterized by high spatial resolution (3.7 m pixels). After discarding 20 water absorption bands, the size of this data image was 512 × 217, with 204 bands. Salinas ground truth contains 16 classes, including vegetables, bare soils, and vineyard fields.


### 4.2. Analysis of Experimental Parameters

Parameters can affect the classification accuracy, such as the number of spectral bandsbandNum, the patch size W, the number of sampling points nr, the number of graphs kg, and the number of features kf. The adaptive properties of the proposed SS-MNF-ARMG are such that most optimal parameters for W, nr, kg, and kf can be determined without artificial auxiliaries.Based on the existing result [44], we let W∈15,19,23, nr∈15,18,21, kg∈19,25,31, and kf∈2dimVspe_spa,4dimVspe_spa,8dimVspe_spa, where dimVspe_sparepresents the dimensionality of the spectral-spatial vector Vspe_spa.In this study, we set the number of bands from 3 to 35 to evaluate the impact on the three HSI datasets. We conducted the experiment five times, and the optimal experimental results are shown in Figure4.Figure 4
Influence of the spectral band numbers.It can be observed that, on the Indian Pines dataset and the Salinas scene dataset, the value of overall classification accuracy (OA) shows a trend of steady growth. In other words, the increase in the number of spectral bands number of training has a positive promotion on the classification performance. In particular, on the Pavia University dataset, there is a sharp growth when the number of bands is above 15. In general, the OA of the proposed method is above 78% for all three HSI datasets. Especially in the Salinas scene dataset, the OA of the method is not less than 92% even with a small number of spectral bands, demonstrating the robustness of our method.


### 4.3. Comparison and Analysis

The proposed SS-MNF-ARMG method was compared with several state-of-the-art spectral-spatial fusion methods, such as Pixon-based classifier [19], PCA-SPCA-2D-SSA [14], R-VCANet [20], RN-FSC [31], iCapsNet [30], RPNet [29], and MBFSDA [21]. A comparison of the above algorithms can be seen in Table 1.Table 1
A comparison of different methods for hyperspectral image classification.
GroupSubgroupMethodAdvantagesDifficultiesSegmentation-based methodsObject-based classificationPixon-based classifierThe noise pixels in the classification map were removed and the suitable land cover smoothing map was obtainedThe spectral characteristics of abnormal pixels are often similar to the background, resulting in the loss of information when the abnormal pixels are removedFeature fusionFeatures stackingPCA-SPCA-2D-SSACombined with appropriate spatial features, the classification efficiency of the algorithm will be higherComplex structureJoint spectral-spatial FER-VCANetThe use of high correlation among SSIComplex structureMeta-learning-based classifiersRN-FSCLess sensitive to the number of training samplesLimited generalization ability for large-scale hyperspectral datasetsDeep learning-based classifiersiCapsNetWell-initialized shallow layersComplex structureRPNetFE and classification are carried out under a unified frameworkInsufficient training samples will lead to overfittingDecision fusionDecision fusionMBFSDACombination of supplemental information and several advanced classifiersSelection of suitable feature extractorOur method(i) Self-adaptability of parameters(ii) Leveraging tiny labeled samplesComplex structureTo quantitatively compare the classification performance of the methods shown in Table1, we used the average classification accuracy (AA), overall classification accuracy (OA), and the kappa coefficient (kappa) to assess the classification performance. To demonstrate the superior performance of the SS-MNF-ARMG with a limited number of training samples, we randomly selected 10 samples from each class as training samples. Tables 2–4 show the ground truth classes and their respective training and testing numbers for the three HSI datasets. Tables 5–7 summarize the experimental results for the three HSI datasets, from which the following conclusions can be drawn:(1)
The results on the Indian Pines dataset show that almost all algorithms are effective. We can observe from Table5 that the proposed SS-MNF-ARMG achieves a 3.78–28.6% advantage over the other methods in OA. In addition, for the classes that other methods do not recognize accurately, SS-MNF-ARMG can obtain better results, such as objects 1#, 2#, 3#, 5#, 6#, and 7# in Table 5.(2)
The results on the Pavia University dataset demonstrate that our method has advantages over some state-of-the-art methods. As a decision fusion-based algorithm, the proposed SS-MNF-ARMG surpasses MBFSDA by 3.87% in OA. In addition, for the classes that other methods do not recognize accurately, SS-MNF-ARMG can obtain better results, such as objects 2#, 3#, 6#, and 8# in Table6.(3)
The results on the Salinas dataset show that all methods show close results in AA, but SS-MNF-ARMG has a competitive advantage in OA and kappa. From Table7, we can observe that, compared with R-VCANet, SS-MNF-ARMG obtains a 0.1% improvement in OA, a 3.13% improvement in AA, and a 7.31% improvement in kappa. Furthermore, the proposed SS-MNF-ARMG surpasses iCapsNet by 11.01% in OA. In addition, for the classes that other methods do not recognize accurately, SS-MNF-ARMG can obtain better results, such as objects 3#, 7#, 8#, 9#, 15#, and 16# in Table 7.(4)
In general, the decision fusion-based methods outperform the segmentation-based methods and feature fusion-based methods. First and foremost, the LBP features improve the performance of the SS-MNF-ARMG, and the application of MNF reduces the HSI data dimension, controlling the noise in the HSI data. In addition, the randomness in the SS-MNF-ARMG can be regarded as a regularization technique, which alleviates the phenomenon of overfitting. Finally, benefitting from the semisupervised deep learning framework, the classification accuracy is relatively satisfactory even with only a few labeled samples.Table 2
The ground truth classes and their respective training and testing numbers of the Indian Pines datasets (10 classes).
No.Name of classTrainingTesting1Corn-notill1014182Corn-mintill108203Grass-pasture104734Grass-trees107205Hay-windrowed104686Soybean-notill109627Soybean-mintill1024458Soybean-clean105839Woods10125510Bldg-Grass-Tree-Drives10376Table 3
The ground truth classes and their respective training and testing numbers of the Pavia University datasets.
No.Name of classTrainingTesting1Asphalt1066212Meadows10186393Gravel1020894Trees1030545Painted metal sheets1013356Bare Soil1050197Bitumen1013208Self-Blocking Bricks1036729Shadows10937Table 4
The ground truth classes and their respective training and testing numbers of the Salinas datasets.
No.Name of classTrainingTesting1Brocoli_green_weeds_11019992Brocoli_green_weeds_21037163Fallow1019664Fallow_rough_plow1013845Fallow_smooth1026686Stubble1039497Celery1035698Grapes_untrained10112619Soil_vineyard_develop10619310Corn_senesced_green_weeds10326811Lettuce_romaine_4weeks10105812Lettuce_romaine_5weeks10191713Lettuce_romaine_6weeks1090614Lettuce_romaine_7weeks10106015Vineyard_untrained10725816Vineyard_vertical_trellis101797Table 5
Classification results of different techniques for the Indian Pines dataset (10 classes).
No.Pixon-basedPCA-SPCA-2D-SSAR-VCANetRN-FSCiCapsNetRPNetMBFSDASS-MNF-ARMG141.4261.5554.9472.2964.7160.3270.5775.38233.9386.4974.0978.9795.5673.3881.0695.6371.4399.3283.5490.5982.0993.5681.09100.00423.1696.0594.7889.8299.0684.282.0693.2587.7399.3796.1399.7999.5996.7399.3999.79665.8187.9775.5586.6173.3586.4777.4893.53776.0986.2861.4389.9488.0181.4866.3792.43822.9697.0180.6498.2574.5960.173.9497.69995.4498.8588.1582.6198.8458.7390.897.631091.5893.5282.6394.8296.8495.7959.2194.82AA60.9690.6479.1988.3787.2779.0878.293.01OA62.4587.2786.2886.1685.8375.9476.4291.05Kappa56.785.372.0583.9783.6372.3272.9589.61Bold values represent the best results among these methods.Table 6
Classification results of different techniques for the Pavia University dataset (9 classes).
No.Pixon-basedPCA-SPCA-2D-SSAR-VCANetRN-FSCiCapsNetRPNetMBFSDASS-MNF-ARMG145.2094.8379.0987.2865.8483.5286.6870.83282.0081.9171.7684.3384.5554.1989.2296.78390.2385.4785.4590.4229.9170.7087.2896.24474.0587.9994.2878.0982.7091.6479.3466.42585.1397.5599.9899.56100.00100.0099.3396.51618.5571.5558.4463.2540.2265.3089.52100.00792.4898.7794.5252.0921.4092.4188.1295.49884.1779.9073.7084.8157.1974.9368.8895.11997.5796.3099.8295.940.0093.2482.8977.51AA74.3888.2584.1281.9466.6380.6685.7088.32OA69.6384.5083.2381.7553.5468.8186.4590.32Kappa57.7279.8667.9775.8456.0761.7882.3187.26Bold values represent the best results among these methods.Table 7
Classification results of different techniques for the Salinas dataset (16 classes).
No.Pixon-basedPCA-SPCA-2D-SSAR-VCANetRN-FSCiCapsNetRPNetMBFSDASS-MNF-ARMG199.9096.3096.9599.2693.4198.0196.9597.11299.3888.9499.0693.2167.1699.1997.6598.98396.2092.0997.3197.8786.4996.9199.60100.00491.6190.2399.4685.5090.3699.21100.0086.21586.8988.9497.3584.8190.3998.3696.3088.67688.7996.0899.8799.3599.0299.6099.1489.47797.3291.3695.2696.6595.0796.4293.5597.65887.2096.4464.0166.2482.1276.0178.3396.66999.8485.9797.7997.3498.6396.5897.81100.001046.3486.3881.3882.6693.1384.6694.2387.221152.9094.1793.1773.9660.0996.2597.3888.301230.1080.4996.4896.8494.7897.8799.2797.181389.7496.1798.7085.3949.5193.5698.6986.421484.2199.3090.3590.3976.7697.7675.7095.791594.7986.6856.9265.8549.7666.2680.3895.361692.0998.1790.7986.8996.8997.5799.89100.00AA83.5891.4390.9387.6477.2693.3994.0594.06OA87.1591.7393.5693.1282.7288.1690.9693.73Kappa85.6390.9085.7285.4474.7286.8389.9593.03Bold values represent the best results among these methods.Our experiments were performed using MATLAB 2018b on a computer with an IntelCore(TM) i5-4300M 2.60 GHz CPU, 16 GB memory, and a 64-bit Windows 7 system. For the three real HSI datasets, the duration to execute our algorithm was several minutes to several hours; notably, the duration to execute the other methods that we reviewed in this study was shorter than that of our method.


## 4.1. Experimental Datasets

Three hyperspectral datasets were employed to evaluate the performance of the SS-MNF-ARMG.Indian Pines: this scene was gathered by the AVIRIS sensor over the Indian Pines test site in northwestern Indiana and consists of 145 × 145 pixels and 224 spectral reflectance bands in the wavelength range 0.4 to 2.5μm. This scene, which includes 16 different ground truths, contains two-thirds of agriculture and one-third of forest or other natural perennial vegetation. The number of bands was reduced to 200 by removing the 24 water absorption bands.Pavia University: this scene was acquired by the ROSIS sensor during a flight campaign over Pavia, northern Italy. The number of spectral bands was 103 at Pavia University. It is a 610 × 340 pixel image containing nine different ground objects with a geometric resolution of 1.3 meters.Salinas: this scene was collected by the 224-band AVIRIS sensor over the agricultural area of Salinas Valley, California, and is characterized by high spatial resolution (3.7 m pixels). After discarding 20 water absorption bands, the size of this data image was 512 × 217, with 204 bands. Salinas ground truth contains 16 classes, including vegetables, bare soils, and vineyard fields.


## 4.2. Analysis of Experimental Parameters

Parameters can affect the classification accuracy, such as the number of spectral bandsbandNum, the patch size W, the number of sampling points nr, the number of graphs kg, and the number of features kf. The adaptive properties of the proposed SS-MNF-ARMG are such that most optimal parameters for W, nr, kg, and kf can be determined without artificial auxiliaries.Based on the existing result [44], we let W∈15,19,23, nr∈15,18,21, kg∈19,25,31, and kf∈2dimVspe_spa,4dimVspe_spa,8dimVspe_spa, where dimVspe_sparepresents the dimensionality of the spectral-spatial vector Vspe_spa.In this study, we set the number of bands from 3 to 35 to evaluate the impact on the three HSI datasets. We conducted the experiment five times, and the optimal experimental results are shown in Figure4.Figure 4
Influence of the spectral band numbers.It can be observed that, on the Indian Pines dataset and the Salinas scene dataset, the value of overall classification accuracy (OA) shows a trend of steady growth. In other words, the increase in the number of spectral bands number of training has a positive promotion on the classification performance. In particular, on the Pavia University dataset, there is a sharp growth when the number of bands is above 15. In general, the OA of the proposed method is above 78% for all three HSI datasets. Especially in the Salinas scene dataset, the OA of the method is not less than 92% even with a small number of spectral bands, demonstrating the robustness of our method.


## 4.3. Comparison and Analysis

The proposed SS-MNF-ARMG method was compared with several state-of-the-art spectral-spatial fusion methods, such as Pixon-based classifier [19], PCA-SPCA-2D-SSA [14], R-VCANet [20], RN-FSC [31], iCapsNet [30], RPNet [29], and MBFSDA [21]. A comparison of the above algorithms can be seen in Table 1.Table 1
A comparison of different methods for hyperspectral image classification.
GroupSubgroupMethodAdvantagesDifficultiesSegmentation-based methodsObject-based classificationPixon-based classifierThe noise pixels in the classification map were removed and the suitable land cover smoothing map was obtainedThe spectral characteristics of abnormal pixels are often similar to the background, resulting in the loss of information when the abnormal pixels are removedFeature fusionFeatures stackingPCA-SPCA-2D-SSACombined with appropriate spatial features, the classification efficiency of the algorithm will be higherComplex structureJoint spectral-spatial FER-VCANetThe use of high correlation among SSIComplex structureMeta-learning-based classifiersRN-FSCLess sensitive to the number of training samplesLimited generalization ability for large-scale hyperspectral datasetsDeep learning-based classifiersiCapsNetWell-initialized shallow layersComplex structureRPNetFE and classification are carried out under a unified frameworkInsufficient training samples will lead to overfittingDecision fusionDecision fusionMBFSDACombination of supplemental information and several advanced classifiersSelection of suitable feature extractorOur method(i) Self-adaptability of parameters(ii) Leveraging tiny labeled samplesComplex structureTo quantitatively compare the classification performance of the methods shown in Table1, we used the average classification accuracy (AA), overall classification accuracy (OA), and the kappa coefficient (kappa) to assess the classification performance. To demonstrate the superior performance of the SS-MNF-ARMG with a limited number of training samples, we randomly selected 10 samples from each class as training samples. Tables 2–4 show the ground truth classes and their respective training and testing numbers for the three HSI datasets. Tables 5–7 summarize the experimental results for the three HSI datasets, from which the following conclusions can be drawn:(1)
The results on the Indian Pines dataset show that almost all algorithms are effective. We can observe from Table5 that the proposed SS-MNF-ARMG achieves a 3.78–28.6% advantage over the other methods in OA. In addition, for the classes that other methods do not recognize accurately, SS-MNF-ARMG can obtain better results, such as objects 1#, 2#, 3#, 5#, 6#, and 7# in Table 5.(2)
The results on the Pavia University dataset demonstrate that our method has advantages over some state-of-the-art methods. As a decision fusion-based algorithm, the proposed SS-MNF-ARMG surpasses MBFSDA by 3.87% in OA. In addition, for the classes that other methods do not recognize accurately, SS-MNF-ARMG can obtain better results, such as objects 2#, 3#, 6#, and 8# in Table6.(3)
The results on the Salinas dataset show that all methods show close results in AA, but SS-MNF-ARMG has a competitive advantage in OA and kappa. From Table7, we can observe that, compared with R-VCANet, SS-MNF-ARMG obtains a 0.1% improvement in OA, a 3.13% improvement in AA, and a 7.31% improvement in kappa. Furthermore, the proposed SS-MNF-ARMG surpasses iCapsNet by 11.01% in OA. In addition, for the classes that other methods do not recognize accurately, SS-MNF-ARMG can obtain better results, such as objects 3#, 7#, 8#, 9#, 15#, and 16# in Table 7.(4)
In general, the decision fusion-based methods outperform the segmentation-based methods and feature fusion-based methods. First and foremost, the LBP features improve the performance of the SS-MNF-ARMG, and the application of MNF reduces the HSI data dimension, controlling the noise in the HSI data. In addition, the randomness in the SS-MNF-ARMG can be regarded as a regularization technique, which alleviates the phenomenon of overfitting. Finally, benefitting from the semisupervised deep learning framework, the classification accuracy is relatively satisfactory even with only a few labeled samples.Table 2
The ground truth classes and their respective training and testing numbers of the Indian Pines datasets (10 classes).
No.Name of classTrainingTesting1Corn-notill1014182Corn-mintill108203Grass-pasture104734Grass-trees107205Hay-windrowed104686Soybean-notill109627Soybean-mintill1024458Soybean-clean105839Woods10125510Bldg-Grass-Tree-Drives10376Table 3
The ground truth classes and their respective training and testing numbers of the Pavia University datasets.
No.Name of classTrainingTesting1Asphalt1066212Meadows10186393Gravel1020894Trees1030545Painted metal sheets1013356Bare Soil1050197Bitumen1013208Self-Blocking Bricks1036729Shadows10937Table 4
The ground truth classes and their respective training and testing numbers of the Salinas datasets.
No.Name of classTrainingTesting1Brocoli_green_weeds_11019992Brocoli_green_weeds_21037163Fallow1019664Fallow_rough_plow1013845Fallow_smooth1026686Stubble1039497Celery1035698Grapes_untrained10112619Soil_vineyard_develop10619310Corn_senesced_green_weeds10326811Lettuce_romaine_4weeks10105812Lettuce_romaine_5weeks10191713Lettuce_romaine_6weeks1090614Lettuce_romaine_7weeks10106015Vineyard_untrained10725816Vineyard_vertical_trellis101797Table 5
Classification results of different techniques for the Indian Pines dataset (10 classes).
No.Pixon-basedPCA-SPCA-2D-SSAR-VCANetRN-FSCiCapsNetRPNetMBFSDASS-MNF-ARMG141.4261.5554.9472.2964.7160.3270.5775.38233.9386.4974.0978.9795.5673.3881.0695.6371.4399.3283.5490.5982.0993.5681.09100.00423.1696.0594.7889.8299.0684.282.0693.2587.7399.3796.1399.7999.5996.7399.3999.79665.8187.9775.5586.6173.3586.4777.4893.53776.0986.2861.4389.9488.0181.4866.3792.43822.9697.0180.6498.2574.5960.173.9497.69995.4498.8588.1582.6198.8458.7390.897.631091.5893.5282.6394.8296.8495.7959.2194.82AA60.9690.6479.1988.3787.2779.0878.293.01OA62.4587.2786.2886.1685.8375.9476.4291.05Kappa56.785.372.0583.9783.6372.3272.9589.61Bold values represent the best results among these methods.Table 6
Classification results of different techniques for the Pavia University dataset (9 classes).
No.Pixon-basedPCA-SPCA-2D-SSAR-VCANetRN-FSCiCapsNetRPNetMBFSDASS-MNF-ARMG145.2094.8379.0987.2865.8483.5286.6870.83282.0081.9171.7684.3384.5554.1989.2296.78390.2385.4785.4590.4229.9170.7087.2896.24474.0587.9994.2878.0982.7091.6479.3466.42585.1397.5599.9899.56100.00100.0099.3396.51618.5571.5558.4463.2540.2265.3089.52100.00792.4898.7794.5252.0921.4092.4188.1295.49884.1779.9073.7084.8157.1974.9368.8895.11997.5796.3099.8295.940.0093.2482.8977.51AA74.3888.2584.1281.9466.6380.6685.7088.32OA69.6384.5083.2381.7553.5468.8186.4590.32Kappa57.7279.8667.9775.8456.0761.7882.3187.26Bold values represent the best results among these methods.Table 7
Classification results of different techniques for the Salinas dataset (16 classes).
No.Pixon-basedPCA-SPCA-2D-SSAR-VCANetRN-FSCiCapsNetRPNetMBFSDASS-MNF-ARMG199.9096.3096.9599.2693.4198.0196.9597.11299.3888.9499.0693.2167.1699.1997.6598.98396.2092.0997.3197.8786.4996.9199.60100.00491.6190.2399.4685.5090.3699.21100.0086.21586.8988.9497.3584.8190.3998.3696.3088.67688.7996.0899.8799.3599.0299.6099.1489.47797.3291.3695.2696.6595.0796.4293.5597.65887.2096.4464.0166.2482.1276.0178.3396.66999.8485.9797.7997.3498.6396.5897.81100.001046.3486.3881.3882.6693.1384.6694.2387.221152.9094.1793.1773.9660.0996.2597.3888.301230.1080.4996.4896.8494.7897.8799.2797.181389.7496.1798.7085.3949.5193.5698.6986.421484.2199.3090.3590.3976.7697.7675.7095.791594.7986.6856.9265.8549.7666.2680.3895.361692.0998.1790.7986.8996.8997.5799.89100.00AA83.5891.4390.9387.6477.2693.3994.0594.06OA87.1591.7393.5693.1282.7288.1690.9693.73Kappa85.6390.9085.7285.4474.7286.8389.9593.03Bold values represent the best results among these methods.Our experiments were performed using MATLAB 2018b on a computer with an IntelCore(TM) i5-4300M 2.60 GHz CPU, 16 GB memory, and a 64-bit Windows 7 system. For the three real HSI datasets, the duration to execute our algorithm was several minutes to several hours; notably, the duration to execute the other methods that we reviewed in this study was shorter than that of our method.


## 5. Conclusions and Further Research

In this study, we developed a novel decision fusion method for HSI data classification. In the proposed SS-MNF-ARMG, MNF and multiscale LBP were integrated to extract local SSFs. On the one hand, MNF helped reduce the dimension of the HSI, remove the noise in the HSI, and extract the spectral features from the HSI data. On the other hand, multiscale LBP was applied to the MNF dimension-reduced images to derive spatial features at different scales. These spatial features were further fused with the MNF spectral feature to form the SSFs. Compared with some state-of-the-art spectral-spatial classification methods, our experimental results have demonstrated that SS-MNF-ARMG can achieve higher classification accuracy with limited training samples. This method is effective for distinguishing different land cover types. In addition, a set of optimal parameters for different hyperspectral data can be obtained automatically.Although the SS-MNF-ARMG algorithm has provided promising results, the classification accuracies of various datasets remain different. Further research could attempt to further improve the generalization ability of our method. Due to human ecological destruction or natural disasters (e.g., earthquakes), the vegetation in some areas has significantly changed. Monitoring vegetation restoration in these areas is of substantial importance. Therefore, we plan to research the application of the HSI classification in vegetation restoration monitoring.



---

*Source: 9998185-2021-06-24.xml*