# Convolution Neural Network Based on Two-Dimensional Spectrum for Hyperspectral Image Classification

Authors: Hongmin Gao; Shuo Lin; Yao Yang; Chenming Li; Mingxiang Yang
Journal: Journal of Sensors
 (2018)

---

## Abstract

Inherent spectral characteristics of hyperspectral image (HSI) data are determined and need to be deeply mined. A convolution neural network (CNN) model of two-dimensional spectrum (2D spectrum) is proposed based on the advantages of deep learning to extract feature and classify HSI. First of all, the traditional data processing methods which use small area pixel block or one-dimensional spectral vector as input unit bring many heterogeneous noises. The 2D-spectrum image method is proposed to solve the problem and make full use of spectral value and spatial information. Furthermore, a batch normalization algorithm (BN) is introduced to address internal covariate shifts caused by changes in the distribution of input data and expedite the training of the network. Finally, Softmax loss models are proposed to induce competition among the outputs and improve the performance of the CNN model. The HSI datasets of experiments include Indian Pines, Salinas, Kennedy Space Center (KSC), and Botswana. Experimental results show that the overall accuracies of the 2D-spectrum CNN model can reach 98.26%, 97.28%, 96.22%, and 93.64%. These results are higher than the accuracies of other traditional methods described in this paper. The proposed model can achieve high target classification accuracy and efficiency.


---

## Body

## 1. Introduction

Hyperspectral images (HSIs) are typically composed of hundreds of spectral data channels in the same scene. HSIs can provide continuous data in space and spectrum through combined imaging and spectrum technology. Hyperspectral data are important in monitoring information of the Earth’s surface because the spectral information provided by the hyperspectral sensor increases the accuracy of the resolution of target materials and thus improves classification accuracy [1].At first, scholars mainly use artificial extraction of image features for object identification classification of remote sensing images using a local binary pattern, histogram of oriented gradient [2], and Gabor filter [3]. However, this method is ineffective in processing hyperspectral data with the increase in dimension. Thus, feature extraction and classifier are combined, thereby yielding a satisfactory classification effect. Methods for feature extraction include principal component analysis (PCA) [4], independent component analysis (ICA) [5], and linear discriminant analysis (LDA) [6] and robust PCA. The classifier went through a process from fuzzy K-nearest neighbor algorithm [7], naive Bayes with deep feature weighting [8], and logistic regression [9] to support vector machine (SVM) [10]. SVM improves classification performance by extending classification kernel [11]. However, these combinatorial methods demonstrate the following significant limitations. (1) Feature extraction uses linear transformations to extract potentially useful features from the input data. Hyperspectral data are essentially nonlinear considering a complex light-scattering mechanism [12]. (2) Most traditional classification methods only consider single-layer processing, which reduces the capability of feature learning, and are unsuitable for high-dimensional data.Neural networks (NNs) with multiple layers and hidden nodes are more suitable than shallow classifiers, such as SVM, in building an HSI data model [13]. The NNs, including multilayer perceptron [14] and radial basis function [15], have been studied for classifying remote sensing data. Researchers have proposed a semisupervised NN framework for large-scale HSI classification [16]. Various deep NNs (DNN) have been developed according to system architecture and activation functions; these networks include deep belief network (DBN) [17], deep Boltzmann machine [18], and AutoEncoder (AE) [19]. In 2014, a stacked AE (SAE) was used for HSI classification [20]. An improved AE based on sparse constraints was then proposed [21]. The DBN is another DNN model that was proposed in 2015 [22]. The depth model can extract robust features and is superior to other methods in terms of classification accuracy.Convolution NN (CNN) [23] uses local receptive fields in efficiently extracting spatial information and sharing weights to significantly reduce the number of parameters. CNNs are used to extract the spatial spectral features of hyperspectral images for classification [24], and their performance was better than that of traditional classifiers such as SVM. In addition, a method using a virtual sample enhanced to limited labeled samples was proposed in [25]. A previous study proposed the use of a greedy layer unsupervised pretraining to form a CNN model [26]. However, the application technology of CNN in hyperspectral classification remains imperfect, and several shortcomings, such as easy saturation of the training gradient, low classification accuracy, and poor model generalization, should be addressed.The spectral values of the HSIs in the third dimension are approximately continuous, and the curves of each feature possess a unique spectral plot that is different from those of other classes. In the traditional classification methods, one-dimensional spectral vectors are used as the final form of input data [27, 28] or neighboring pixels are used to form small regional pixel blocks as input data [29, 30]. Although the former simplifies the complexity of deep learning network training, it omits spatial dimension information of spectral values at the same time. The latter combines multiple pixels into one sample, which introduces heterogeneous noises and aggravates the problem of missing hyperspectral data.Compared with the traditional CNN methods, this study designs a 2D-spectrum CNN model as follows:(i)
Hyperspectral pixels have rich spectral information. The traditional data processing methods which use small area pixel block or one-dimensional spectral vector as input unit bring many heterogeneous noises. In this paper, we convert the spectral value vector to 2D-spectrum image, so that the optimization of all CNN model parameters (including the BN parameters) is based on the spectral values of the pixel points and spectral space information. The target of fully extracting spectral spatial information can be achieved while heterogeneous noises are also avoided. In addition, a multilevel BN algorithm is achieved for the first time, and the effect of network acceleration is obvious.
(ii)
A BN algorithm is introduced to reduce the vanishing gradient problem and dynamically accelerate the training speed of the DNN by reducing the scaling and initialization of the dependent parameters. A small area pixel block was selected as the input unit. Liu et al. [30] used the BN algorithm to the CNN for the HIS. However, the introducing heterogeneous noises and wasting scarce samples will weaken the BN algorithm’s role in network regularization and accelerated training.
(iii)
Softmax loss models are used instead of combining Softmax regression and multinomial logistic loss models; thus, the output of the last layer competes with one another to improve the classification accuracy. The experimental results show that the proposed CNN-based HSI classification model exhibits high accuracy and efficiency in the HSI dataset.


## 2. CNN-Based Classification Model

The researchers found that the human visual system can effectively solve the problem of classification, detection, and identification, with the rapid development of modern nervous systems. This development motivates researchers on biological visual systems to establish advanced data processing methods [31]. Cells in the cortex of a human visual system are only susceptible to small areas, and accepting cells in the field can exploit the local spatial correlation in the image.The CNN architecture uses two special methods, namely, local receptive field and shared weights. The activation value of each convolution neuron is calculated by multiplying the local input with weightW, which is shared in the entire input space (Figure 1). Neurons that belong to the same layer share the same weight. The use of specific architectures, such as local receptive field and shared weights, reduces the total number of training parameters and facilitates the development of an efficient training model.Figure 1
Local receptive field and shared weights.The complete CNN architecture consists of convolution and pooling layers. The convolution layer alternates with the pooling layer, thereby mimicking the properties of complex and simple cells in the mammalian visual cortex [32]. In the CNN, the input data are a matrix or tensor with a 3D spatial structure, where (H, W), (H′, W′), and (H″, W″) represent the size of the spatial dimension of input data, convolution kernel, and output data, respectively. The number of convolution kernel feature channels is represented by D, and D″ represents the 3D data.
(1)x∈ℝH×W×D,f∈ℝH′×W′×D×D″,y∈ℝH″×W″×D″,where x is the input data, f is the convolution filter, and y is the output data. The 1D signal x is convoluted by filter f to calculate signal y as follows:
(2)yi″j″d″=bd″+∑i′=1H′∑j′=1W′∑d′=1Dfi′j′d×xi″+i′−1,j″+j′−1,d′,d″,where bd″ is the neuron offset and fi′j′d is the convolution kernel matrix of the dth i′×j′.The pooling layer is typically obtained after the convolution layer. The most common pooling function is max pooling. This function calculates the maximum response of each feature channel in theH′×W′ region. The feature map becomes robust to the distortion of the data and achieves a high invariance through the pooling. The pooling layer can also decrease the size of the feature map, thereby reducing computational burden.
(3)yi″j″d″=max1≤i′≤H′,1≤j′≤W′xi″+i′−1,j″+j′−1,d.The traditional CNN processing flow is designed in accordance with the data structure characteristics of the HSI classification task (Figure2). The training data are obtained through forward propagation to determine the actual output and then compared with the real data tag competition. Stochastic gradient descent (SGD) is used to modify the synapses and parameters of the network structure, and several iterative trainings are conducted to form a network model. Test data are inputted into the network model. The output data obtained by feature extraction are matched with the actual data, and the result is classified by the competitive output model.Figure 2
Traditional CNN processing flow.The 3D data of the HSIs reach hundreds, and a numerical approximation is considered continuous. The curve of each pixel possesses a unique spectral plot that differs from those of other categories, and this plot is difficult to distinguish by the human eye. However, the CNN exhibits a better performance than several human visual aspects. Therefore, this study uses spectral labels to improve the classification performance of the CNN models on the HSIs.


## 3. Proposed CNN-Based HSI Classification Model

### 3.1. BN

The SGD is used to train NN in CNN training. This method is simple and effective, but the model parameters should be carefully adjusted. In particular, the learning rate and initialization parameters of the model are added in the optimization. This step significantly reduces the speed of the CNN training. The entire network should conform to the new data distribution when the distribution of the input data at the network layer changes, thereby resulting in the decreased training speed of the network and saturated gradient. If the nonlinear input data distribution is ensured to be stable, then the probability of nonlinear saturation problem will be minimal and can accelerate the training of the network. Therefore, the BN algorithm is proposed to eliminate this phenomenon and expedite the training of the network.In the BN algorithm, for each hidden layer of neurons, the input distribution, the value interval of which is closer to the limit saturation region through nonlinear function mapping, is forced back to the normal distribution of the comparison standard, with a mean of 0 and a variance of 1. Thus, the input value of the nonlinear transformation function falls into the region, which is sensitive to the input data, to avoid a gradient disappearance problem. The most mature technique in the early DNN normalization operation is whitening; however, whitening the input of each layer would result in excessive computational costs and computational time and not all differential. Thus, BN is used in two simplified ways.The first step indicates that zero mean and variance normalization for scalar features are used instead of whitening. Simultaneously, the input and output of the layer are normalized. The following formula is used for NNs withd-dimensional input x=x1,…,xd to normalize each dimension, where the predicted value and variance can be calculated from the corresponding batches.
(4)x⌢k=xk−Exkvarxk.The primitive activation valuex that corresponds to the neuron is converted by subtracting the corresponding batch mean Ex and dividing it by the variance Varx. If the normalization for input of NN layer is rather simple, then the characterization capability of the layer is reduced. Thus, the BN introduces a pair of parameters γk and βk for each activation value xk, which can scale and translate the normalized input as
(5)yk=γkx⌢k+βk.The two parameters are similar to the network parameters and are trained and modified in the same way. The characterization capability of the model is then restored. Whenγk=varxk,βk=Exk, it can obtain the original activation value based on the output and restore representation ability. That is, the network can restore the feature distribution to be learned by the original network.The second step denotes that NN training is based on the entire training dataset. Theoretically, the entire training dataset can be used to normalize the activation value. However, it cannot be applied to the SGD because of a large amount of calculation of the dataset. The BN introduces min-batch in the SGD and calculates the corresponding mean and variance by using the minimum batch data. If a minimum batch dataB exists, then the corresponding size is m. Only one of the activation values xk is considered because each dimension of the activation value is normalized, while the variable k is then ignored. Each B has m activation values. The BN algorithm can be obtained by Formula (5).
(6)Algorithmγ,β:x1…m⟶x⌢1…m⟶y1…m.The meanμB and variance σB2 of the minimum batch data are defined as follows:
(7)μB⟵1m∑i=1mxi,σB2⟵1m∑i=1mxi−μB2,in which x⌢ and y can be obtained after the normalization.
(8)x⌢i⟵xi−μBσB2−ε,yi⟵γx⌢i+β≡BNγ,βxi,where ε is a constant (tends to be 0) to ensure the stability of the calculation of variance.The data computation becomes large and complex in applying the BN algorithm to the CNN if each neuron is normalized in every layer. Thus, this algorithm is based on the idea of weight sharing, and the mean and variance of the activation value are obtained for the entire map. Inputx, output y, parameter W, and b are defined as
(9)x,y∈ℝH×W×K×T,W∈ℝK,b∈ℝK,where H, W, K, and T represent the length and width of the input and output data, number of characteristic channels, and min-batch size, respectively. We explicitly define the input and output arrays as 4D data to process the feature mappings by batch. The output characteristic map is expressed by the following formula:
(10)yijkt=Wkxijkt−μkσk2+ε+bk,μk=1HWT∑i=1H∑j=1W∑t=1Txijkt,σk2=1HWT∑i=1H∑j=1W∑t=1Txijkt−μk.The BN is performed to the activation value of each hidden layer of neurons, which can be regarded as an add-on operation layer. It is located after the activation valuex=wu+b obtained and before the nonlinear activation function, as depicted in Figure 3.Figure 3
Operation layer of BN.The proposed BN algorithm possesses the following characteristics after the theoretical analysis and experimental verification. (1) A large initial learning rate can be selected so as to improve training speed. (2) Dropout can be removed and L2 weight attenuation coefficient can be reduced. (3) Local response normalization (LRN) can be replaced. (4) The training data can be completely disrupted (i.e., a sample is not frequently selected in each batch of training).


### 3.2. Softmax Loss Models

The Softmax regression model is a spread of logistic regression model in solving multiclass problems. Following the data structure of Section4.1, it can obtain the following output of the regression function.
(11)yijk=exijk∑t=1Dexijt.This formula is not limited by the number of feature channels and is applied to all spatial positions in a convoluted manner to translate linear predictions into categorical probabilities. The linear prediction resultsxijk of the kth category serve as input to the regression model, and D probability values (likelihood), which represent the possibility of the data belonging to different categories, are obtained. The Softmax regression model can be regarded as the combination of the exponential form of activation function and normalized operators. The classification loss function ℓx,c is aimed at comparing the prediction x with the real class label c. The classification loss is defined as follows:
(12)ℓx,c=∑ijnωij1nℓxij:n,cij:n,where x∈ℝH×W×C×N, c∈1,…,CH×W×1×N, and N is the size of the 3D data. The 1D vector xij:n represents class C fraction, and cij:n represents a real class label. Logarithmic loss function or logarithmic likelihood loss function is commonly used in logistic regression, which is based on the maximum likelihood principle. Thus, vector x represents the posterior probability pk=xk,k=1,…,C of the different classes. The output of the loss function is the negative logarithmic probability of the real label.
(13)ℓx,c=−logxc⋅x≥0,∑kxk=1,where x is the output of the Softmax regression model and is numerically unstable. On the one hand, the score xc should compete with other scores xkk≠c to obtain a meaningful logarithmic loss. If it is not the case, then the minimization of Formula (13) can be achieved by maximizing all xk, but the real prediction effect is that xc is larger than xkk≠c. On the other hand, the Softmax regression model allows the score x to compete through the normalization factor. This study proposes the Softmax loss models, which combines the calculation module of the regression model and calculation module of logarithmic loss into a single one.
(14)ℓx,c=−logexc∑k=1Cexk=−xc+log∑k=1Cexk.The combined modules result in a stable value of the output fractionx. By combining the logarithmic loss with Softmax, the loss model automatically makes the score compete ℓbx,c≈0, when xc≫∑k≠cxk. Although this model is similar to the final output result of the logarithmic loss function, the experimental results show that the Softmax loss model has the following advantages. (1) The calculation steps are minimal, while the calculated amount is small. (2) Numerical gradients are relatively stable. (3) Competitive output can improve the classification accuracy.


### 3.3. Two-Dimensional Spectrum CNN Model

The hyperspectral data has a small total number of pixels, while a single pixel has rich spectral values. According to this, the 2D-spectrum method is proposed in the paper, which converts the pixel spectral vectors into two-dimensional spectral images as the input data of CNN. The convolution network can fully utilize the spatial position information between the different spectral values to improve the classification accuracy, as shown in Figure4. The network adopts multiple convolution and max pooling layers alternately. BN layers, which regulate the data distribution and accelerate network training, are insert among the one, four, and seven layers. The Softmax loss model is then used to control the output. The 2D-spectrum CNN structure is shown in Figure 5.Figure 4
Hyperspectral image spectral vector preprocessing. Compared with the one-dimensional curve, the 2D spectrum has more obvious distinguishing features between different categories.Figure 5
2D-spectrum CNN structure.


## 3.1. BN

The SGD is used to train NN in CNN training. This method is simple and effective, but the model parameters should be carefully adjusted. In particular, the learning rate and initialization parameters of the model are added in the optimization. This step significantly reduces the speed of the CNN training. The entire network should conform to the new data distribution when the distribution of the input data at the network layer changes, thereby resulting in the decreased training speed of the network and saturated gradient. If the nonlinear input data distribution is ensured to be stable, then the probability of nonlinear saturation problem will be minimal and can accelerate the training of the network. Therefore, the BN algorithm is proposed to eliminate this phenomenon and expedite the training of the network.In the BN algorithm, for each hidden layer of neurons, the input distribution, the value interval of which is closer to the limit saturation region through nonlinear function mapping, is forced back to the normal distribution of the comparison standard, with a mean of 0 and a variance of 1. Thus, the input value of the nonlinear transformation function falls into the region, which is sensitive to the input data, to avoid a gradient disappearance problem. The most mature technique in the early DNN normalization operation is whitening; however, whitening the input of each layer would result in excessive computational costs and computational time and not all differential. Thus, BN is used in two simplified ways.The first step indicates that zero mean and variance normalization for scalar features are used instead of whitening. Simultaneously, the input and output of the layer are normalized. The following formula is used for NNs withd-dimensional input x=x1,…,xd to normalize each dimension, where the predicted value and variance can be calculated from the corresponding batches.
(4)x⌢k=xk−Exkvarxk.The primitive activation valuex that corresponds to the neuron is converted by subtracting the corresponding batch mean Ex and dividing it by the variance Varx. If the normalization for input of NN layer is rather simple, then the characterization capability of the layer is reduced. Thus, the BN introduces a pair of parameters γk and βk for each activation value xk, which can scale and translate the normalized input as
(5)yk=γkx⌢k+βk.The two parameters are similar to the network parameters and are trained and modified in the same way. The characterization capability of the model is then restored. Whenγk=varxk,βk=Exk, it can obtain the original activation value based on the output and restore representation ability. That is, the network can restore the feature distribution to be learned by the original network.The second step denotes that NN training is based on the entire training dataset. Theoretically, the entire training dataset can be used to normalize the activation value. However, it cannot be applied to the SGD because of a large amount of calculation of the dataset. The BN introduces min-batch in the SGD and calculates the corresponding mean and variance by using the minimum batch data. If a minimum batch dataB exists, then the corresponding size is m. Only one of the activation values xk is considered because each dimension of the activation value is normalized, while the variable k is then ignored. Each B has m activation values. The BN algorithm can be obtained by Formula (5).
(6)Algorithmγ,β:x1…m⟶x⌢1…m⟶y1…m.The meanμB and variance σB2 of the minimum batch data are defined as follows:
(7)μB⟵1m∑i=1mxi,σB2⟵1m∑i=1mxi−μB2,in which x⌢ and y can be obtained after the normalization.
(8)x⌢i⟵xi−μBσB2−ε,yi⟵γx⌢i+β≡BNγ,βxi,where ε is a constant (tends to be 0) to ensure the stability of the calculation of variance.The data computation becomes large and complex in applying the BN algorithm to the CNN if each neuron is normalized in every layer. Thus, this algorithm is based on the idea of weight sharing, and the mean and variance of the activation value are obtained for the entire map. Inputx, output y, parameter W, and b are defined as
(9)x,y∈ℝH×W×K×T,W∈ℝK,b∈ℝK,where H, W, K, and T represent the length and width of the input and output data, number of characteristic channels, and min-batch size, respectively. We explicitly define the input and output arrays as 4D data to process the feature mappings by batch. The output characteristic map is expressed by the following formula:
(10)yijkt=Wkxijkt−μkσk2+ε+bk,μk=1HWT∑i=1H∑j=1W∑t=1Txijkt,σk2=1HWT∑i=1H∑j=1W∑t=1Txijkt−μk.The BN is performed to the activation value of each hidden layer of neurons, which can be regarded as an add-on operation layer. It is located after the activation valuex=wu+b obtained and before the nonlinear activation function, as depicted in Figure 3.Figure 3
Operation layer of BN.The proposed BN algorithm possesses the following characteristics after the theoretical analysis and experimental verification. (1) A large initial learning rate can be selected so as to improve training speed. (2) Dropout can be removed and L2 weight attenuation coefficient can be reduced. (3) Local response normalization (LRN) can be replaced. (4) The training data can be completely disrupted (i.e., a sample is not frequently selected in each batch of training).


## 3.2. Softmax Loss Models

The Softmax regression model is a spread of logistic regression model in solving multiclass problems. Following the data structure of Section4.1, it can obtain the following output of the regression function.
(11)yijk=exijk∑t=1Dexijt.This formula is not limited by the number of feature channels and is applied to all spatial positions in a convoluted manner to translate linear predictions into categorical probabilities. The linear prediction resultsxijk of the kth category serve as input to the regression model, and D probability values (likelihood), which represent the possibility of the data belonging to different categories, are obtained. The Softmax regression model can be regarded as the combination of the exponential form of activation function and normalized operators. The classification loss function ℓx,c is aimed at comparing the prediction x with the real class label c. The classification loss is defined as follows:
(12)ℓx,c=∑ijnωij1nℓxij:n,cij:n,where x∈ℝH×W×C×N, c∈1,…,CH×W×1×N, and N is the size of the 3D data. The 1D vector xij:n represents class C fraction, and cij:n represents a real class label. Logarithmic loss function or logarithmic likelihood loss function is commonly used in logistic regression, which is based on the maximum likelihood principle. Thus, vector x represents the posterior probability pk=xk,k=1,…,C of the different classes. The output of the loss function is the negative logarithmic probability of the real label.
(13)ℓx,c=−logxc⋅x≥0,∑kxk=1,where x is the output of the Softmax regression model and is numerically unstable. On the one hand, the score xc should compete with other scores xkk≠c to obtain a meaningful logarithmic loss. If it is not the case, then the minimization of Formula (13) can be achieved by maximizing all xk, but the real prediction effect is that xc is larger than xkk≠c. On the other hand, the Softmax regression model allows the score x to compete through the normalization factor. This study proposes the Softmax loss models, which combines the calculation module of the regression model and calculation module of logarithmic loss into a single one.
(14)ℓx,c=−logexc∑k=1Cexk=−xc+log∑k=1Cexk.The combined modules result in a stable value of the output fractionx. By combining the logarithmic loss with Softmax, the loss model automatically makes the score compete ℓbx,c≈0, when xc≫∑k≠cxk. Although this model is similar to the final output result of the logarithmic loss function, the experimental results show that the Softmax loss model has the following advantages. (1) The calculation steps are minimal, while the calculated amount is small. (2) Numerical gradients are relatively stable. (3) Competitive output can improve the classification accuracy.


## 3.3. Two-Dimensional Spectrum CNN Model

The hyperspectral data has a small total number of pixels, while a single pixel has rich spectral values. According to this, the 2D-spectrum method is proposed in the paper, which converts the pixel spectral vectors into two-dimensional spectral images as the input data of CNN. The convolution network can fully utilize the spatial position information between the different spectral values to improve the classification accuracy, as shown in Figure4. The network adopts multiple convolution and max pooling layers alternately. BN layers, which regulate the data distribution and accelerate network training, are insert among the one, four, and seven layers. The Softmax loss model is then used to control the output. The 2D-spectrum CNN structure is shown in Figure 5.Figure 4
Hyperspectral image spectral vector preprocessing. Compared with the one-dimensional curve, the 2D spectrum has more obvious distinguishing features between different categories.Figure 5
2D-spectrum CNN structure.


## 4. Experimental Results

In this chapter, we first describe the dataset used in the experiment. Furthermore, the BN algorithm and Softmax loss model are used to verify the HSI classification performance of the CNN model. Finally, the proposed method is compared with other similar methods to determine the advantages and disadvantages of the 2D-spectrum CNN model. The overall accuracy (OA) and Kappa coefficients are used as the performance measurements for each type of classification accuracy (percentage). The mathematical relationship between error and OA is(15)OA=1−error×100%.Each classification result is averaged over 10 runs to avoid any deviation caused by random sampling. All of the experiments are tested on a desktop with NVIDIA GeForce GTX 1070 8G GPU, 16 GB memory, and 64-bit Windows 7 OS using MATLAB 2014b.


### 4.1. Hyperspectral Datasets

In the experiment, the hyperspectral data of the network model include Indian Pines, Salinas, and Kennedy Space Center (KSC) datasets and are applied to the new dataset, Botswana. Table1 summarizes the detailed information of the four datasets.Table 1
Hyperspectral data set features.
Dataset
Size
Spectral bands (original)
Labels
Instrument
Indian Pines
145 × 145
200 (224)
16
AVIRIS
Salinas
512 × 217
204 (224)
16
AVIRIS
KSC
512 × 614
176 (224)
13
NASA AVIRIS
Botswana
1476 × 256
145 (242)
14
NASA EO-1Figures6–8 exhibit the characteristics of the four datasets. Tables 2–5 list the number of samples trained and tested for the corresponding dataset and are assigned according to (train) 1 : (test) 3. According to previous experiments, a considerably minimal training data will produce an underfitting phenomenon. The labels 1, 7, 9, and 16 of the Indian Pines datasets are insufficient. The above labels allocate the total amount of samples according to 3 (train) : 1 (test) to avoid underfitting and to ensure that the training sample is sufficient.Figure 6
(a) AVIRIS Infrared image, (b) Indian Pines ground-truth map, and (c) classification map.
(a)
(b)
(c)Figure 7
(a) RGB color image from random three bands, (b) Salinas ground-truth map, and (c) Salinas classification map.
(a)
(b)
(c)Figure 8
(a) KSC third band, (b) KSC ground-truth map, (c) Botswana fourth band, and (d) Botswana ground-truth map.Table 2
Number of labeled training samples and testing samples of Indian Pines.
Class number
Class name
Training samples
Testing samples
1
Alfalfa
35
11
2
Corn-no till
357
1071
3
Corn-min till
207
623
4
Corn
59
178
5
Grass-pasture
120
363
6
Grass-trees
182
548
7
Grass-pasture-mowed
21
7
8
Hay-windrowed
119
359
9
Oats
15
5
10
Soybean-no till
243
729
11
Soybean-min till
613
1842
12
Soybean-clean
148
445
13
Wheat
51
154
14
Woods
316
949
15
Buildings-grass-trees-drives
96
290
16
Stone-steel-towers
70
23
Total
2652
7597Table 3
Number of labeled training samples and testing samples of Salinas.
Class number
Class name
Training samples
Testing samples
1
Brocoli_green_weeds_1
503
1506
2
Brocoli_green_weeds_2
932
2794
3
Fallow
494
1482
4
Fallow_rough_plow
349
1045
5
Fallow_smooth
670
2008
6
Stubble
990
2969
7
Celery
895
2684
8
Grapes_untrained
2818
8453
9
Soil_vinyard_develop
1550
4653
10
Corn_senesced_green_weeds
820
2458
11
Lettuce_romaine_4wk
267
801
12
Lettuce_romaine_5wk
482
1445
13
Lettuce_romaine_6wk
229
687
14
Lettuce_romaine_7wk
268
802
15
Vinyard_untrained
1817
5451
16
Vinyard_vertical_trellis
452
1355
Total
13536
40593Table 4
Number of labeled training samples and testing samples of KSC.
Class number
Class name
Training samples
Testing samples
1
Scrub
190
571
2
Willow swamp
60
183
3
CP hammock
64
192
4
Slash pine
63
189
5
Oak/broadleaf
40
121
6
Hardwood
58
171
7
Swamp
27
78
8
Graminoid marsh
108
323
9
Spartina marsh
130
390
10
Cattail marsh
101
303
11
Salt marsh
105
314
12
Mud flats
126
377
13
Water
232
695
Total
1304
3907Table 5
Number of labeled training samples and testing samples of Botswana.
Class number
Class name
Training samples
Testing samples
1
Water
67
203
2
Hippo grass
25
76
3
Floodplain grasses 1
62
189
4
Floodplain grasses 2
53
162
5
Reeds
67
202
6
Riparian
67
202
7
Firescar
64
195
8
Island interior
50
153
9
Acacia woodlands
78
236
10
Acacia shrublands
62
186
11
Acacia grasslands
76
229
12
Short mopane
45
136
13
Mixed mopane
67
201
14
Exposed soils
23
72
Total
806
2442The learning rate of the CNN network model is [1e-03 5e-04 1e-05], the corresponding number of training iterations is 50, 30, and 20, and the number of samples per batch is 100.


### 4.2. BN Performance Verification

The BN operation layer is added in the CNN to solve the problem of modified distribution of the internal nodes caused by the change in the input data of the traditional CNN. The addition of this layer can solve the problem of training saturation and gradient disappearance, significantly increase the training speed, and improve the classification performance. According to Section4.1, the BN operator layer can replace the traditional dropout layer and the local response normalization layer (LRN) and achieve a more satisfactory result, as presented in Figure 9 and Table 6.Figure 9
BN performance verification curve:y-axis is error.
(a)
(b)
(c)
(d)Table 6
Performance comparison of dropout, LRN, and BN.
Dataset
Operating layer
Unit time of batch/s∗
Error
Kappa
Indian pines
Dropout
0.940
0.0860
0.905
LRN
0.898
0.0629
0.930
B N
0.742
0.0174
0.978
Salinas
Dropout
0.671
0.0392
0.945
LRN
0.360
0.0323
0.939
B N
0.342
0.0272
0.972
KSC
Dropout
0.115
0.0893
0.892
LRN
0.114
0.0592
0.922
B N
0.099
0.0378
0.956
Botswana
Dropout
0.362
0.1709
0.829
LRN
0.323
0.0970
0.886
B N
0.283
0.0207
0.963
∗Owing to the number of samples between the datasets and the specific parameters of the training, this entry only applies to the comparison within the same dataset.Table6 indicates that dropout and LRN are trained on the network to accelerate and prevent the overfitting phenomenon. However, the BN operation layer in training acceleration and classification accuracy are better than the two areas by data comparison, and the experimental results depict obvious differences. Figure 10 denotes the test results for error in Table 6.Figure 10
Classification accuracy of the different output models.


### 4.3. Softmax Loss Models

The CNN in this study uses the Softmax loss models in the output to replace the traditional Softmax regression and multinomial logistic loss models. Table7 compares the classification accuracy among the three models.Table 7
Classification accuracy of different outputs model (error).
Output model
Performance
Indian Pines
Salinas
KSC
Botswana
Multinomial logistic loss
OA
0.0390
0.0352
0.0412
0.0842
Kappa
0.952
0.948
0.952
0.914
Softmax loss
OA
0.0174
0.0272
0.0378
0.0207
Kappa
0.978
0.972
0.956
0.963Figure10 is a more intuitive histogram. The results of four hyperspectral data experiments show that the Softmax loss model has the highest classification accuracy, compared with the multinomial logistic loss model.


### 4.4. Comparison with Other Approaches

The proposed method in this study is compared with recent works on the HSI classification studies. The training accuracy evaluation indicators, namely, OA and Kappa coefficients, are important criteria for assessing network model classification performance. In Table8, the proposed 2D-spectrum CNN model is compared with other advanced deep learning methods. The comparative data is derived from the work of Hu et al. [27], and the CNN data based on PPFs is from the research of Li et al. [28]. The data of Band-Adaptive Spectral-Spatial (BASS) net architecture is from the paper of Santara et al. [29].Table 8
Comparison of CNNs with other algorithms in Indian Pines, Salinas, KSC, and Botswana.
Dataset
Performance
CNN (vector)
PPF (vector)
BASS (blocks)
2D-spectrum CNN
Indian Pines
OA
86.44
94.34
96.77
98.26
Kappa
0.856
0.922
0.955
0.978
Salinas
OA
89.28
94.80
95.36
97.28
Kappa
0.878
0.931
0.928
0.962
KSC
OA
88.38
93.18
94.53
96.22
Kappa
0.870
0.912
0.939
0.956
Botswana
OA
89.45
91.57
95.42
97.93
Kappa
0.885
0.908
0.939
0.963
Vector: one-dimensional spectral vectors. Blocks: small-area pixel division blocks.Figures11 and 12 display the ground truth, training set, test set, and classification maps of Indian Pines and Salinas. It can be seen from the comparison of the performance that the classification performance of the 2D-spectrum CNN model is superior to other methods in different hyperspectral data in terms of overall accuracy and Kappa coefficient.Figure 11
Indian Pines dataset. (a) The ground truth, (b) training samples, (c) test samples, and (d) classification maps.
(a)
(b)
(c)
(d)Figure 12
Salinas data set. (a) The ground truth, (b) training samples, (c) test samples, and (d) classification maps.
(a)
(b)
(c)
(d)


## 4.1. Hyperspectral Datasets

In the experiment, the hyperspectral data of the network model include Indian Pines, Salinas, and Kennedy Space Center (KSC) datasets and are applied to the new dataset, Botswana. Table1 summarizes the detailed information of the four datasets.Table 1
Hyperspectral data set features.
Dataset
Size
Spectral bands (original)
Labels
Instrument
Indian Pines
145 × 145
200 (224)
16
AVIRIS
Salinas
512 × 217
204 (224)
16
AVIRIS
KSC
512 × 614
176 (224)
13
NASA AVIRIS
Botswana
1476 × 256
145 (242)
14
NASA EO-1Figures6–8 exhibit the characteristics of the four datasets. Tables 2–5 list the number of samples trained and tested for the corresponding dataset and are assigned according to (train) 1 : (test) 3. According to previous experiments, a considerably minimal training data will produce an underfitting phenomenon. The labels 1, 7, 9, and 16 of the Indian Pines datasets are insufficient. The above labels allocate the total amount of samples according to 3 (train) : 1 (test) to avoid underfitting and to ensure that the training sample is sufficient.Figure 6
(a) AVIRIS Infrared image, (b) Indian Pines ground-truth map, and (c) classification map.
(a)
(b)
(c)Figure 7
(a) RGB color image from random three bands, (b) Salinas ground-truth map, and (c) Salinas classification map.
(a)
(b)
(c)Figure 8
(a) KSC third band, (b) KSC ground-truth map, (c) Botswana fourth band, and (d) Botswana ground-truth map.Table 2
Number of labeled training samples and testing samples of Indian Pines.
Class number
Class name
Training samples
Testing samples
1
Alfalfa
35
11
2
Corn-no till
357
1071
3
Corn-min till
207
623
4
Corn
59
178
5
Grass-pasture
120
363
6
Grass-trees
182
548
7
Grass-pasture-mowed
21
7
8
Hay-windrowed
119
359
9
Oats
15
5
10
Soybean-no till
243
729
11
Soybean-min till
613
1842
12
Soybean-clean
148
445
13
Wheat
51
154
14
Woods
316
949
15
Buildings-grass-trees-drives
96
290
16
Stone-steel-towers
70
23
Total
2652
7597Table 3
Number of labeled training samples and testing samples of Salinas.
Class number
Class name
Training samples
Testing samples
1
Brocoli_green_weeds_1
503
1506
2
Brocoli_green_weeds_2
932
2794
3
Fallow
494
1482
4
Fallow_rough_plow
349
1045
5
Fallow_smooth
670
2008
6
Stubble
990
2969
7
Celery
895
2684
8
Grapes_untrained
2818
8453
9
Soil_vinyard_develop
1550
4653
10
Corn_senesced_green_weeds
820
2458
11
Lettuce_romaine_4wk
267
801
12
Lettuce_romaine_5wk
482
1445
13
Lettuce_romaine_6wk
229
687
14
Lettuce_romaine_7wk
268
802
15
Vinyard_untrained
1817
5451
16
Vinyard_vertical_trellis
452
1355
Total
13536
40593Table 4
Number of labeled training samples and testing samples of KSC.
Class number
Class name
Training samples
Testing samples
1
Scrub
190
571
2
Willow swamp
60
183
3
CP hammock
64
192
4
Slash pine
63
189
5
Oak/broadleaf
40
121
6
Hardwood
58
171
7
Swamp
27
78
8
Graminoid marsh
108
323
9
Spartina marsh
130
390
10
Cattail marsh
101
303
11
Salt marsh
105
314
12
Mud flats
126
377
13
Water
232
695
Total
1304
3907Table 5
Number of labeled training samples and testing samples of Botswana.
Class number
Class name
Training samples
Testing samples
1
Water
67
203
2
Hippo grass
25
76
3
Floodplain grasses 1
62
189
4
Floodplain grasses 2
53
162
5
Reeds
67
202
6
Riparian
67
202
7
Firescar
64
195
8
Island interior
50
153
9
Acacia woodlands
78
236
10
Acacia shrublands
62
186
11
Acacia grasslands
76
229
12
Short mopane
45
136
13
Mixed mopane
67
201
14
Exposed soils
23
72
Total
806
2442The learning rate of the CNN network model is [1e-03 5e-04 1e-05], the corresponding number of training iterations is 50, 30, and 20, and the number of samples per batch is 100.


## 4.2. BN Performance Verification

The BN operation layer is added in the CNN to solve the problem of modified distribution of the internal nodes caused by the change in the input data of the traditional CNN. The addition of this layer can solve the problem of training saturation and gradient disappearance, significantly increase the training speed, and improve the classification performance. According to Section4.1, the BN operator layer can replace the traditional dropout layer and the local response normalization layer (LRN) and achieve a more satisfactory result, as presented in Figure 9 and Table 6.Figure 9
BN performance verification curve:y-axis is error.
(a)
(b)
(c)
(d)Table 6
Performance comparison of dropout, LRN, and BN.
Dataset
Operating layer
Unit time of batch/s∗
Error
Kappa
Indian pines
Dropout
0.940
0.0860
0.905
LRN
0.898
0.0629
0.930
B N
0.742
0.0174
0.978
Salinas
Dropout
0.671
0.0392
0.945
LRN
0.360
0.0323
0.939
B N
0.342
0.0272
0.972
KSC
Dropout
0.115
0.0893
0.892
LRN
0.114
0.0592
0.922
B N
0.099
0.0378
0.956
Botswana
Dropout
0.362
0.1709
0.829
LRN
0.323
0.0970
0.886
B N
0.283
0.0207
0.963
∗Owing to the number of samples between the datasets and the specific parameters of the training, this entry only applies to the comparison within the same dataset.Table6 indicates that dropout and LRN are trained on the network to accelerate and prevent the overfitting phenomenon. However, the BN operation layer in training acceleration and classification accuracy are better than the two areas by data comparison, and the experimental results depict obvious differences. Figure 10 denotes the test results for error in Table 6.Figure 10
Classification accuracy of the different output models.


## 4.3. Softmax Loss Models

The CNN in this study uses the Softmax loss models in the output to replace the traditional Softmax regression and multinomial logistic loss models. Table7 compares the classification accuracy among the three models.Table 7
Classification accuracy of different outputs model (error).
Output model
Performance
Indian Pines
Salinas
KSC
Botswana
Multinomial logistic loss
OA
0.0390
0.0352
0.0412
0.0842
Kappa
0.952
0.948
0.952
0.914
Softmax loss
OA
0.0174
0.0272
0.0378
0.0207
Kappa
0.978
0.972
0.956
0.963Figure10 is a more intuitive histogram. The results of four hyperspectral data experiments show that the Softmax loss model has the highest classification accuracy, compared with the multinomial logistic loss model.


## 4.4. Comparison with Other Approaches

The proposed method in this study is compared with recent works on the HSI classification studies. The training accuracy evaluation indicators, namely, OA and Kappa coefficients, are important criteria for assessing network model classification performance. In Table8, the proposed 2D-spectrum CNN model is compared with other advanced deep learning methods. The comparative data is derived from the work of Hu et al. [27], and the CNN data based on PPFs is from the research of Li et al. [28]. The data of Band-Adaptive Spectral-Spatial (BASS) net architecture is from the paper of Santara et al. [29].Table 8
Comparison of CNNs with other algorithms in Indian Pines, Salinas, KSC, and Botswana.
Dataset
Performance
CNN (vector)
PPF (vector)
BASS (blocks)
2D-spectrum CNN
Indian Pines
OA
86.44
94.34
96.77
98.26
Kappa
0.856
0.922
0.955
0.978
Salinas
OA
89.28
94.80
95.36
97.28
Kappa
0.878
0.931
0.928
0.962
KSC
OA
88.38
93.18
94.53
96.22
Kappa
0.870
0.912
0.939
0.956
Botswana
OA
89.45
91.57
95.42
97.93
Kappa
0.885
0.908
0.939
0.963
Vector: one-dimensional spectral vectors. Blocks: small-area pixel division blocks.Figures11 and 12 display the ground truth, training set, test set, and classification maps of Indian Pines and Salinas. It can be seen from the comparison of the performance that the classification performance of the 2D-spectrum CNN model is superior to other methods in different hyperspectral data in terms of overall accuracy and Kappa coefficient.Figure 11
Indian Pines dataset. (a) The ground truth, (b) training samples, (c) test samples, and (d) classification maps.
(a)
(b)
(c)
(d)Figure 12
Salinas data set. (a) The ground truth, (b) training samples, (c) test samples, and (d) classification maps.
(a)
(b)
(c)
(d)


## 5. Conclusion

This paper proposes the 2D-spectrum CNN model that adds multilevel BN operating layers for HSI classification. The output uses a Softmax loss model for classification. In addition, many of the relevant factors include the number of iterations, learning rate, size of the input data, and size of the filter and feature graph. All of them will affect the final classification performance. The parameters obtained the optimal solution and reached the ideal effect in the experiments. The experiment results show that the CNN model proposed in this paper provides excellent performance. The training process converges quickly, thereby indicating that this method can be applied to large datasets. Furthermore, high classification accuracy can be achieved by applying sufficient iterations and number of datasets.



---

*Source: 8602103-2018-08-28.xml*