# Material Discrimination Algorithm Based on Hyperspectral Image

Authors: Jian Zhou; Zhuping Wang; Yingjie Jiao; Cong Nie
Journal: Scientific Programming
 (2021)

---

## Abstract

Hyperspectral information can be used to express the material properties of objects, which has a strong effect on camouflage recognition. However, it is difficult to process it directly because of the huge hyperspectral image data. Therefore, this paper proposes a new band selection algorithm to achieve band selection by simulating visual perception. The subspace clustering self-attention adversarial network is constructed to realize the initial selection of band. According to the visual chromatic aberration principle, a model is constructed to determine the band that combines the strongest response intensity of a particular material, and then this band is selected as the final band, therefore realizing the algorithm of material demarcation in this way.


---

## Body

## 1. Introduction

Visible near-infrared band images are obtained by sensors through detecting the electromagnetic radiation reflection of objects. It can precisely characterize ground objects so that each object has a spectral fingerprint which is of great significance to the identification of object materials [1, 2]. However, a hyperspectral image has high spectral dimension and spatial resolution, so it is difficult to process it directly because of a large amount of data [3, 4]. Thus, more in-depth studies have been carried out: in 2010, Yang et al. [5] used a supervised way to select band signals; Di et al. [6] applied a band selection to human face recognition and achieved good results. In 2010, Li and Qian [7] constructed a sparse matrix to analyze different bands; Samadzadegan and Mahmoudi [8] constructed the swarm intelligence to optimize band selection strategy. In 2012, Du et al. [9] established a collaborative sparse model to select hyperspectral bands; Hedjam and Cheriet [10] realized a band selection based on graph clustering. In 2013, Feng et.al [11] realized the band selection based on trivariate mutual information and clonal selection. Nakamura et al. [12] proposed a nature-inspired framework for band selection. In 2014, Su et al. [13] used the particle swarm optimization to optimize the band selection process; Xiurui Geng et al. [14] realized a band selection through gradient analysis of different band images. In 2015, Jia et al. [15] proposed a band selection scheme based on the idea of sorting; Patra et al. [16] introduced the idea of rough set to select bands. In 2016, Feng et al. [17] utilized the multiple kernel learning based on discriminative kernel clustering for hyperspectral band selection; Liu et al. [18] proposed a band selection algorithm based on the distribution of adjacent pixels. In 2017, Cao et al. [19] improved a classification map algorithm for fast hyperspectral selection; Shah et al. [20] proposed an algorithm of the dynamic frequency domain to realize band selection. In 2018, Wang et al. [21] proposed the optimal clustering framework to achieve hyperspectral band selection; Xie et al. [22] made modeling and analysis according to the representativeness of the bands. In 2019, Sun et al. [23] used a weighted kernel regulation to realize band selection. Sun et al. [24] calculated the variance between spectral bands and built a model for band selection. In 2020, Torres et al. [25] applied a band selection into the field of signal enhancement. Sun et al. [26] used the idea of low rank to cluster hyperspectral bands. Patra and Barman [27] focused on the image boundary intensity to realize band selection based on the fuzzy set.To sum up, main problems of hyperspectral band selection are as follows. (1) It is difficult to establish a unified band selection model due to high dimensions of hyperspectral data. (2) The quality of band selection cannot directly show its effect. Therefore, according to the above problems, (1) a hyperspectral band selection algorithm is constructed based on vision and (2) a subspace clustering framework is proposed based on deep adversary for realizing the preliminary clustering of spectral information. A perception model based on color is proposed to visualize the difference between the target and the background to show the perception effect.


## 2. A Visual Perception Algorithm

More than 80% information is obtained by human vision. One object can be recognized and distinguished from the background mainly by the color. At present, the captured natural images can be regarded as the superposition of RGB three channels. On this basis, a large number of research studies on target extraction, image retrieval, and analysis have been carried out, and a series of achievements have been achieved. The natural images can be regarded as hyperspectral data with low number of channels. Therefore, we migrate the related algorithms of RGB images to the hyperspectral field, and the process is shown in Figure1. (1) The subspace clustering network of deep countermeasure is constructed to realize the initial band selection. (2) According to the color difference of vision principle, a model is constructed to determine the band combination with the strongest response intensity of a specific material, and then the band is selected as the final band.Figure 1
Flow chart of the band selection algorithm.


### 2.1. Subspace Clustering Based on Self-Attention Adversarial

When high-dimensional data are encoded to output low-dimensional feature representation, a large amount of information will be lost. However, the attention model, which is based on encoder-decoder framework, can lose less information.(1)AttentionQ,K,V=sQ,KTV,where S⋅ is the similarity function and Q is the output information. The self-attention adversarial model we built is structured as Figure 2.Figure 2
Self-attention adversarial network.For true sample acquisition,k groups of A = {A1, A2, …, Ak} are obtained through the similarity matrix learned by word expression layer. the projection residual from Ai to the corresponding subspace Si is calculated as follows:(2)LrZi=Zi−ViViTZi22,where Z is the characteristic matrix, Vi is the projection matrix, T represents matrix transpose, and LR represents the projection residual. m data with small residuals are selected as positive samples. The corresponding generator resistance loss function is as follows:(3)Lg=1km∑i=1k∑j=1mLrZ¯j.For false samples, the sampling layer randomly samples from the estimated subspaceSi to generate m false samples Z¯j=∑θjZj. In order to make the generated data closer to the learning subspace of discriminator, the antiloss is introduced to revise the existing loss function:(4)LGC=Lg+λ2X−X^2+α2U−UC2+β2Cp,where λ is the balanced sparse.A discriminator is constructed by projection residuals to distinguish true and false samples, and the probability loss function of samples belonging to subspace is established.(5)Ld=1m∑j=1mLrZj+max0,ε−LrZj,where ε is the parameter. The loss function of the discriminator corresponding to k-rent is as follows:(6)Ld=minV1,V2,…,Vk1k∑i=1kLd.The second term of the following formula is introduced to increase the separation of different groups of subspaces after introducing regular terms. The third term of the following formula is to reduceVi:(7)LE=Ld+μ1∑i≠jViTVjF2+μ2∑i=1kViTVi−IF2,where μ1 and μ2 are two constants greater than 0.In order to make better use of the local manifold structure information of the image, Laplacian regularization term is introduced into the loss function of the generator to construct the image connection relationship. The weight can be expressed as follows:(8)Wij=expxi−xj2σ2,xi,xj∈Nk,0,others,where Nk is the K neighborhood of n vertices. For the nonlinear manifold structure, the energy function is defined as follows:(9)Q=minq∑i,j=1nqxi−qxj2Wij.According to the definition of the Laplace matrix,(10)L=D−Ws.t.Dii=∑jWij.Q can be rewritten as(11)Q=minqtrQ¯LQ¯T,Q¯=qx1,qx2,…,qxn.The final generator loss function is obtained as follows:(12)LGC=Lg+12λX−X^2+αU−UC2+βCp+γtrCLCT.


### 2.2. Perception Model Based on Color

According to Gestalt psychocognitive analysis, objects can be recognized, mainly by the eyes and brain. When the eyes observe the images, they cluster themselves according to certain rules to make them become a comprehensible structural entity. Among them, color feature is an effective way.On the basis of the previous analysis in the last section, in order to more comprehensively express the spectral information between bands, a model is established on the basis of relative entropy:(13)SX,Y=DKLP|Q−λMtX,Y,DKLP|Q=∑PxlnPxQx,Px,Qx∈0,1,Mt=mX1,X1…mX1,XM…mXM,X1…mXM,XM,mX,Y=∑i=1n∑j=1mpxiyjlog2pxi/yjpxi,where S (X, Y) is the spectral correlation between band X and band Y; DKL (P|Q) is the relative entropy of probability distribution corresponding to band X and band Y; and Mt (X, Y) is the average mutual information between band X and band y. λ is the weight coefficient, which is determined by the relative amplitude of DKL (P|Q) and Mt (X, Y).In order to better integrate the entropy information into it, we modifyS (X, Y). Because the entropy divergence and the average mutual information are basically equally important to the band selection process, in order to make the contribution of the corresponding matrix MKL and Mt consistent to S, the normalization function is constructed.(14)SX,Y=C×MKL−Mt,MKL=0…DKLX1|XM…DKLXM|X1…0,C=∑i=1M∑j=1MMi,jt∑i=1M∑j=1MMKLi,j.Different pixel values may correspond to the same color name. For this purpose, we construct the mapping relation; given the dataD = {d1, …, dN}, the corresponding word is W=w1,…,wM; these words are considered from the potential theme Z = {z1, …, zK}. Therefore, a probability model is constructed as follows:(15)pw|d=∑z∈Zpw|zpz|d,where p (w|z) and p (z|d) are prior probabilities. The EM algorithm is used to estimate the maximum similarity as follows:(16)L=∑d∈D∑w∈Wnd,wlogpdpw|d,where n (d, w) is the frequency of occurrence. Through training, we get the following results:(17)pz|w,d⟵θdzϕzw,ϕzw⟵∑dnd,wpz|w,d,θdz⟵α−1+∑wnd,wpz|w,d,where α is a parameter. The corresponding maximum similarity can be written as follows:(18)L=∑d∈D∑w∈Wnd,wlogd,w−γ∑z∈Z∑w∈Wpz|w−ρw2.Based on this, the color mapping is realized, and the colors are sorted according to Figure3 to construct the differentiation degree.Figure 3
Mapping color sequence graph.


## 2.1. Subspace Clustering Based on Self-Attention Adversarial

When high-dimensional data are encoded to output low-dimensional feature representation, a large amount of information will be lost. However, the attention model, which is based on encoder-decoder framework, can lose less information.(1)AttentionQ,K,V=sQ,KTV,where S⋅ is the similarity function and Q is the output information. The self-attention adversarial model we built is structured as Figure 2.Figure 2
Self-attention adversarial network.For true sample acquisition,k groups of A = {A1, A2, …, Ak} are obtained through the similarity matrix learned by word expression layer. the projection residual from Ai to the corresponding subspace Si is calculated as follows:(2)LrZi=Zi−ViViTZi22,where Z is the characteristic matrix, Vi is the projection matrix, T represents matrix transpose, and LR represents the projection residual. m data with small residuals are selected as positive samples. The corresponding generator resistance loss function is as follows:(3)Lg=1km∑i=1k∑j=1mLrZ¯j.For false samples, the sampling layer randomly samples from the estimated subspaceSi to generate m false samples Z¯j=∑θjZj. In order to make the generated data closer to the learning subspace of discriminator, the antiloss is introduced to revise the existing loss function:(4)LGC=Lg+λ2X−X^2+α2U−UC2+β2Cp,where λ is the balanced sparse.A discriminator is constructed by projection residuals to distinguish true and false samples, and the probability loss function of samples belonging to subspace is established.(5)Ld=1m∑j=1mLrZj+max0,ε−LrZj,where ε is the parameter. The loss function of the discriminator corresponding to k-rent is as follows:(6)Ld=minV1,V2,…,Vk1k∑i=1kLd.The second term of the following formula is introduced to increase the separation of different groups of subspaces after introducing regular terms. The third term of the following formula is to reduceVi:(7)LE=Ld+μ1∑i≠jViTVjF2+μ2∑i=1kViTVi−IF2,where μ1 and μ2 are two constants greater than 0.In order to make better use of the local manifold structure information of the image, Laplacian regularization term is introduced into the loss function of the generator to construct the image connection relationship. The weight can be expressed as follows:(8)Wij=expxi−xj2σ2,xi,xj∈Nk,0,others,where Nk is the K neighborhood of n vertices. For the nonlinear manifold structure, the energy function is defined as follows:(9)Q=minq∑i,j=1nqxi−qxj2Wij.According to the definition of the Laplace matrix,(10)L=D−Ws.t.Dii=∑jWij.Q can be rewritten as(11)Q=minqtrQ¯LQ¯T,Q¯=qx1,qx2,…,qxn.The final generator loss function is obtained as follows:(12)LGC=Lg+12λX−X^2+αU−UC2+βCp+γtrCLCT.


## 2.2. Perception Model Based on Color

According to Gestalt psychocognitive analysis, objects can be recognized, mainly by the eyes and brain. When the eyes observe the images, they cluster themselves according to certain rules to make them become a comprehensible structural entity. Among them, color feature is an effective way.On the basis of the previous analysis in the last section, in order to more comprehensively express the spectral information between bands, a model is established on the basis of relative entropy:(13)SX,Y=DKLP|Q−λMtX,Y,DKLP|Q=∑PxlnPxQx,Px,Qx∈0,1,Mt=mX1,X1…mX1,XM…mXM,X1…mXM,XM,mX,Y=∑i=1n∑j=1mpxiyjlog2pxi/yjpxi,where S (X, Y) is the spectral correlation between band X and band Y; DKL (P|Q) is the relative entropy of probability distribution corresponding to band X and band Y; and Mt (X, Y) is the average mutual information between band X and band y. λ is the weight coefficient, which is determined by the relative amplitude of DKL (P|Q) and Mt (X, Y).In order to better integrate the entropy information into it, we modifyS (X, Y). Because the entropy divergence and the average mutual information are basically equally important to the band selection process, in order to make the contribution of the corresponding matrix MKL and Mt consistent to S, the normalization function is constructed.(14)SX,Y=C×MKL−Mt,MKL=0…DKLX1|XM…DKLXM|X1…0,C=∑i=1M∑j=1MMi,jt∑i=1M∑j=1MMKLi,j.Different pixel values may correspond to the same color name. For this purpose, we construct the mapping relation; given the dataD = {d1, …, dN}, the corresponding word is W=w1,…,wM; these words are considered from the potential theme Z = {z1, …, zK}. Therefore, a probability model is constructed as follows:(15)pw|d=∑z∈Zpw|zpz|d,where p (w|z) and p (z|d) are prior probabilities. The EM algorithm is used to estimate the maximum similarity as follows:(16)L=∑d∈D∑w∈Wnd,wlogpdpw|d,where n (d, w) is the frequency of occurrence. Through training, we get the following results:(17)pz|w,d⟵θdzϕzw,ϕzw⟵∑dnd,wpz|w,d,θdz⟵α−1+∑wnd,wpz|w,d,where α is a parameter. The corresponding maximum similarity can be written as follows:(18)L=∑d∈D∑w∈Wnd,wlogd,w−γ∑z∈Z∑w∈Wpz|w−ρw2.Based on this, the color mapping is realized, and the colors are sorted according to Figure3 to construct the differentiation degree.Figure 3
Mapping color sequence graph.


## 3. Experiment Result and Analysis

The experiment is composed of visible infrared hyperspectral data and software simulation data [28], as shown in Figure 4, including grassland, sand, vehicles, buildings, and other typical targets. It can be seen from the figure that the pixel values displayed by different ground objects in different bands are different, and there are also differences in the pixel values of ground objects in the same band, which is the basis of band selection. We normalized the hyperspectral images to 512 × 512 × 300.Figure 4
Data display: (a) simulated overlook image sequence; (b) simulated sideview image sequence; (c) simulated overlook image sequence; (d) simulated sideview image sequence; (e) real image sequence.
(a)(b)(c)(d)(e)


### 3.1. Display of Spectral Curves of Typical Ground Objects

In order to show the spectral curves of typical ground objects, the spectral curves of leaves and sky are selected to display, as shown in Figure5. The horizontal axis represents the band number and the vertical axis represents the pixel value. It can be seen that the same kind of features has strong similarity, and different types of features have differences. Although the internal targets have volatility, the overall volatility is small. In the areas of leaves and sky, the most significant area is concentrated in the 0–300 band. In the area of sky, the pixel value of 0–300 band reaches saturation state. Based on the above analysis, the sky and leaves can be effectively distinguished.Figure 5
Display of spectral curves of typical ground objects: (a) spectral curve of leaves; (b) spectral curve of sky.
(a)(b)


### 3.2. Band Clustering Effect

In order to verify the accuracy of moving target extraction, we introduce the OA [29] overall accuracy and kappa coefficient. OA represents the proportion of samples with correct classification to all samples. Kappa is an index used for the consistency test [30]:(19)OA=1N∑i=1rxii,kappa=N∑i=1rxii−∑i=1rxi+×x+iN2−∑i=1rxi+×x+i.The band selection effect of real scene hyperspectral image is shown in Figure6, and the band selection effect of simulated scene hyperspectral image is shown in Figure 7. It can be seen from the figure that the effect of the real image is slightly lower than that of the simulated image, which is due to the stable noise and spectral curve contained in the simulated image. However, the information contained in the real image is more complex and has a certain volatility.Figure 6
Experimental results of real hyperspectral sequences: (a) OA; (b) kappa.
(a)(b)Figure 7
Experimental results of simulated hyperspectral sequences: (a) OA; (b) kappa.
(a)(b)The sparse nonnegative matrix factorization (SNMF) algorithm [6] transforms the problem of band selection into the problem of sparse decomposition, and it has a certain effect to extract significant spectral images. The fast volume gradient (FVG) algorithm [13] establishes the model according to the gradient to realize the band selection, and the effect is better for the region with obvious boundary. The variable precision neighborhood (VPN) algorithm [17] constructs fuzzy sets according to the relationship between adjacent pixels to realize band selection. The fast and late low rank (FLLR) algorithm [20] introduces the idea of low rank to calculate the redundancy between bands and realize band selection. In this paper, a subspace clustering algorithm is proposed based on depth confrontation, which fully considers the correlation of bands and optimizes the loss function to achieve band selection. On the real data set, OA reaches 90% and kappa reaches 0.67. OA and kappa are 96% and 0.92, respectively, in the simulation data set, which reach better results.


### 3.3. Target Detection Effect

On the basis of the optimal band selection, different target material clustering algorithms are used for comparison. The detection results of real data and simulated data are shown in Figure8. Hu et al. [31] proposed the SVM algorithm to extract image features for clustering to achieve enhancement. Han et al. [32] constructed CNN to extract target features. Shi and Pun [33] built multiscale RESNET to realize target detection. Li et al. [34] detected targets based on boundary features. The above algorithms analyze the target from the perspective of morphology to achieve target detection. Based on clustering, in this paper, we construct a visual perception model to detect the target and use the difference of visual mapping to measure the detection rate of the target, which has a good effect, and the ROC curve value is the highest.Figure 8
Roc curve: (a) real data; (b) simulated data.
(a)(b)The proposed algorithm constructs the mapping model of visual perception by fusing images with three bands. The mapping results of real data using different band combinations are shown in Figure9(a). {0, 38, 187} segment maps the leaf region to red, but it cannot distinguish the building and sky areas effectively. {1, 161, 35} spectrum segment can distinguish the building area from other areas, which verifies the effectiveness of the proposed algorithm. The mapping results of simulated data using different band combinations are shown in Figure 9(b). {0.4, 8.0, 12} spectrum can distinguish grassland from land but cannot distinguish grassland from vehicle. {0.6, 0.6, 10.4} bands can extract vehicles effectively and suppress grassland and land areas. The effectiveness of the proposed algorithm is verified.Figure 9
Target detection experimental results: (a) real data; (b) simulated data.
(a)(b)


## 3.1. Display of Spectral Curves of Typical Ground Objects

In order to show the spectral curves of typical ground objects, the spectral curves of leaves and sky are selected to display, as shown in Figure5. The horizontal axis represents the band number and the vertical axis represents the pixel value. It can be seen that the same kind of features has strong similarity, and different types of features have differences. Although the internal targets have volatility, the overall volatility is small. In the areas of leaves and sky, the most significant area is concentrated in the 0–300 band. In the area of sky, the pixel value of 0–300 band reaches saturation state. Based on the above analysis, the sky and leaves can be effectively distinguished.Figure 5
Display of spectral curves of typical ground objects: (a) spectral curve of leaves; (b) spectral curve of sky.
(a)(b)


## 3.2. Band Clustering Effect

In order to verify the accuracy of moving target extraction, we introduce the OA [29] overall accuracy and kappa coefficient. OA represents the proportion of samples with correct classification to all samples. Kappa is an index used for the consistency test [30]:(19)OA=1N∑i=1rxii,kappa=N∑i=1rxii−∑i=1rxi+×x+iN2−∑i=1rxi+×x+i.The band selection effect of real scene hyperspectral image is shown in Figure6, and the band selection effect of simulated scene hyperspectral image is shown in Figure 7. It can be seen from the figure that the effect of the real image is slightly lower than that of the simulated image, which is due to the stable noise and spectral curve contained in the simulated image. However, the information contained in the real image is more complex and has a certain volatility.Figure 6
Experimental results of real hyperspectral sequences: (a) OA; (b) kappa.
(a)(b)Figure 7
Experimental results of simulated hyperspectral sequences: (a) OA; (b) kappa.
(a)(b)The sparse nonnegative matrix factorization (SNMF) algorithm [6] transforms the problem of band selection into the problem of sparse decomposition, and it has a certain effect to extract significant spectral images. The fast volume gradient (FVG) algorithm [13] establishes the model according to the gradient to realize the band selection, and the effect is better for the region with obvious boundary. The variable precision neighborhood (VPN) algorithm [17] constructs fuzzy sets according to the relationship between adjacent pixels to realize band selection. The fast and late low rank (FLLR) algorithm [20] introduces the idea of low rank to calculate the redundancy between bands and realize band selection. In this paper, a subspace clustering algorithm is proposed based on depth confrontation, which fully considers the correlation of bands and optimizes the loss function to achieve band selection. On the real data set, OA reaches 90% and kappa reaches 0.67. OA and kappa are 96% and 0.92, respectively, in the simulation data set, which reach better results.


## 3.3. Target Detection Effect

On the basis of the optimal band selection, different target material clustering algorithms are used for comparison. The detection results of real data and simulated data are shown in Figure8. Hu et al. [31] proposed the SVM algorithm to extract image features for clustering to achieve enhancement. Han et al. [32] constructed CNN to extract target features. Shi and Pun [33] built multiscale RESNET to realize target detection. Li et al. [34] detected targets based on boundary features. The above algorithms analyze the target from the perspective of morphology to achieve target detection. Based on clustering, in this paper, we construct a visual perception model to detect the target and use the difference of visual mapping to measure the detection rate of the target, which has a good effect, and the ROC curve value is the highest.Figure 8
Roc curve: (a) real data; (b) simulated data.
(a)(b)The proposed algorithm constructs the mapping model of visual perception by fusing images with three bands. The mapping results of real data using different band combinations are shown in Figure9(a). {0, 38, 187} segment maps the leaf region to red, but it cannot distinguish the building and sky areas effectively. {1, 161, 35} spectrum segment can distinguish the building area from other areas, which verifies the effectiveness of the proposed algorithm. The mapping results of simulated data using different band combinations are shown in Figure 9(b). {0.4, 8.0, 12} spectrum can distinguish grassland from land but cannot distinguish grassland from vehicle. {0.6, 0.6, 10.4} bands can extract vehicles effectively and suppress grassland and land areas. The effectiveness of the proposed algorithm is verified.Figure 9
Target detection experimental results: (a) real data; (b) simulated data.
(a)(b)


## 4. Conclusion

Hyperspectral images have spatial resolution and interspectral resolution, which plays an important role in material recognition. Aiming at the difficulty of hyperspectral band selection, a deep adversarial subspace clustering network is constructed to select the representative band, which can select a representative band. From the perspective of psychology, a color perception model is constructed to highlight the significant areas. Experiments show that the proposed algorithm has good results. On this basis, it can carry out material recognition of typical targets and hidden targets.



---

*Source: 8329974-2021-09-13.xml*