MCM-aware Twin-least-square GAN for Hyperspectral Anomaly Detection

1 INTRODUCTION . Hyperspectral image ( HSI ) appears as a three-dimensional ( 3D ) data cube , two dimensions of which show the spatial information of materials , and the other reveals hundreds of contiguous bands to perceive each scene ( Yokoya et al. , 2012 ) . Among a wealth of HSIs interpretation techniques in practical situations , anomaly detection has many potential applications in video surveillance , activity recognition , and scene understanding , etc ( Lanaras et al. , 2015 ; Eyal et al. , 2019 ; Tu et al. , 2020 ) . However , due to the insufficient prior information , inaccurate labels , complex scenes , and unbalanced samples , it is high-cost and sometimes infeasible to accurately detect different types of anomalies in HSI . Consequently , hyperspectral anomaly detection without any priors is a challenging task and is of great importance . Deep learning-based methods have powerful and unique advantages in modeling and characterizing complex data ( Stanislaw et al. , 2020 ) . A lot of research has appeared in the field of anomaly detection , which can be roughly divided into three categories : supervised , semi-supervised , and unsupervised . However , due to the difficulty of annotation and collection of label training , supervised methods are rarely applied ( Grnitz et al. , 2013 ; Raghavendra & Sanjay , 2019 ) . Semi-supervised work aims to break the dilemma between the number of samples and detection performance , but it still requires pure background training samples ( Blanchar et al. , 2010 ; Wu & Prasad , 2018 ) . On the one hand , unsupervised learning based hyperspectral anomaly detection has become a new trend ( Schlegl et al. , 2017 ; Zhang et al. , 2019 ) . On the other hand , the detection performance is limited due to the lack of prior knowledge . Therefore , we propose an MCM-aware strategy to adaptively obtain reliable and stable pseudo-labeled prior information to alleviate these problems . Concretely , motivated by the observations mentioned above , we estimate the priors and model the background with multi-scale covariance matrices as the necessary preparation fed into the MTGAN model , which generates discriminative representations with second-order statistics in covariance pooling and is conducive to exploiting the intrinsic spatial-spectral information of HSI . The progress of MCM-aware priors construction strategy is illustrated in Figure 1 . Furthermore , though GAN performs well in anomaly detection tasks according to the literature , the real objective of GAN is supposed to capture more separable latent features between background and anomalies instead of minimizing the pixel-wise reconstruction error ( Gong et al. , 2020 ) . The gradient vanishing problem , which is partly caused by the hypothesize that the discriminator as a classifier with the sigmoid cross-entropy loss function in regular GANs , is not conducive to the generation of background and discrimination of anomalies . Hence , to facilitate the training stability and alleviate the gradient-vanishing problem , we present twin-least-square loss to perform background modeling in feature and image domains . Accordingly , we can solve the problem of gradient vanishing and enhance the representation directly aiming at the reconstruction of each pixel . In light of the difficulties of the separability between the anomaly and background , we also impose an anomaly rejection loss to avoid anomalies contamination in background estimation . In this way , the network can reconstruct resembled background dictionaries , but dramatically changed anomalies , thereby increasing the degree of difference between them and endow better detection accuracy . To verify the effectiveness of the proposed method , we implement evaluations on five public HSI data sets . In MTGAN , the average AUC scores of ( Pd , Pf ) and ( Pf , τ ) are 0.99809 and 0.00518 , respectively , which outperform previous state-of-the-art methods . To summary , our contributions are mainly three-fold : • To solve the problem of insufficient samples that previous methods suffer from , we propose an MCM-aware strategy to reliably and adaptively generate prior dictionaries . In specific , we calculate a series of multi-scale covariance matrices , taking advantage of the second-order statistics to naturally model the distribution with integrated spectral and spatial information . • The twin-least-square loss is introduced into both the feature and image domains to overcome the gradient vanishing problem . Meanwhile , the generative ability and training stability can be improved , which can fit the characteristics of high-dimension and complexity of HSI data . • To further reduce the false alarm rate , we design a novel anomaly rejection loss to enlarge the distribution diversity between background regions and anomalies , aiming to distinguish between background and anomalies . Experimental results illustrate that the AUC score of ( Pf , τ ) in MTGAN is one order of magnitude lower than other state-of-the-art methods . 2 RELATED WORK . For traditional methods , the RX method assumes that each spectral channel is Gaussian-distributed , and the pixel is L-dimensional multi-variate Gaussian distributed ( Guo et al. , 2014 ; Luo et al. , 2019 ; Ahmed et al. , 2020 ) . As a non-RX based methods , the ADLR method obtains abundance vectors by spectral decomposition and constructs a dictionary based on the mean value clustering of abundance vectors ( Qu et al. , 2018 ) . The PAB-DC model imposed with low-rank and sparse constraints considers the homogeneity of background and the sparsity of anomalies to construct the dictionaries ( Huyan et al. , 2019 ) . The emerging typical algorithm AED removes the background mainly by attribute filtering and difference operation . Additionally , the LSDM-MoG method combines the mixed noise models and low-rank background to characterize complex distributions more accurately ( Li et al. , 2020 ) . However , these conventional methods are based on single-scale Gaussian assumption and can not represent complex and high-dimensional data sets well , leading to the exploration of deep learning-based methods ( Ben et al. , 2014 ) . In deep auto-encoding Gaussian mixture model ( DAGMM ) ( Zong et al. , 2018 ) , an autoencoder ( AE ) is introduced to the model to generate a low-dimensional representation and reconstruction error for each input data point as the input of the Gaussian Mixture Model ( GMM ) . GAN has attracted a lot of attention for providing a generative model to minimize the distance between the training data distribution and the generative model samples without explicitly defining a parametric function ( Goodfellow et al. , 2014 ; Yuan et al. , 2019 ; Gu et al. , 2020 ) . A novel single-objective generative adversarial active learning ( SO-GAAL ) method for outlier detection considers the anomaly detection as a binary-classification issue by sampling potential outliers from a uniform reference distribution ( Liu et al. , 2019 ) . Nevertheless , these deep learning-based methods can not achieve a balance between good performance and limited prior information . What ’ s more , the network structure of these methods is not specially designed for hyperspectral anomaly detection . Therefore , we propose MTGAN concerning hyperspectral anomaly detection for the first time , to approximate the performance of the supervised methods while releasing the limitation of training samples . 3 PROBLEM STATEMENT AND FRAMEWORK . In this work , we elaborate on MTGAN for hyperspectral anomaly detection , as shown in Figure 2 . The three key components of the framework include : 1 ) the MCM module for background dictionary construction ; 2 ) the twin-least-square GAN module for background reconstruction ; 3 ) the anomaly rejection loss added joint learning . The modules are cascaded together for hyperspectral anomaly detection . 3.1 CONSTRUCTING THE OVERALL MODEL . We denote the HSI as H ∈ Rh×w×d , where d is the number of the spectral bands . h and w represent the spatial size of the data . For convenience , as the input of the network , we transform the 3-D cube H into a 2-D matrix H = { hi } ni=1 ∈ R d×n , where each column of H is a spectral pixel vector in the HSI and n = h × w is the number of the pixels . The HSI data matrix is decomposed into two components : background and anomaly . We denote background and anomaly as Y = [ y1 , y2 , ... , ynB ] , X = [ x1 , x2 , ... , xnA ] , and nA + nB = n , respectively , where yi and xi represent the ith vectors . Based on the defined background and anomaly dictionaries , we formulate the anomaly detection method as L ( Y , X ) = LTLS ( Y ) + Lauto ( Y , Ŷ ) + Lenlarge ( Ŷ , X ) = LLS1 ( Z ) + LLS2 ( Y , Ŷ ) + ∥∥∥Y − Ŷ∥∥∥− α ∥∥∥Ŷ −X∥∥∥ , ( 1 ) s.t . Z = Enc ( Y ; arg ( G ) ) Ŷ = Dec ( Z ; arg ( D ) ) α ∼ N ( 0 , I ) , where ∥∥∥Y − Ŷ∥∥∥ denotes the reconstruction error of the basic AE network . The twin-least-square loss LTLS added for the two discriminators are denoted by LLS1 and LLS2 , respectively , which make up the whole twin-least-square loss LTLS . Lauto and Lenlarge represent the spectral reconstruction loss and separability loss between background-anomaly , respectively . Enc and Dec represent the encoder and decoder , respectively . Z is the output of the encoder . And the encoding and decoding process can be demonstrated as Ŷ = σ ( WWTY + B ) , ( 2 ) where σ ( · ) represents the activation function . Ŷ is the output of the network , and W is the weight of the encoder . B is the bias of the whole network . 3.2 MCM-AWARE PRIOR FOR BACKGROUND CONSTRUCTION . Inspired by estimating the Mahalanobis distance between the test pixels and the constructed pixels on one scale , we generate pseudo priors for GAN training through multi-scale covariance maps construction . Thus , we can meet the requirement of sufficient prior information and take advantage of the spatial and spectral information of HSI . The whole generation progress by pseudo-labeling can be expressed as ( Y , X ) = fMCM ( H ) , ( 3 ) where fMCM ( · ) represents the nolinear learning process of MCM strategy . Y and X denote the background and anomaly dictionaries , respectively . 3.2.1 MULTI-SCALE LOCALIZING . For each central pixel , we try to realize multi-scale localizing first based on the Euclidean distance with a classical classifier , i.e. , K nearest neighbors ( KNN ) , to obtain the local pixel cubes at different scales . Then , we generate a series of gradually increasing cubes of different scales . For each of the cubes , we transfer it to a vector . After that , the covariance matrix is calculated between the vectors . 3.2.2 GENERATING CO-VARIANCE MAPS . For the central pixel hi , taking the scaleR×R as an example , the covariance map of hi on the fixed one scale is extracted as Ck = 1 R2 − 1 R2∑ i=1 ( hi − µ ) ( hi − µ ) T ∈ RL×L , ( 4 ) where µ represents the mean of the set of input HSI vectors { hi|i = 1 , 2 , · · · , R } . { hi|i = 2 , · · · , R2 } represents the corresponding adjacent pixels in a window of R × R pixels . In addition , M scales of Rk , i.e . k = 1 , · · · , M , are taken into account . The covariance maps of other scales are denoted by Ck , k = 1 , · · · , M , which make up the co-variance pool to construct the background .

Summary: In this paper, the authors proposed the MTGAN framework, a GAN-based approach to the task of anomaly detection in hyperspectral images. The main idea behind this work is to exploit twin-least-square loss to perform background modeling in feature and image domains to alleviate the gradient vanishing problem of the previous GAN-based anomaly detection methods. Specifically, they proposed i) an MCM-aware strategy to construct the multi-scale priors, ii) a twin-least-square loss on GAN for training stabilization, and iii) an anomaly rejection loss for background estimation. The experiments on multiple benchmarks show the superiority of the MTGAN to the state of the art hyperspectral anomaly detection methods.