MCM-aware Twin-least-square GAN for Hyperspectral Anomaly Detection
Authors: Jiaping Zhong
Weiying Xie
Jie Lei
Yunsong Li
Zan Li
Venue:  (2021)
Keywords: Multiscale covariance map (MCM)
least square loss
hyperspectral anomaly detection
generative adversarial network (GAN)

Abstract: Hyperspectral anomaly detection under high-dimensional data and interference of deteriorated bands without any prior information has been challenging and attracted close attention in the exploration of the unknown in real scenarios. However, some emerging methods based on generative adversarial network (GAN) suffer from the problems of gradient vanishing and training instability with struggling to strike a balance between performance and training sample limitations. In this work, aiming to remedy the drawbacks of existing methods, we present a novel multi-scale covariance map (MCM)-aware twin-least-square GAN (MTGAN). Instead of the widely used single-scale Gaussian hypothesis background estimation, in MTGAN, we introduce the MCM-aware strategy to construct multi-scale priors with precise second-order statistics, thereby implicitly bridging the spatial and spectral information. Thus, we reliably and adaptively represent the prior of HSI to change the priors-lack situation. Moreover, we impose the twin-least-square loss on GAN, which helps improve the generative ability and training stability in feature and image domains, overcoming the gradient vanishing problem. Finally, the network enforced with a new anomaly rejection loss establishes a pure and discriminative background estimation. Experiments demonstrate that the average detection accuracy of MTGAN reaches 0.99809, which is superior to the state-of-the-art algorithms.

Under review as a conference paper at ICLR 2021
MCM- TWIN-LEAST-SQUARE GAN FOR HY- PERSPECTRAL ANOMALY DETECTION
Anonymous authors
Paper under double-blind review
ABSTRACT
Hyperspectral anomaly detection under high- -igh-dimensional data and interference of deteriorated bands without any prior information has been challenging and attract- ed close attention in the exploration of the unknown in real scenarios. However, some emerging methods based on generative adversarial network (GAN) suffer from the problems of gradient vanishing and training instability with struggling to strike a balance between performance and training sample limitations. In this work, aiming to remedy the drawbacks of existing methods, we present novel multi-scale covariance map (MCM))aware twin-least-squaat GAN (MTGAN). In- stead of the widely used single-scale Gaussian hypothesis background estimation, in MTGAN, we introduce the MCM-aware strategy to construct multi-scale pri- ors with precise second-order statistics, thereby implicitly bridging the spatial and spectral information. Thus, we reliably and adaptively represent the prior of HSI change the priors-lack situation. Moreover, we impose the twin-least-squarc loss on GAN, which helps improve the generative ability and training stability in feature and image domains, overcoming the gradient vanishing problem. Finally, the network enforced with new anomaly rejection loss establishes pure and discriminative background estimation. Experiments demonstrate that the average detection accuracy of MTGAN reaches 0.99809, which is superior to the state-of- the-art algorithms.
INTRODUCTION
Hyperspectral image (HSI) appears as three-dimensional (3D) data cube, two dimensions of which show the spatial information of materials, and the other reveals hundreds of contiguous bands to perceive each scene (Yokoya et al. 2012). Among a wealth of HSIs interpretation techniques in practical situations, anomaly detection has many potential applications in video surveillance, ac- tivity recognition, and scene understanding, etc (Lanaras et al,, 2015 Eyal et al., 2019 Tu et al., 2020). However, due to the insufficient prior information, inaccurate labels, complex scenes, and unbalanced samples, it is high-cost and sometimes infeasible to accurately detect different types of anomalies in HSI. Consequently, hyperspectral anomaly detection without any priors is challeng- ing task and is of great importance.
Deep (earning-based methods have powerful and unique advantages in modeling and characteriz- ing complex data (Stanislaw et al. (020). A lot of research has appeared in the field of anomaly detection, which can be roughly divided into three categories: supervised, semi-supervised, and un- supervised. However, due to the difficulty of annotation and collection of label training, supervised methods are rarely applied (Griizz et al., 2013] Raghavendra work aims to break the dilemma between the number of samples and detection performance, but it still requires pure background training samples (Blanchar et al. 2010: Wu & Prasad, 2018). On the one hand, unsupervised learning based hyperspectral anomaly detection has become
Sanjay 2019). Semi-supervised
trend (Schlegl et al. 2017 Zhang et al. 2019). On the other hand, the detection performance is limited due to the lack of prior knowledge. Therefore, we propose MCM-aware strategy to adaptively obtain reliable and stable pseudo-labeled prior information to alleviate these problems.
Concretely, motivated by the observations mentioned above, we estimate the priors and model the background with multi-scale covariance matrices as the necessary preparation fed into the MTGAN model, which generates discriminative representations with second-order statisties in covariance

---

Under review as a conference paper at ICLR 2021
pooling and is conducive to exploiting the intrinsic spatial-spectral information of HSI. The progress of MCM-aware priors construction strategy is illustrated in Figure]
Furthermore, though GAN performs well in anomaly detection tasks according to the liter- ature, the real objective of GAN is supposed to capture more separable latent features between background and anomalies instead of minimiz- ing the pixel-wise reconstruction error (Gong al.] 2020). The gradient vanishing prob- lem, which Is partly caused by the hypothesize that the discriminator as classifier with the sigmoid cross-entropy loss function in regular GANs, is not conducive to the generation of background and discrimination of anomalies.
Anomaly regron
0ov(c,
cov(c,
Vocter space (u Aarr Jud
Veetor matrix
Covariance unotix
Figure I: The progress of MCM-aware priors con- struction strategy.
Hence, to facilitate the training stability and alleviate the gradient-vanishing problem, we present twin-least-square loss to perform background modeling in feature and image domains. Ac- solve the problem of gradient vanishing and enhance the representation directly aiming at the reconstruction of each pixel.
cordingly,
In light of the difficulties of the separability between the anomaly and background, also impose an anomaly rejection loss to avoid anomalies contamination in background estimation. In this way, the network can reconstruct resembled background dictionaries, but dramatically changed anoma- lies, thereby increasing the degree of difference between them and endow better detection accuracy.
To verify the effectiveness of the proposed method, we implement evaluations five public HST data sets. In MTGAN, the average AUC scores of (Pa, Py) and (P,, r) are 0,99809 and 0.005 18, respectively, which outperform previous sate-of-the-as methods. To summary, our contributions are mainly three-fold:
To solve the problem of insufficient samples that previous methods suffer from, pro- pose an MCM-aware strategy to reliably and adaptively generate prior dictionaries. In specific, calculate series of multi-scale covariance matrices, taking advantage of the second-order statistics to naturally model the distribution with integrated spectral and spa- tial information.
The wwin-least-squar loss is introduced into both the feature and image domains to over- come the gradient vanishing problem. Meanwhile, the generative ability and training sta- bility can be improved, which can fit the characteristics of high-dimension and complexity HSI data.
To further reduce the false alarm rate, design novel anomaly rejection loss to enlarge the distribution diversity between background regions and anomalies, aiming to distinguish between background and anomalies. Experimental results illustrate that the AUC score of (P,,T) in MTGAN is one order of magnitude lower than other state-of-the-art methods.
RELATED WORK
For traditional methods, the RX method assumes that each spectral channel is and the pixel is L-dimensional multi-variate Gaussian distributed (Guo et al. 2014 Luo et al., 2019) Ahmed et 2020). As non-RX based methods, the ADLR method obtains abundance vectors by spectral decomposition and constructs dictionary based on the mean value clustering of abun- dance vectors (Qu et al., 2018). The PAB-DC model imposed with low-rank and sparse constraints considers the homogeneity of background and the sparsity of anomalies to construct the dictionar- ies (Huyan et al. 2019). The emerging typical algorithm AED removes the background mainly by attribute filtering and difference operation. Additionally, the LSDM-MoG method combines the mixed noise models and low-rank background to characterize complex distributions more accurately (Li al. 2020). However, these conventional methods are based single- scale Gaussian assump- tion and cannot represent complex and figh-dimensional data sets well, leading to the exploration of deep leasning-based methods (Ben al. 2014).
2

---

Under review as a conference paper at ICLR 2021
In deep auto-encoding Gaussian mixture model (DAGMM) Zong al. 2018 autoencoder (AE) introduced to the model to generate a low-dimensional representation and reconstruction error for each input data point the input of the Gaussian Mixture Model (GMM). GAN has attracted lot of attention for providing a generative model to minimize the distance between the training da- distribution and the generative model samples without explicitly defining parametric function (oodffellw et al. 2014 Yuan et al. [2019] Gu et al.) 22020). A novel single-objective generative ad- versarial active learning (SO-GAAL) method for outlier detection considers the anomaly detection issue by sampling potential outliers from a uniform reference distribution (Liu al., 2019). Nevertheless, these deep learning- -ased methods cannot achieve balance be- tween good performance and limited prior information. What's more, the network structure of these methods not specially designed for hyperspectral anomaly detection. Therefore, we propose MT- GAN concerning hyperspectral anomaly detection for the first time, to approximate the performance of the supervised methods while releasing the limitation of training samples.
as dinary-classifiee
3 PROBLEM STATEMENT AND FRAMEWORK
In this work, we elaborate on MTGAN for hyperspectral anomaly detection, as shown in Figure 2 The three key components of the framework include: 1) the MCM module for background dictionary construction; 2) the twin-least-squan GAN module for background reconstruction; 3) the anomaly rejection loss added joint learning. The modules are cascaded together for hyperspectral anomaly detection.
Output
Vectorizing
Generating CMs
Inport HSI
Reloaded
Muliscale lecalizinng
MCM-bseed backgroudd modeling
MCM
diserimituation
Dg
Detection map
Anomaly rejection constraimt T wwin-least--sp lcast-square for backgroand reconstruction
Dy 6.98
MCM for background dictionary construction
Discrimination
Figure Overview our proposed MTGAN framework.
3.1 CONSTRUCTING THE OVERALL MODEL
where d is the number of the spectral bands. h and represent the spatial size the data. For convenience, as the input of the network, we transform the 3-D cube H into 2-D matrix H (h:) R 4xm where each column of H is spectral pixel vector the HSI and n x w is the number of the pixels. The HSI data matrix is decomposed into two components: background and anomaly. We denote background and anomaly Y -2,.... the ith vectors.
We denote the HSI as H TK Axwxd
[x1, X2,..., X and n + ng n, respectively, where y; and x; represent
Based on the defined background and anomaly dictionaries, we formulate the anomaly detection method
L L(Y.X) = LTLS (Y) + Lauto (Y, + Lentarge (f,x)
Lr.s (Z) + L1.52 Y Y + Y Y - 4 Y-

---

Under review as a conference paper at ICLR 2021
Z Enc (Y; argr(G)) Y Dec (Z; arg D))
8.b.
a N (0,
denotes the reconstruction error of the basic AE network. The squale loss LTLS added for the discriminators are denoted by LLS, and L1s2, respectively, which make up the whole twin-least-squaee loss LTL.S- Lauto and Lentarge represent the spectral re- construction loss and separability loss between sngground-anomalle respectively. Enc and Dec represent the encoder and decoder, respectively. Z is the output of the encoder. And the encoding and decoding process can be demonstrated
where Y Y
Y
Y = +WWWY+B)
(2)
where a (.) represents the activation function. Y the output the network, and W is the weight of the encoder. B is the bias of the whole network.
3.2 MCM-AWARE PRIOR FOR BACKGROUND CONSTRUCTION
Inspired by estimating the Mahalanobis distance between the test pixels and the constructed pixels on scale, generate pseudo priors for GAN training through multi-scale covariance maps construction. Thus, we can meet the requirement of sufficient prior information and take advantage of the spatial and spectral information of HSI. The whole generation progress by pseudo-labeling can be expressed as
(Y, X) SMCM (H),
where SMCM (.) represents the nolinear learning process of MCM strategy. Y and denote the background and anomaly dictionaries, respectively.
3.2.1 MULTI-SCALE LOCALIZING
For each central pixel, we try to realize multi-scale localizing first based on the Euclidean distance with classical classifier, i.e., K nearest neighbors (KNN), to obtain the local pixel cubes different scales. Then, we generate series of gradually increasing cubes of different scales. For each of the cubes, we transfer vector. After that, the covariance matrix is calculated between the vectors.
3.2.2 GENERATING CO-VARIANCE MAPS
For the central pixel h, taking the scale Rt x R an example, the covariance map of h, on the fixed one scale is extracted as
C₁ R2
Σ h, (1) (h; -n)² E LL
(4)
where / represents the mean of the set of input HSI vectors (h:li = 1,2, ... R). [hali = 2, _r2) represents the corresponding adjacent pixels in a window of R x R pixels. In addition, M scales of R& i.e. k 1, .M.. taken into account. The covariance maps of other scales are denoted by Ck. ko the background.
M, which make up the co-variance pool to construct
3.3
CASCADED ARCHITECTURE WITH TWIN-LEAST-SQUARE LOSS
3.3.1 STABILITY BRANCH
As mentioned above, as the improved version of the original GAN, we present the architecture based on twin-least-squaat square loss solve the gradient vanishing problem and enhance the stability. As shown in Figure 2 the MTGAN network consists of two discriminators and generator. Instead of the cross-entropy in GAN, we impose twin-least-square loss on the two discriminators. The twin-least-qquare loss can be expressed as
LLS, =
2 (orppau..(y)|((Dppppr - 1))1+ (D + +1)
(S)
4

---

Under review as a conference paper at ICLR 2021
L1.9 1 (y))p
1 (9 1)'}.
(6)
From the binary classification point of view, introduce the twin-least-squee loss to move false samples to the decision boundary of being anomaly or background and punish those being far from the decision boundary, even if on the right side. Thus, perform adversarial training least-square-log imposed discriminators DR and Dy against the generator, aiming to overcome gradient vanishing and stably match the distribution of the decoded vectors y and the known input data distribution y.
the two
3.3.2 SEPARABILITY BRANCH
As mentioned earlier, we impose the spectral reconstruction loss of AE using mean squared error (MSE) minimize the deviation between the decoded images and the original input image:
Laato Y. (YY))
Y
Y
= = Y
(WWYY Y + B)
To ensure that the learning samples come entirely from the background, we introduce the following distance function based on Lauto- The second item of which is expected be large possible:
Taanorm
Y, (,
Trauto (Y, 8)
Trentarge
Y Y
Y - 8 a 8 |
x
Y - (WWTY + B)|| a ( x||
Let x; and y; represent the ith component X and Y, respectively. denotes the mean of all the x; in X. When the distance between the reconstructed spectrum vector ÿ; and the average spectrum vector X is small, y, is suspected to be the anomaly. Then the suppression coefficient a of the function aims to reduce it rapidly for adjustment. When the distance between X and y; is large, from a statistical point of view, y; is the background dictionary to be estimated. Then the value of the suppression coefficient a approximate 1.
3.3.3 JOINT TRAINING AND DISCRIMINATION
The twin-least-square loss, spectral reconstruction loss, and anomaly rejection loss jointly learned with the weighting coefficient of 1 by doing alternative updates of each component as fol- lows. Subsequently, we obtain the detection maps by Gaussian statistics-based discrimination.
Minimize Lt.s; by updating parameters of Dr.
Minimize LLS, by updating parameters of DR-
Minimize Lunorm by updating parameters of En and the decoder De.
EXPERIMENT
4.1
EXPERIMENTAL. SETUP
Data Sets. HYDICE describes urban scenes in the United States that were captured by the hyper- spectral digital image acquisition experiment (HYDICE) airborne sensors above the city. sub- image with spatial size of 80 x 100 is cut out from the entire 307 x 307 original image with 162 spectral channels ranging from 400 nm to 2500 nm. Sirport-Beach-Unig the Airborn visible/infrared imaging spectrometer (AVIRIS) sensors. Among airport, beach, and urban scenes, we conduct experiment on two the urban scenes. The sample images in ABU con- the corresponding reference maps. EI Segundo is captured by AVIRIS sensor. The scene contains the area the refinery, several residential areas, parks, and school areas. has 250 300 pixels in the spatial domain and 224 bands in spectral,
(ABU) was captured by
tain 100 100 or 150 150 pixels and
5

---

Under review as a conference paper at ICLR 2021
ranging from 366 2496 nm. Grand Island is also acquired by the AVIRIS sensor at the location of Grand Island on the Gulf Coast, which contains 300 x 480 pixels and 224 spectral channels with a wavelength range of 366 2496 nm. The pseudo-color images of each data set and their corresponding ground truth maps are shown in Figure 4 respectively.
Hyper-parameters. In experiment, set the number of training iterations 5000. The learning rate is set to 0.0001. There are three layers for the encoder, decoder, and discriminators networks, respectively. The number of the extracted feature is set 20. With back propagation, we optimize all parameters by the Adam optimization algorithm.
4.2 EVALUATION RESULTS
For quantitative comparison, we can observe from Tables [ and 2 that for all data sets, the results of the MTGAN method and comparison methods are close ideal values.
Method
HYDICE
urban1
urban2
Grand Island
EI Segundo
Average
RX
0.97637
0.99463
0.98870
0.99990
0.97826
0.98757
LSDM-MoG
0.95643
0,99321
0.97504
0.99989
0.96798
0.97851
ADLR
0,99471
0.98774
0.99223
0,99993
0.99145
0,99321
PAB-DC
0,99760
0,98327
0.54635
0,98768
0.75911
0,85480
AAE 0.92185
0.96806
0.99284
0,99940
0.99260
0.97495
MTGAN
0.99945
0.99816
0.99312
0.99991
0.99982
0.99809
Table I: Evaluation AUC score of (Pa, Pf) obtained by MTGAN and compared methods.
Method
HYDICE
urban1
urban2
Grand Island
El Segundo
Average
RX
0.03798
0.01351
0.01 140
0.00738
0.00952
0.01596
LSDM-MoG
0.18799
0.04555
0.01338
0.02848
0.14127
0.08333
ADLR
0.00316
0.08011
0.07451
0.00079
0.09622
0.05096
PAB-DC
0.10525
0.16974
0.32350
0.06799
0.10506
0.15431
AAE 0,00962
0.04229
0.06192
0,04818
0.04355
0.04111
MTGAN
0.01541
0.00200
0.00145
0.00069
0.00218
0.00518
Table Evaluation AUC score of (P),T) obtained by MTGAN and compared methods.
Specifically, for the HYDICE data set, though the (Pr,7) obtained by MTGAN does not reach the order of magnitude of three decimal places, the AUC score of (Pa, Py) is 0.99945 which outper- forms other methods, demonstrating that the MTGAN can maintain the detection ability well with extremely low missed detection. Compared with RX and PAB-DC methods, for the ABU-urbanl data set, MTGAN can detect more anomalies, showing better performance. According to the results in Table 1. the AUC score of (P of MTGAN is smaller than other comparison methods for the ABU-urban- data set, indicating that the MTGAN can inhibit background for the scenarios better. For Grand Island, the AUC score of (Pa, Pý) obtained by MTGAN is 0.99991, which is much higher than most typical methods but less than the ADLR algorithm of 0.99993. However, the AUC score of (P),7) of MTGAN 0.00069, which better than the ADLR of 0.00079. The detection of the ABU-urban2 and EI Segundo data sets also performs well, which indicates that MTGAN has com- promised the AUC scores of (Pa, Py) and 7P).7). Generally speaking, the comparison methods may achieve good results for some specific data sets, but MTGAN can achieve promising detection results both the AUC scores of (Pa, Pf) and (P,.T) for all the data sets. From the ROC curves shown in Figure 3 we can conclude that the MTGAN obtains higher AUC score of (Pa, Py) with identical AUC score of (P illustrating a superior performance to other methods.
4.3 PARAMETER SENSITIVITY ANALYSIS
There are mainly two parameters to be analyzed: the size of the window and the scales of the MCMs. Due to the pixel-level processing characteristics of the MTGAN and its sensitivity to the size of the

---

Under review as a conference paper at ICLR 2021
2 & S 2
MILGAN
MIGGAN
LADDL-AS
RX
BX
AAT
AAT.
FOUB-ICC
FABLDC
XDUR
ABER
ABER
19²
30*
18
18:
TalePaatii Bax
EnbePPartte
Endor Part
MIGGAN LSBAL-NoR
VETGAN
LADY-MBc
MIGAN
MCM
Fake Yaheeeeckie
(d)
Figure 3: The ROC curves of different methods for (a) HYDICE, (b) ABU-urbanI, (c) ABU-urban2, (d) Grand Island, (e) EI Segundo, (f) models in MTGAN without MCM or twin-least-square loss.
windows, we fixed the scales of the MCMs as 5. With the size of the windows varying from 30 to 50, the values differ from each other, and the optimal results are achieved for different data sets. For the HYDICE data set, it achieves the highest AUC score of (Pa. Pf) when the size is 25. And the value 15 for the ABU-urban-2 data set. As for the Grand Island and Segundo data sets, the optimal size values are 20 and 24, respectively.
Similarly, when the scales the MCM are changed from 10 at steps, the results of the AUC score of (Pa.Pr) illustrate different performance. When set the scales to for the HYDICE data set, the performance of the experimental data set can achieve the best. For the rest the data sets, make balance between the performance and the computation set the scales of the MCM costs.
4.4 ABLATION STUDY
To better understand the effect of each component on the output detection result in our method, we analyzed the performances under the following four training models: 1) MTGAN without MCM module; 2) MTGAN without twin-least-square loss module; 3) MTGAN without anomaly rejection we have the following observations.
loss; 4) MTGAN. In 3able
Configuration
AUC of (Pa, Pr)
AUC of (Pr,))
without MCM without twin-least-square loss without anomaly rejection loss MTGAN
0.99534
0.00572
0,99544
0.00554
0.99654
0.00525
0.99809
0.00518
Table 3: Average AUC scores of and (Py:7) 7) for component analysis.
The MTGAN achieves higher AUC score of (Pa, Py) and lower AUC score (Py, r) than other models. The AUC score of (Pa, Py) on average compared to other configurations are improved by about 0.276%, 0.266%. and 0.156% respectively. And the AUC score of (Py. r) on average are optimized by about 10.425% 6.950%, and 1.351% %, respectively. The results indicate the effective-
7

---

Under review as a conference paper at ICLR 2021
ness and necessity of MCM, twin-least-square loss, and anomaly rejection loss, which contribute effectively to improve the detection performance.
4.5 VISUALIZING THE DETECTION MAPS
For qualitative comparison, shown in Figure 4 the PAB-DC method almost visually loses all anomalies for the HYDICE data set. Most anomalies are visually mixed with the background in the detection results of AED and LSDM-MoG methods. The RX method can detect almost all anomalies with high intensity, but it cannot suppress background interference well. For the ABU- urbanl data set, the MTGAN can well detect the anomalies with the highest intensity and relatively low false alarm rate in urban scenes. The RX, AAE, ADLR, and AED methods can identify the anomalies locations, but cannot retain the shape of them. For ABU-urban2 data set, both AAE and MTGAN can produce excellent detection results with good background suppression visually, but the ADLR gets poor performance. For the Grand Island data set, RX, LSDM-MoG, and PAB-DC methods can almost completely detect anomalies but they cannot effectively suppress background. For the EI Segundo data set, MTGAN also shows the best detection performance in both quantitative evaluation and visual analysis.
Prcade-color image
Ground truth
LSDX
PAB-IK:
(b)
61
(e)
Figure 4: Pseudo-color image, ground truth, and detection maps of MTGAN and the compared methods for (a) HYDICE, (b) ABU-urbanl, (c) ABU-urban2, (d) Grand Island, (e) El Segundo.
5 CONCLUSION
In this paper, we propose a new MCM-aware twin-least-squares GAN model for hyperspectral anomaly detection. Several aspects of our framework deserve consideration. For the first time, we propose the MCM strategy to construct multi-scale prior information, reuse and embed the covari- ance map at multi-scale create reliable and stable priors adaptively. Therefore, we can solve the lack of priors with pseudo-labeling and make full use of spectral and spatial information. To over- come the problem of gradient vanishing and generate high-quality images, we introduce twin-least- square loss to the architecture in both feature and image domains. Finally, the network enforced with a novel anomaly rejection loss establishes a pure and discriminative background estimation, sepa- rating background and anomalies to greater extent. Through experiments, have proved that the MTGAN framework exhibits superior performance in background reconstruction and outperforms the sate-of-the-aas methods.
8

---

Under review as a conference paper at ICLR 2021
REFERENCES
Imtiaz Ahmed, Xia Ben Hu, Mithun P. Acharya, and Yu Ding. Neighborhood structure assisted non- negative matrix factorization and its application unsupervised point anomaly detection. arXiv preprint arXiv:2001.066 065411, 2020.
Salem Manel Ben, Ettabaa Karim Saheb, and Hamdi Mohamed Ali. Anomaly detection in hyper- spectral imagery: overview. In International Image Processing, Applications and Systems Conference (IPAS), pp. 1 1-6, Nov. 2014.
Gilles Blanchar, Gyemin Lee, and Clayton Scott. Semi-supervised novelty detection. Journal of Machine Learning Research, 11:2973-3000 Feb. 2010.
Gutflaish Eyal, Kontorovich Aryeh, Sabato Sivan, Biller Ofer, and Sofer Oded. Temporal anomaly detection: Calibrating the surprise. 33rd AAAI Conference on Artificial Intelligence (AAAI), pp. 689-692- 2019.
Zhiqiang Gong, Ping Zhong, and Weidong Hu. Statistical loss and analysis for deep learning in hyperspectral image classification. IEEE Transactions on Neural Networks and Learning Systems, 12, Mar. 2020. doi: 10.1099/NNLS
lan Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, and Yoshua Bengio. Generative adversarial nets. In Proceedings of the Conference on Advances in Neural Information Processing Systems (NIPS), pp. 2672-2680, 2014.
Nico Grnitz, Marius Kloft, Konrad Rieck, and Ulf Brefeld. Toward supervised anomaly detection. Journal of Artificial Intelligence Research, 46(2):235-26) 262, Feb. 2013.
Jinjin Gu, Yujun Shen, and Bolei Zhou. Image processing using multi-code gan prior. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 13-19, June 2020. doi: 10. 11011////2222200222...0000
Qiandong Guo, Bing Zhang, Qiong Ran. Lianru Gao, Jun li, and Antonio Plaza. Weighted-rxd and linear filter-based rxd: Improving background statistics estimation for anomaly detection in hyperspectral imagery. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 766):2351-2366, Jun. 2014.
Ning Huyan, Xiangrong Zhang, Huiyu Zhou, and Licheng Jiao. Hyperspectral anomaly detection via background and potential anomaly dictionaries construction. IEEE Transactions on Geoscience and Remote Sensing, 57(4):2263-2276, 2019.
Charis Lanaras, Emmanuel Baltsavias, and Konrad Schindler. Hyperspectral super-resolution by coupled spectral unmixing. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), volume 50, pp.
2015.
Lu Li, Wei Li, Qian Du, and Ran Tao. Low-rank and sparse decomposition with mixture of gaussian for hyperspectral anomaly detection. IEEE Transactions on Cybernetics, pp. 1-10, Feb. 2020.
Yezheng Liu, Zhe Li, Chong Zhou, Yuanchun Jiang, Jianshan Sun, Meng Wang, and Xiangnan He. Generative adversarial active learning for unsupervised outlier detection. IEEE Transactions on Knowledge and Data Engineering, 2019. doi: 200.9//KEE.219.99000..0
Weixin Luo, Wen Liu, Dongze Lian, Jinhui Tang, and Shenghua Gao. Video anomaly detection with sparse coding inspired deep neural networks. IEEE Transactions Patlern Analysis and Machine Intelligence, 129:1-1, Sep. 2019.
Ying Qu, Wei Wang, Rui Guo, Bulent Ayhan, Chiman Kwan, Steven Vance, and Hairong Qi. Hy- perspectral anomaly detection through spectral unmixing and dictionary- low-rank decom- position. IEEE Transactions Geoscience and Remote Sensing, 56(8):1-15, 2018.
Chalapathy Raghavendra and Chawla Sanjay. Deep learning for anomaly detection: A survey, arXiv preprint 1901::901.0300 003477 2019.

---

Under review as a conference paper at ICLR 2021
Thomas Schlegl, Philipp Seebek, Sebastian M. Waldstein, Ursula Schmidt-Erfurte and Georg Langs. Unsupervised anomaly detection with generative adversarial networks to guide marker discovery. In Information Processing in Medical Imaging(IPMI), pp. 146-157, May 2017. doi: 2L.1077977-33331-59055 12.
Jastrzebski Stanislaw, Szymczak Maciej, Fort Stanislav, Arpit Devansh, Tabor Jacek, Cho Kyunghyun, and Geras Krzysztof. The break-even point on optimization trajectories of deep neural networks. In International Conference on Learning Representations (ICLR), 2020.
Bing Tu, Xianchang Yang, Nanying Li, Chengle Zhou, and Danbing He. Hyperspectral anomaly detection via density peak clustering. Puttern Recognition Letters, 129: 144-149, Jan. 2020.
Hao Wu and Saurabh Prasad. Semi-supervised deep learning using pseudo labels for hyperspectral image classification. JEEE Transactions Image Processing. 2753):
Mar. 2018.
Naoto Yokoya, Takehisa Yairi, and Akira Iwasaki. Coupled nonnegative matrix factorization un- mixing for hyperspectral and multispectral data fusion. IEEE Transactions on Geoscience and Remote Sensing, 50:528-531 2012.
Qiangqiang Yuan, Qiang Zhang. Jie Li, Huanfeng Shen, and Liangpei Zhang. Hyperspectral im- age denoising employing spatial-spectral deep residual convolutional neural network. IEEE Transactions on Geoscience and Remote Sensing, 5752):1255-1218, Feb. 2019.
Chuxu Zhang, Dongjin Song, Yuncong Chen, Xinyang Feng, Cristian Lumezanu, Wei Cheng, Jingchao Ni, Bo Zong, Haifeng Chen, and Nitesh V. Chawla. A deep neural network for un- supervised anomaly detection and diagnosis in multivariate time series data. In Proceedings of the AAAI Conference on Artificial Intelligence (AAAI), volume 33, 2019.
Zong, Qi Song, Martin Renqiang Min, Wei Cheng, Cristian Lumezanu, Daeki Cho, and Haifeng Chen. Deep autoencoding gaussian mixture model for unsupervised anomaly detection. In Inter- national Conference on Learning Representations (ICLR), 2018.
10


Reviews:
[{"review_id": "n8THYslpQwq", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "[Summary]\n\nIn this paper, the authors proposed the MTGAN framework, a GAN-based approach to the task of anomaly detection in hyperspectral images. The main idea behind this work is to exploit twin-least-square loss to perform background modeling in feature and image domains to alleviate the gradient vanishing problem of the previous GAN-based anomaly detection methods. Specifically, they proposed i) an MCM-aware strategy to construct the multi-scale priors, ii) a twin-least-square loss on GAN for training stabilization, and iii) an anomaly rejection loss for background estimation. The experiments on multiple benchmarks show the superiority of the MTGAN to the state of the art hyperspectral anomaly detection methods.\n\n[Strengths]\n+ The interesting idea of generating prior dictionaries to alleviate the lack of data for anomaly detection \n+ Tackling the GAN's gradient vanishing problem with the proposed twin-least-square loss sounds general-purpose and applicable to other applications\n+ Experiments are performed on multiple datasets\n\n[Weaknesses] \n- Contributions are not clearly and accurately stated (e.g., the method section is very hard to follow)\n- There is a lack of motivation and discussion as well as theoretical analysis on the proposed method\n- I am not very familiar with the task of anomaly detection in HSI, but the experimental results seem just marginally better than the other methods \n- The English could be improved\n\n[Detailed Questions/Comments]\n- More in-depth discussion of the method is necessary, For example: Why does it work? When does it fail? \n- Theoretical discussion is missing: e.g., there is no theoretically evidence is provided to support why twin-least-square loss stabilizes the GAN for training compared to the normal discriminator in GAN, etc. \n- A discussion on the computational costs (training time, convergence speed) and complexity of the proposed method could be helpful. \n- The authors observed that their proposed twin-least-square loss can alleviate the GAN's gradient vanishing problem: it would be nice to evaluate this phenomenon for the generation with GANs and inspect the quality of the generated images \n- Providing experimental results on the outlier detection datasets would be very helpful. (see [M1] or an extensive outlier detection benchmark)\n- Some important references on the adversarial-discriminator based methods for anomaly/novelty/abnormality/outlier detection are missing: e.g., [M2, M3]\n\n[Missing References]\n\n[M1] On the Evaluation of Unsupervised Outlier Detection: Measures, Datasets, and an Empirical Study by G. O. Campos, A. Zimek, J. Sander, R. J. G. B. Campello, B. Micenková, E. Schubert, I. Assent and M. E. Houle Data Mining and Knowledge Discovery 30(4): 891-927, 2016, DOI: 10.1007/s10618-015-0444-8\n\n[M2] Sabokrou, M., Khalooei, M., Fathy, M., Adeli, E.: Adversarially learned one-class classifier for novelty detection. CVPR(2018)\n\n[M3] Ravanbakhsh, M., et al.: Training adversarial discriminators for cross-channel abnormal event detection in crowds. arXiv preprint arXiv:1706.07680 (2017)", "extra_scores": {}, "extra_text": {}}, {"review_id": "J6H5XdZKupR", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "In this work, the authors find that some emerging methods based on generative adversarial network suffer from the problems of gradient vanishing and training instability with struggling to strike a balance between performance and training sample limitations. To solve this problem, they propose a novel method MTGAN, which introduces the MCM-aware strategy to construct multi-scale priors with precise second-order statistics, and imposes the twin-least-square loss on GAN to help improve the generative ability and training stability in feature and image domains, overcoming the gradient vanishing problem and finally establishes a pure and discriminative background estimation with a new anomaly rejection loss. At the same time, the author also used ablation experiments to prove the effectiveness of each part. But I think this paper is rather unclear, and some symbols are used without explanation, and the explanation of many formulas is not clear enough.\n\nOverall, I vote for 5: Marginally below acceptance threshold. The method is novel and the experiments prove the effectiveness of the method. My major concern is about the clarity of the paper, and the explanation of some formulas is not clear enough.\n\nPros:\n1.The MCM-aware strategy is proposed to reliably and adaptively generate prior dictionaries, which can solve the problem of insufficient samples.\n2.The twin-least-square loss is introduced into both the feature and image domains to overcome the gradient vanishing problem.\n\nCons:\n1.This paper is rather unclear, and some symbols are used without explanation, and the explanation of many formulas is not clear enough.\n2.Although experiments have proved the effectiveness of the method, it seems that the author rarely explains the reasons", "extra_scores": {}, "extra_text": {}}, {"review_id": "3icTXc16FuK", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper proposes a MCM-aware twin-least-squares GAN (MTGAN) model for hyperspectral anomaly detection. MTGAN introduce the MCM-aware strategy to construct multi-scale priors with precise second-order statistics, impose the twin-least-square loss on GAN and enforce a new anomaly rejection loss to establish a pure and discriminative background estimation.\nSome major comments are as follows: \n1.\tIn the MCM for background dictionary construction component, after generating covariance matrixes, how to obtain the background dictionary? Please state clearly how to construct the background dictionary. How to construct the background dictionary robustly which directly influence the following detection accuracy.\n2.\tIn the twin-least-square for background reconstruction component, the authors impose least square loss instead of cross entropy loss on the two discriminators. As we know, the GAN model usually use the cross entropy loss. It’s not innovation by using least square loss strategy instead of cross entropy loss.\n3.\tIn the experiment part, to make the proposed method MTGAN more convinced, the authors are suggested that using GAN-based anomaly detection methods and background dictionary construction based method as the compared methods.\n4.\tIt is suggested to state clearly in the whole framework, the components of framework should describe more concretely. In fact, we can use traditional methods to obtain the detection results after background dictionary construction. Why using the following GAN-based model to detect and what’s the advantage by using GAN-based method rather than traditional process.\n5.\tOverall, the MTGAN framework is lack of innovation. The component of MTGAN is independent. We can use the other background dictionary construction methods instead of MCM-aware strategy. Thus, how to design an end-to-end network architecture to solve hyperspectral anomaly detection problem sounds more interesting and novelty.", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: In this paper, the authors propose a MCM-aware twin-least-squares GAN (MTGAN) model for hyperspectral anomaly detection. The proposed method is somewhat novel, and the efficacy of the proposed method is validated through experiments. However, the clarity of the paper is low, and the explanation of some formulas is not clear enough. Therefore, the quality of the current version is below the acceptance threshold.  I encourage authors to update the paper based on the reviewer's comments and resubmit it to a future venue.
Decision: Reject