LAND-COVER MONITORING USING TIME-SERIES HYPERSPECTRAL DATA VIAFRACTIONAL-ORDER DARWINIAN PARTICLE SWARM OPTIMIZATIONSEGMENTATION
Naoto Yokoya1,2,3 and Pedram Ghamisi1,21

2

Remote Sensing Technology Institute (IMF), German Aerospace Center (DLR), Germany
Signal Processing in Earth Observation (SiPEO), Technische Universität München (TUM), Germany
3
Department of Advanced Interdisciplinary Studies, University of Tokyo, JapanABSTRACTThis paper presents a new method for unsupervised detection of multiple changes using time-serires hyperspectral data.
The proposed method is based on fractional-order Darwinianparticle swarm optimization (FODPSO) segmentation. The
proposed method is applied to monitor land-cover changes
following the Fukushima Daiichi nuclear disaster using multitemporal Hyperion images. Experimental results indicatethat the integration of segmentation and a time-series of hyperspectral images has great potential for unsupervised detection of multiple changes.
Index Terms— FODPSO-based segmentation, unsupervised change detection, multiple changes, land-cover monitoring, time-series analysis.
1. INTRODUCTION
Unsupervised change detection using multitemporal hyperspectral data has received great attention in the last decadesince ground reference is not available in many practical casesof change detection. In particular, the detection of multiplechanges is of great interest to utilize rich spectral information in hyperspectral data. One of the major approaches isbased on linear transformation, such as canonical correlation
analysis (CCA) [1] and principal component analysis (PCA)[2]. Multivariate alteration detection (MAD) analyzes differences between CCA variates and it has been successfully investigated in a wide range of applications using multitemporal
spectral images. Change vector analysis was also commonly
considered in various applications using multispectral images[3] and it has been extended to detection of multiple changes
for multitemporal hyperspectral images [4].
Image segmentation has been extensively studied owing
to its meaningful image representation, which enables objectbased analysis. Segmentation has been applied to change detection using high-resolution remote sensing images in [5].
This research has been partly supported by Alexander von Humboldt
Fellowship for postdoctoral researchers.

However, to the best of the authors’ knowledge, very few publications are available in the literature in which segmentation
is applied for unsupervised detection of multiple changes using time-series hyperspectral data.
Among different segmentation approaches, thresholdingbased techniques have gained attention in computer visionand remote sensing [6]. One common way to select optimal thresholds is the exhaustive Otsu criterion [7]. However,this approach is computationally expensive since for findingn − 1 optimal thresholds, it demands to evaluate its corresponding fitness n(L − n + 1)n−1 times [8]. For the sake ofsimplicity, the task of determining n − 1 optimal thresholds
for n-level image thresholding can be formulated as a multidimensional optimization problem. Recently, an extensionof particle swarm optimization (PSO) was proposed, which
considers two modifications: (1) fractional calculus to control the convergence rate of the algorithm [9, 6], and (2) multiple swarms of test solutions in which each swarm moves
just like an ordinary PSO with a few punishment and reward
rules. The FODPSO was successfully compared with PSOfamilies for some benchmark mathematical tests in [9] andfor thresholding-based segmentation in [6].
In this paper, we present a new method for unsupervised
detection of multiple changes based on FODPSO-based segmentation using a set of time-series hyperspectral data. The
proposed method is applied for the detection of land-coverchanges following the Fukushima Daiichi nuclear disaster using four temporal Hyperion images. The contributions of thiswork are twofold: 1) FODPSO-based segmentation is appliedfor change detection for the first time in the remote sensingcommunity; and 2) the potential of combining segmentation
and a time-series of hyperspectral images for land-cover monitoring is validated via a case study.
The rest of the paper is organized as follows: Section 2
describes materials used in this study. Section 3 is devoted tomethodology. Section 4 presents experimental results. Section 5 wraps up the paper by providing the main concluding
remarks.

Input...

Study Area

M-CCA

RMSE
...

...

FODPSO-based Segmentation

1 kmFukushima Daiichi
nuclear power plant

Segmentation Maps

Binary Change Detection...

Fig. 1: (Left) Location of the study area and (right) color composite of Hyperion data taken on April 29, 2012.

...

Output (Multiple Change Detection)...

2. MATERIALS
Fig. 2: Flowchart of the proposed method.
2.1. Study Area
3. METHODOLOGY
The study area is on the Fukushima Daiichi Nuclear Power
Plant (37◦ 250 1600 , 141◦ 10 5800 ), Fukushima, Japan, and its adjacent areas (Fig. 1). The Great East Japan Earthquake andthe tsunami damaged the power plant, resulting in meltdowns
and the leakage of radioactive materials. Since the disaster,
visiting on site is forbidden and storage tanks have been built
to store polluted water. Many of the abandoned areas havebeen covered by weeds.

2.2. Data and Preprocessing
The data set is composed of four temporal Hyperion images
acquired over the study area on April 29, 2012, May 6, 2013,
May 25, 2014, and May 2, 2015 [10]. All the images werefirst co-registered using geocoordinate information and further registered using phase-correlation-based image matchingif a misregistration of more than one pixel is detected. Imageregistration was performed only by shifting the whole image
at a pixel scale to avoid interpolation while keeping the same
spatial resolution. We used 156 bands (bands 8–57, 79–117,135–165, 183–185, 188–220) after removing bands that either include only zero values or influenced by strong water
vapor absorption. ATCOR was used for atmospheric correction [11]. Atmospheric normalization (relative process) was
further performed to mitigate residuals of non-optimal atmospheric correction. We adopted a pseudo-invariant-featurebased method, which corrects images using pixels whose
spectral reflectance is consistent over time. Color composite
images of the data set after atmospheric normalization are
shown in the first row of Fig. 3.

3.1. Change Detection via Segmentation
The proposed method consists of three parts: 1) dimensionality reduction; 2) segmentation; and 3) multiple change detection. Fig. 2 shows the flowchart of the proposed methodology,
which can be briefly explained as follows.
1. First multiset canonical correlation analysis (M-CCA)[12] is performed on a set of time-series hyperspectral
data. Lower-order canonical variates, which represent
more than 99% of the variance contained in the input
data, are used for further processing assuming that theycan explain major variations of the surface. Root meansquare errors (RMSEs) between a master image (e.g.,
the first temporal image) and the others are calculatedusing these canonical variables to obtain the degree ofchanges.
2. FODPSO-based segmentation is performed on both aset of M-CCA images and that of RMSE maps to obtain segmentation maps and binary change detection
maps, respectively. M-CCA images of each order orRMSE maps are spatially stacked before performing
segmentation. The level of segmentation for the MCCA images can be defined by taking into accountprior knowledge about an approximate number of landcover classes in the study area. The level of segmentation for the RMSE maps is set to two to conduct binarychange detection.
3. Multiple changes with various degrees of changes aredetected by generating the difference of segmentation
between the master image and the others. They are integrated with the binary change detection maps and fi-nally multiple change detection maps are obtained as
the output.
More details of FODPSO-based segmentation, which forms
the basis for the proposed method, are provided in the following section.
3.2. FODPSO-based Segmentation
As suggested in [6], the simplest and computationally mostefficient fitness function to define optimal thresholds inthresholding-based segmentation approaches is the one thatmaximizes the between-class variance. In order to find optimal thresholds tCo
j , the following fitness function can beformulated:
ϕCo =2Co
max σBCo (tj )Co
1 < tCo
1 < . . . < tn−1 < LCo={R,G,B}(1)where L is the intensity level in each RGB component of theinput image. These levels are in the range {0, 1, 2, . . . , L1}. For example, the number of intensity levels L for an 8-bit
image is between 0 and 255. Co represents the componentof the image. Here, we discuss the the approach for an RGB
image but it can be simply generalized to high dimensional
data.
FODPSO was proposed in [6] for image segmentation toaddress the main shortcoming of a simple PSO (i.e., the stagnation of particles around sub-optimal solutions). In moredetail, the following modifications have been taken into account:.

1. FODPSO is composed of many simultaneous parallel
PSO on the same test problem and considers a simple
natural selection mechanism. When a search tends to a
sub-optimal solution, the search in that area is simply
discarded and another area is searched instead. For
more information regarding how these rewards andpunishments can be applied, please see [6, 8, 13].
2. Fractional calculus is used to control the convergence
rate of the algorithm. This method has been further investigated for gray scale and hyperspectral image segmentation and feature selection in [6, 14] and [15], respectively. The main advantage of fractional calculus
is that while an integer-order derivative just implies afinite series, the fractional-order derivative requires aninfinite number of terms. Therefore, integer derivatives are considered as “local” operators, while fractional derivatives have, implicitly, a “memory” of all
past events, which is useful to control the dynamic ofeach swarm.
In order to mathematically derive the FODPSO, in eachstep t, the fitness value (i.e., Eq. (1)) is estimated. To model
the swarm, each particle n flies in a multidimensional search

space considering the position (xn [t]), and velocity (vn [t]),which are highly dependent on local best (x̆n [t]) and global
best (ğn [t]) information as follows:
vns [t + 1] =(2)

wns [t + 1] + ρ1 r1 (ğns [t] − xsn [t]) + ρ2 r2 (x̆sn [t] − xsn [t]),wns [t + 1] =(3)11αvns [t] + α(1 − α)vns [t − 1] + α(1 − α)(2 − α)vns [t − 2]261+ α(1 − α)(2 − α)(3 − α)vns [t − 3].
24The superscript ‘s’ shows the number of each swarm. Thecoefficients ρ1 and ρ2 control the inertial influence of theglobal best and the local best, respectively. In general, ρ1and ρ2 are constant integer values, which represent “cognitive” and “social” behavior of each particle with ρ1 + ρ2 < 2[16]. The parameters r1 and r2 are random vectors in which
each unit generally is a uniform random number between 0and 1. These random vectors are considered to increase therandomness behavior of each particle to increase the diversityof the swarm in order to avoid getting trapped in local optimum. The fractional coefficient α can be considered as aweight to control the influence of past events for determining
a new velocity, 0 < α < 1 [6]. A small α causes the particles ignore their previous activities. In this way, the system
dynamics will be ignored and become susceptible to get stuck
in a local solution (i.e., exploitation behavior). In contrast, alarge value of α forces the particles to have a more diversified behavior, which allows exploration of new solutions andimproves the long-term performance (i.e., exploration behavior). However, if the exploration level is too high, then thealgorithm may take longer to find the global solution. Basedon [16], a good α value can be traced in the range of 0.6 to0.8. In this work, this value is set to 0.65.
4. EXPERIMENTAL RESULTS
Fig. 3 shows the experimental results obtained by our method
using the four Hyperion images. Color composites of reflectance data, M-CCA variates, FODPSO-based segmentation, and final change detection maps are shown in rows from
top to bottom of Fig. 3(a). M-CCA images are correlated
each other compared to those of reflectance data as shownin the second row of Fig. 3(a). This implies that M-CCA
successfully extracts surface features as the low-order canonical variates by exploring common spatial patterns included
in all the multitemporal images. Residual signals that appear
in a specific temporal image due to various atmospheric conditions can be excluded from the low-order canonical variates, which can be seen from the images taken on May 6,
2013. Consistent segmentation results were obtained among
the multitemporal images by performing thresholding-based

Fig. 3: (a) Color composites of reflectance data, M-CCA, FODPSO segmentation, and detected changes from top to bottom
rows obtained from four temporal Hyperion data. (b) Enlarged images of change detection maps at areas A and B in (a) andcorresponding high-resolution images from Google Earth (Google, DigitalGlobe).
segmentation to the spatially stacked M-CCA images. Owingto this property, it is easy to discriminate multiple changes bysubtracting segmentation results.
In the fourth row of Fig. 3(a), multiple changes are clearlyshown in different colors. For example, light green pixels
around the power plant indicate changes from vegetation to
man-made objects, and hot pink pixels in the Northeast correspond to changes from soil to vegetation due to the increaseof weeds. Fig. 3(b) shows the enlarged images of change
detection maps at areas A and B in Fig. 3(a) and their corresponding high-resolution images obtained from Google Earth(Google, DigitalGlobe). These high-resolution images weretaken on January 31, 2012, March 12, 2013, March 24, 2014,
and June 1, 2015, which are the closest acquisition dates withthose of the Hyperion images. In the area A, it can be seenthat deforestation and construction of man-made objects areclearly detected by the proposed method. The area B includes
a schoolyard, where bags filled with radioactive waste fromdecontamination efforts were started to be temporarily storedin 2015. This change is also detected by the proposed method.

segmentation was proposed to use for unsupervised change
detection. The potential of our method was validated via
a case study that analyzes land-cover changes following theFukushima Daiichi nuclear disaster using four temporal Hyperion images. Experimental results showed that the presented method is capable of detecting multiple changes, suchas deforestation, construction of man-made objects, and increasing weeds in the considered study area. Our future research includes numerical validation and an extension of themethod to handle multisensor time-series data.
6. REFERENCES[1] M. Frank and M. Canty, “Unsupervised change detection for hyperspectral images,” in Proc. 12th JPL Airborne Earth Sci. Workshop, 2003.

5. CONCLUSION[2] V. Ortiz-Rivera, M. Vélez-Reyes, and B. Roysam,“Change detection in hyperspectral imagery using temporal principal components,” in Algorithms Technologies for Multispectral, Hyperspectral, and Ultraspectral
Imagery XII, 2006, vol. 6233.

In this paper, we presented a new method for unsupervised
detection of multiple changes for land-cover monitoring using
time-series hyperspectral data. In this paper, FODPSO-based[3] W. A. Malila, “Change vector analysis: An approachfor detecting forest changes with landsat,” in 6th Annu.
Symp. Mach. Process. Remotely Sensed Data, 1980.

[4] S. Liu, L. Bruzzone, F. Bovolo, M. Zanetti, and P. Du,“Sequential spectral change vector analysis for iteratively discovering and detecting multiple changes in hyperspectral images,” IEEE Trans. Geosci. Remote Sens.,
vol. 53, no. 8, pp. 4363–4378, August 2015.
[5] L. Wu, Z. Zhang, Y. Wang, and Q. Liu, “A segmentation based change detection method for high resolution
remote sensing image,” Pattern Recognition, vol. 483,pp. 314–324, 2014.
[6] P. Ghamisi, M. S. Couceiro, J. A. Benediktsson, andN. M. F. Ferreira, “An efficient method for segmentation of images based on fractional calculus and natural selection,” Expert Syst. Appl., vol. 39, no. 16, pp.
12407–12417, 2012.
[7] N. Otsu, “A threshold selection method from gray-level
histogram,” IEEE Trans. Syst. Man Cyber., vol. 9, pp.
62–66, 1979.
[8] M. S. Couceiro and P. Ghamisi, Fractional Order Darwinian Particle Swarm Optimization: Applications andEvaluation of an Evolutionary Algorithm, Springer,Londen, 2015.
[9] M. S. Couceiro, R. P. Rocha, N. M. F. Ferreira, andJ. A. T. Machado, “Introducing the fractional order Darwinian PSO,” Sig., Image and Vid. Process., vol. 102,no. 1, pp. 8–16, 2007.
[10] N. Yokoya and X. X. Zhu, “Graph regularized coupledspectral unmixing for change detection,” in 7th WHISPERS, 2015.
[11] R. Richter and D. Schläpfer, Atmospheric/Topographic
Correction for Satellite Imagery: Atcor 2/3 Users
Guide, 2015.
[12] J. R. Kettenring, “Canonical analysis of several sets ofvariables,” Biometrika, vol. 58, pp. 433 – 451, 1971.
[13] J. A. Benediktsson and P. Ghamisi, Spectral-SpatialClassification of Hyperspectral Remote Sensing Images,Artech House Publishers, INC, Boston, USA, 2015.
[14] P. Ghamisi, M. S. Couceiro, F. M.L. Martins, andJ. A. Benediktsson, “Multilevel image segmentation approach for remote sensing images based on fractional–order Darwinian particle swarm optimization,” IEEETrans. Geos. Remote Sens., vol. 52, no. 5, pp. 2382–2394, 2014.
[15] P. Ghamisi, M. S. Couceiro, and J. A. Benediktsson, “Anovel feature selection approach based on FODPSO andSVM,” IEEE Trans. Geos. Remote Sens., vol. 53, no. 5,
pp. 2935–2947, 2015.

[16] M. S. Couceiro, F. M. Martins, R. P. Rocha, and N. M.
Ferreira, “Mechanism and convergence analysis of amulti-robot swarm approach based on natural selection,”
Jour. Intell. Rob. Sys., pp. 1–29, 2014.