Thisarticlehasbeenacceptedforinclusioninafutureissueofthisjournal.Contentisfinalaspresented,withtheexceptionofpagination.
IEEE JOURNAL OF SELECTED TOPICS IN APPLIED EARTH OBSERVATIONS AND REMOTE SENSING

1Hyperspectral Tree Species Classification of JapaneseComplex Mixed Forest With the Aid of Lidar Data
Tomohiro Matsuki, Naoto Yokoya, Member, IEEE, and Akira Iwasaki

Abstract—The classification of tree species in forests is an
important task for forest maintenance and management. With theincrease in the spatial resolution of remote sensing imagery, individual tree classification is the next target of research area for theforest inventory. In this work, we propose a methodology involving
the combination of hyperspectral and LiDAR data for individual tree classification, which can be extended to areas of shadow
caused by the illumination of tree crowns with sunlight. To remove
the influence of shadows in hyperspectral data, an unmixing-based
correction is applied as preprocessing. Spectral features of trees
are obtained by principal component analysis of the hyperspectral
data. The sizes and shapes of individual trees are derived from the
LiDAR data after individual tree-crown delineation. Both spectral
and tree-crown features are combined and input into a support
vector machine classifier pixel by pixel. This procedure is applied
to data taken over Tama Forest Science Garden in Tokyo, Japan,
to classify it into 16 classes of tree species. It is found that both
shadow correction and tree-crown information improve the classification performance, which is further improved by postprocessing
based on tree-crown information derived from the LiDAR data.
Regarding the classification results in the case of 10% training
data, when using the random sampling of pixels to select training samples, a classification accuracy of 82% was obtained, while
the use of reference polygons as a more practical means of sample
selection reduced the accuracy to 71%. These values are, respectively, 21.5% and 9% higher than those that are obtained using
hyperspectral data only.
Index Terms—Classification, data fusion, forest, hyperspectral
data, LiDAR.

I. I NTRODUCTIONT

HE sustainable management of forests is an importanttask from the viewpoint of the environment and economy. Forest inventory, which is the systematic collection ofdata and information, such as the tree species, heights, diameters at breast height, site qualities, ages and, so forth, is used
to help sustainable forest ecosystems. Therefore, tree speciesclassification is an important issue in the study of forest management and land cover [1]. Periodic field surveys by experts
require a large amount of time and human resources. Since

Manuscript received September 30, 2014; revised February 03, 2015;
accepted March 13, 2015. The hyperspectral data and LiDAR data were
provided by Japan Space Systems (JSS). This work was supported under
the contract with the National Institute of Advanced Industrial Science andTechnology (AIST) as part of the Hyperspectral Imager Suite (HISUI) project.
T. Matsuki is with the Department of Aeronautics and Astronautics,
University of Tokyo, Tokyo 113-8656, Japan (e-mail: matsuki@sal.rcast.utokyo.ac.jp).
N. Yokoya and A. Iwasaki are with the Research Center for Advanced
Science and Technology, University of Tokyo 153-8904, Tokyo, Japan.
Color versions of one or more of the figures in this paper are available online
at http://ieeexplore.ieee.org.
Digital Object Identifier 10.1109/JSTARS.2015.2417859

remote sensing enables wide-area observation at a single time,
its contribution to tree species classification over forests is
expected [2], [3]. By monitoring forests frequently, remotesensing is expected to abstract information that will help forest management to preserve biodiversity, which is an important
index used to evaluate environmental health. Remote sensing
is also important for estimating biomass and the natural carbon
sink to study regional environments.
In recent years, many researchers have worked on the classification or identification of tree species using remote sensing
techniques. Since airborne hyperspectral sensors have several
tens of observation bands and high spatial/spectral resolution, acontinuous spectrum can be obtained that enables more detailed
analysis of the specified spectral profiles of individual trees at a
pixel level [4]–[6]. Spectral–spatial approaches, such as the useof morphological profiles, are effective for the accurate classification of hyperspectral data using both spectral and spatial
features, which are often concatenated into feature vectors andclassified by a support vector machine (SVM) [7]. In a forest,
the pixel values corresponding to a tree crown are influenced
by the illumination of sunlight and the tree-crown shape, whichmake it difficult to extract spatial features of tree crowns only
based on hyperspectral imagery without suitable preprocessing
to take account of variations in the illumination.
Light detection and ranging (LiDAR) is a powerful tool
for obtaining canopy height models (CHMs) by using returnpulses from above the tree crown and terrain surface [8]. Since
LiDAR is an active remote sensing instrument, it is unaffectedby the illumination conditions of sunlight. Using CHMs, individual tree heights, crown areas, and shapes are obtained, which
are used in the segmentation of forests into tree crowns [9].
Such information is used to identify species of individual trees
only using LiDAR data [10]. Full-waveform LiDAR records
the complete waveform of a backscattered signal echo, which
is used to reconstruct tree structures, such as the stem volume and diameter at breast height [11]. Therefore, tree-crownfeatures, such as the tree size and shape, obtained from LiDARderived CHMs are also expected to be useful in tree speciesclassification.
In recent years, the fusion of hyperspectral and LiDAR datafor classification has been studied [12]–[19]. Many researchers
use LiDAR data for preprocessing to find trees by thresholding CHMs. The illumination condition is also obtained from
CHMs, and thus the masks of shadow areas are created [14].
Raw height data and pixel-level statistics are used as features of tree crowns, which improve the classification accuracy[15], [16]. Individual tree-crown delineation is effective for

1939-1404 © 2015 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.
See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.

Thisarticlehasbeenacceptedforinclusioninafutureissueofthisjournal.Contentisfinalaspresented,withtheexceptionofpagination.
2

IEEE JOURNAL OF SELECTED TOPICS IN APPLIED EARTH OBSERVATIONS AND REMOTE SENSING

Fig. 1. (Left) Whole RGB image obtained using hyperspectral data. (Right) Enlarged RGB images of study areas A and B with CHM derived from LiDAR data.

preprocessing and postprocessing in the classification procedure [17]. By treating pixels in a tree crown together, similar
merits to those of spectral–spatial techniques are expected.
Volumetric canopy profiles [18] and structural metrics obtained
from point clouds by waveform LiDAR [19] further improvethe capture of tree-crown information.
In this work, we propose a methodology to classify tree
species by coupling hyperspectral and LiDAR data. After
shadow correction is applied to hyperspectral data to decrease
the common source of intensity variability, spectral features are
obtained by extracting principal components. Tree-crown features, such as tree heights, sizes, and shapes, are derived bytree-crown delineation and shape fitting to CHMs. Spectral andtree-crown features are fused and then input to the SVM classifier. A crown-preserving smoothing filter based on tree-crowninformation is applied as a postprocessing technique to improvethe classification. We apply the procedure to the challenging
tree species classification problem of Japanese complex mixed
forests and show its effectiveness.
II. M ATERIALS
A. Study Area
Japan is located in East Asia and consists of subarctic,
temperate, and subtropical climate zones. Therefore, Japanese
forests contain a wide range of tree species. The study area is
Tama Forest Science Garden in the western region of Tokyo[20], which is managed by the Forestry and Forest Products
Research Institute, Japan. This forest has an area of 56 ha
and includes a special region containing many tree species
found in gardens. Since broadleaf evergreen trees from thewarm temperate zone, deciduous trees from the cool temperatezone, and conifers from the subarctic zone coexist in the garden, it is suitable for investigating tree species classification in
Japan.

B. Remote Sensing Data
The left image in Fig. 1 is a color image obtained using
hyperspectral data from the 654, 552, and 449 nm bands, corresponding to red, green, and blue (RGB), respectively. The
right images in Fig. 1 show enlarged RGB images of study
areas A and B and the respective CHM images derived from
LiDAR data. The data were obtained by airborne sensors on
September 10, 2009, and provided by Japan Space Systems.
Since the data were obtained in early fall, all the trees had
green leaves, making it difficult to use their colors for analysis.
The hyperspectral data were obtained by CASI-3 with 72 bands
covering wavelengths of 400–1050 nm. The ground sampling
distance of the hyperspectral data was 1 m, and the data were
resampled to obtain an orthorectified data cube using a digitalsurface model (DSM) derived from LiDAR data. To avoid theeffects of the bidirectional reflectance distribution function, thedata in the target areas were obtained along the nadir path. The
acquired data were processed to obtain radiance data using the
instrumental characteristics and then converted to reflectancedata using fast line-of-sight atmospheric analysis of spectral
hypercubes (FLAASH) software.
The LiDAR data were obtained from LMS-Q560 (Riegel),which observes the first and last return pulses with a groundsampling distance of 0.5 m. The CHM used was the difference between the digital elevation model (DEM) and the DSM,
and was obtained with a sampling distance of 1 m. The CHM
represents the heights of trees excluding terrain effects.

Thisarticlehasbeenacceptedforinclusioninafutureissueofthisjournal.Contentisfinalaspresented,withtheexceptionofpagination.
MATSUKI et al.: HYPERSPECTRAL TREE SPECIES CLASSIFICATION OF JAPANESE COMPLEX MIXED FOREST

3

Fig. 2. Ground reference of 16 major species in the study areas and their average spectra obtained from de-shadowed hyperspectral data.

C. Field Survey Data
A field survey was carried out in this area tree by tree, byhuman inspection. Fig. 2 shows the distribution of the groundreference of 16 major species in the study areas and theiraverage spectra obtained from de-shadowed hyperspectral data.
The species and the crown shapes of some trees were investigated by experts. The field survey data were composed of thetree species and a shapefile showing a polygon of the tree crownfor individual trees. The polygon of the tree crown was overlaid
on the remote sensing data to represent the ground reference.
Although there are more than 150 species of trees in this area,
most of them occupy only a few pixels. We chose 16 major
species with sufficient ground reference data for our investigation. Table I shows the class names and the numbers of trees andpixels. The study areas comprise typical mixed forest consistingof conifers and broadleaf trees.
III. M ETHODOLOGY
The proposed method consists of four parts: 1) shadow
correction of the hyperspectral data; 2) individual tree-crowndelineation from the LiDAR-derived CHM; 3) classification by
an SVM; and 4) postprocessing. Fig. 3 shows the classification
flow for the proposed methodology.
A. Shadow Correction
Shadows are a common source of intensity variability, whichleads to misclassification. A great deal of attention has been

paid to removing shadows from passive optical remote sensing
images. Since the structures of trees are complex owing totheir complicated multilayers, the shadow correction of a forest
is a challenging task. Generally, the shadow correction process comprises two steps: 1) detection and 2) de-shadowing.
One common approach is the joint use of matched-filter-based
thresholding for detection and an atmospheric radiative transfer model for de-shadowing [21], [22]. Several parameters
for thresholding and atmospheric conditions must be carefully defined in this approach. When a DSM is available, ray
tracing is effective for detecting shadows [14], [23], [24]; however, it is necessary to identify the area containing exactly thesame objects as those in the shadow area for de-shadowing,
which may be challenging in the case of a forest. We adopt
an unmixing-based approach for shadow correction [25], since
it can be fully automated and can effectively improve the classification accuracy without requiring the blue-skewed skylightillumination of the shadow.
First, the shadow is assumed to be a “black” (zeroreflectance) endmember. Next, endmember spectra obtainedby vertex component analysis (VCA) [26] are assumed to
be unshadowed. VCA is a data-driven approach with a given
number of endmembers, and the number of endmembers in thisstudy is determined to be 15 by using HySime [27]. Abundance
fractions are estimated using the fully constrained least-squares(FCLS) method satisfying sum-to-one and nonnegativity
constraints [28]. The reflectance spectra are approximately
de-shadowed by dividing the spectra by one minus the shadow
abundance.

Thisarticlehasbeenacceptedforinclusioninafutureissueofthisjournal.Contentisfinalaspresented,withtheexceptionofpagination.
4

IEEE JOURNAL OF SELECTED TOPICS IN APPLIED EARTH OBSERVATIONS AND REMOTE SENSINGTABLE IC LASS NAMES OF M AJOR T REE S PECIES W ITH THE N UMBERS OF T REES AND P IXELS OF R EFERENCE DATA

Fig. 3. Concept of data fusion and data processing flow.

B. Individual Tree-Crown Delineation
Individual tree-crown delineation is an important techniquefor extracting information from forests. Many researchers
have investigated the use of individual tree-crown delineation
methods for passive remote sensing applications, such as region
growing methods, valley following methods, and so forth [29]–[36]. In this study, we use a region growing method to obtain
the LiDAR-derived CHM [37], [38]. First, to reduce noise, weapply a Gaussian smoothing filter with a kernel size of threepixels to the CHM, which is selected so as not to miss small

trees. Then, we find local maxima in the smoothed image,which include nontree objects. To remove these objects, thelocal maxima whose normalized difference vegetation indices(NDVIs) are lower than 0.5 or whose heights are less than 1 m
are excluded [14], [19]. The remaining objects correspond to
the tops of trees. Regions corresponding to tree crowns aregrown from the local maxima. If neighboring pixels satisfy certain conditions, a region is formed from these pixels and theneighboring pixels become the next starting pixels. The conditions are set that the height of each neighboring pixel is lessThisarticlehasbeenacceptedforinclusioninafutureissueofthisjournal.Contentisfinalaspresented,withtheexceptionofpagination.
MATSUKI et al.: HYPERSPECTRAL TREE SPECIES CLASSIFICATION OF JAPANESE COMPLEX MIXED FOREST

than that of the starting pixel and greater than 1 m, so that
regions do not reach the ground. This growth step is repeateduntil there are no more starting pixels. After the growth step,
each region is adjusted so that it is star-shaped to exclude thepixels that are located outside the region centered at the treetop [37], [38]. Finally, overlapping regions are assigned to theregion containing the nearest local maximum in the projected
two-dimensional coordinates.
C. SVM Classification
We adopt the SVM classifier with the radial basis function (RBF) kernel to classify the pixel level using spectral andtree-crown information as input features. SVMs are widely
acknowledged to be effective algorithms for pixel-level classification using hyperspectral data [39], including tree speciesclassification [15]–[17]. The number of spectral bands of thehyperspectral data is 72, but the latent dimension is generally
lower. Therefore, the principal components of the de-shadowed
hyperspectral data are used as the spectral features; we input
15 principal components that represent more than 99.5% of thevariance contained in the data. Then, the crown height, size, andcurvature are used as tree-crown features. The height and size
are defined by the highest value in the CHM and the number ofpixels in each crown, respectively. The curvature is estimatedby the fitting functionz = H − arc ,(1)

where we assume axial symmetry and use cylindrical coordinates (r, z). H denotes the height of the tree, and a and c
are estimated values. The Levenberg–Marquardt algorithm isused to solve the nonlinear least-squares curve-fitting problem.
If a pixel is not included in any of the crowns, the height isthe pixel’s own value, the size is 1, and the curvature is 0.
The optimal parameter of the RBF kernel is defined by crossvalidation [40]. We coded this method using MATLAB with
LIBSVM, which is a library of SVMs that supports multiclassclassification, using the one-against-one method [41].
D. Postprocessing
Although we use tree-crown information derived from themapping of LiDAR data as the input feature, the result of pixellevel classification includes salt-and-pepper noise owing to thespectral variability in each crown. Postprocessing in the spatial
domain is effective for improving the classification outcome[42]. A majority voting procedure for the pixels in each tree
crown is a common postprocessing technique when a rasterized CHM is available [17]; however, the modification entirely
depends on the accuracy of individual tree-crown delineation.
The use of a smoothing filter is a popular postprocessing technique to obtain better classification results. Here, we proposea crown-preserving smoothing filter inspired by bilateral filters[43], [44].
The class label at each pixel in an image is replaced by aweighted vote of the class from neighboring pixels with theaim of reducing the salt-and-pepper noise due to misclassification while preserving the crown edges. This weight can

5

be based on a Gaussian distribution defined by the Euclidean
distance between pixels and crown segments instead of radiometric differences. If a filtered pixel belongs to the same crownas neighboring pixels, the weights of all these pixels are one;otherwise, they are smaller than one. Since the classification
results are categorical values (1, . . . , K), we reclassify the pixel
into the class in which the weighted voting score is highest. This
procedure is formulated as


w
w


m2 + n2ˆ
exp −I(k) P (m, n)f (i, j) = argmax
2σ 2k∈K n=−w m=−w1, iff (i + m, j + n) = k
I (k) =
0,otherwise1,
if p(i, j) = p(i + m, j + n) > 0
P (m, n) =α(0 ≤ α ≤ 1), otherwise(2)
where f is the class label obtained by the pixel-level classification and p is the index of the crown. If a pixel does not belong
to any crown, then p = 0. The parameter α controls the consideration of crown segments, i.e., α → 0 means that the weightis calculated considering only the pixels in a crown, whereas
α → 1 implies no consideration of the crowns. Therefore, theproposed filter with the parameters w → ∞, σ → ∞, and α →0 is equivalent to majority voting. α is set to 0.5 to assign moderate importance to consider crown segments. σ is defined by
setting the full width at half maximum of the Gaussian distribution which is equal to w. The window of the proposed filter
is set to the average size of tree crowns obtained by individualtree-crown delineation, which is 11 × 11 pixels in this study.
IV. R ESULTS AND D ISCUSSION
A. Shadow Correction and Crown Delineation
Fig. 4(a) and (b) shows the de-shadowed image and the
shadow map in study area A obtained by unmixing-based
shadow correction, respectively. The southeast side of the trees,
which is illuminated by sunshine, is bright, whereas the northwest side is darker, indicating that the amount of shadow islarger. Note that the changes in illumination resulting from theuse of the DSM are also corrected. The RGB image in Fig. 4(a)shows that the shadow effect is corrected for most shadows,
which appear as flat patterns in individual tree crowns owing
to shadow correction, showing that a better reflectance image is
obtained than that in Fig. 1. However, the correction of shadowson roads and houses is not satisfactory, although they are outside the target area in this study. The present shadow correction
method does not take account of spectral dependence, which
does not represent the actual situation and leads to the spectral
distortion of roads and houses. Furthermore, because most of
the pixels are located in the forest area, the regions of roads andhouses are not selected as endmembers in the unmixing process. Further improvement in shadow correction by using ray
tracing will lead to better restoration of the spectral profile ofreflectance.
After applying a smoothing filter to the original CHM, thelocal maxima of the CHM are found and then individual tree

Thisarticlehasbeenacceptedforinclusioninafutureissueofthisjournal.Contentisfinalaspresented,withtheexceptionofpagination.
6

IEEE JOURNAL OF SELECTED TOPICS IN APPLIED EARTH OBSERVATIONS AND REMOTE SENSING

Fig. 4. (a) RGB image of de-shadowed hyperspectral data; (b) shadow map; and (c) results of individual tree-crown delineation overlaid on CHM with (d) groundreference in study area A.

crowns are selected using the thresholds of CHM > 1 m and
NDVI > 0.5. Fig. 4(c) and (d) shows the result of individualtree-crown delineation and its ground reference in study area A,
both overlaid on the CHM, respectively. The red dots represent
the tops of trees that correspond to local maxima. It is found thatthe local maxima corresponding to houses in the upper region of
the figures are excluded by this procedure. By comparison withthe ground reference data, several problems in the extraction ofindividual tree crowns are found. Small trees with a high density located in the southwest area are not extracted. Moreover,large trees have several local maxima in the tree crown, which
originate from the region growing method, resulting in a single
tree being separated into several small tree crowns. These problems are related to the smoothing process and further improvements are needed in adaptive data processing. Furthermore,
small trees under a tall tree are not distinguished because theirlocal maxima are not found. Although waveform LiDAR cansolve this problem [9], it is difficult to distinguish a layered
structure using the proposed method.
B. Tree-Crown Classification
We applied the SVM classifier to the input features obtainedby the proposed method and compared the classification resultswith those obtained with and without shadow correction andthe use of tree-crown information. Since the number of trees islimited for each tree species in the ground reference, e.g., lessthan 15 for 9 of the species, it is very challenging to obtain treecrown features via treewise sampling using reference polygons.
In this work, we first adopted the random sampling of pixels
to select training samples, assuming that almost all the treecrown variations are included in the training set. The training
set size is given as a percentage of the entire ground reference
and the remaining reference pixels were used for validation ineach trial. Fig. 5 shows the classification maps and the overall
accuracy (OA) of one trial when the training set size is 10%.
The accuracy of the classification map obtained by the proposed method is greater than those of the maps obtained using
spectral features only, de-shadowed spectral features only, andspectral and tree-crown features. In study area A, owing toshadow correction, we can see an improvement in the classification results in the central area and in the area of Koyama’s
spruce in the northeast. Furthermore, the salt-and-pepper noise
is mitigated by the proposed method by using the tree-crownfeatures. Table II shows the confusion matrix of the proposed
method along with the producer’s accuracy (PA) and user’s
accuracy (UA) for all classes. The PAs for Japanese cypress
and cedars are reasonably high owing to the use of a large number of training samples covering a wide range of within-classvariability; however, this leads to the misclassification of pixels
belonging to other classes, which is indicated by their relatively
low PAs.
Fig. 6 shows the PA of each class for the four methods, which
is obtained by averaging the results of 10 trials. The proposed

Thisarticlehasbeenacceptedforinclusioninafutureissueofthisjournal.Contentisfinalaspresented,withtheexceptionofpagination.
MATSUKI et al.: HYPERSPECTRAL TREE SPECIES CLASSIFICATION OF JAPANESE COMPLEX MIXED FOREST

7

Fig. 5. Classification map of (a) ground reference and classification maps obtained using; (b) de-shadowed spectral and tree-crown features; (c) spectral and treecrown features, (d) de-shadowed spectral features only; and (e) spectral features only. Classification maps after (f) majority voting and (g) applying the proposed
filter to the map in (b).

method is more accurate than the other methods for many of
the classes. In particular, the joint use of the tree-crown features derived from the CHM improves the results for California
incense cedars, Loblolly pines, Koyama’s spruces, and Oriental
raisin trees, whereas those obtained using only spectral features show lower accuracies. The overall accuracy is shown in
Fig. 7(a) for the six methods with training set sizes of 1% to
30% to examine the effects of the use of shadow correction,tree-crown features, and feature extraction from the CHM in
addition to shadow correction. For each size of the training set,
the average accuracy of 10 trials is plotted. When the training set size is larger than 10%, shadow correction and theuse of tree-crown features increase the accuracy by 2.5% and12%, respectively. These results indicate that shadow correction based on unmixing has a positive impact on tree speciesclassification, and that the tree-crown features are effective for
improving the pixel-level classification, which is difficult todiscriminate using only spectral features.
Fig. 7(a) also shows a comparison of the classification results
obtained by individual tree-crown delineation of the groundreference collected by experts, the region growing image processing used in this study and the original CHM as well as theresults obtained without the use of the CHM to demonstrate theimpact of feature extraction using the CHM in addition to thede-shadowed spectral features. The proposed method clearly
outperforms the method using the original pixel values of theCHM. The use of the ground reference obtained by individualtree-crown delineation increases the accuracy by 4% comparedwith that of the proposed method, which implies that better
individual tree-crown delineation may improve classification
results.
C. Postprocessing
Fig. 5(f) and (g), respectively, shows the postprocessed maps
obtained by majority voting and using the proposed filter with
10% of the ground reference as training samples. The proposed
filter can smooth results while preserving smaller structures
more effectively than majority voting. Fig. 7(b) shows theoverall accuracy of the classification results obtained with

Thisarticlehasbeenacceptedforinclusioninafutureissueofthisjournal.Contentisfinalaspresented,withtheexceptionofpagination.
8

IEEE JOURNAL OF SELECTED TOPICS IN APPLIED EARTH OBSERVATIONS AND REMOTE SENSINGTABLE IIC ONFUSION M ATRIX

postprocessing. The proposed filter improves the overall accuracy by 7%, whereas majority voting increases the accuracy
by 1%. Therefore, the proposed methodology increases theaccuracy by a total of 21.5% when using 10% training data.
Although the results obtained by postprocessing depend on thedelineation, the proposed filter can preserve small structureseven if the delineation is incomplete. For example, in the west
part of study area A in Fig. 5(f), the majority voting misclassified a small area of Japanese bigleaf magnolias surroundedby Japanese cedars into the latter because the size of individual tree crowns of the latter species was overestimated. In
contrast, the proposed method correctly modified class labelsof pixels in this area because the number of pixels classified as Japanese bigleaf magnolias in each delineated crown iscomparable to the number classified as Japanese cedars. The
result shows that the proposed crown-preserving smoothing filter has the potential to moderately improve classification resultsby considering delineated crowns even though they include
errors.
D. Accuracy Assessment Using Reference Polygons
When the validation samples are allowed to be in the same
polygons (i.e., trees in this study) as the training samples,the classification accuracy can be overestimated owing to thestrong correlation between the training and validation data[45]. To perform a more practical assessment, we conducted
experiments involving treewise sampling by separating the
ground reference data tree by tree with a ratio of four to one
between the training and validation data. The training set was
randomly sampled on a pixel basis and its size is given as a percentage of the separated training data. Fig. 8 shows the overall
accuracy obtained with this validation scenario. In Fig. 8(a), therelationship between the six methods is similar to that shown in
Fig. 7(a) and the proposed method gives the best result with a
3%–4% increase in the accuracy compared with that obtainedusing spectral features only. However, the overall accuracy for
all the methods is lower than that in Fig. 7(a) obtained using
pixel-based random sampling for accuracy assessment. Thisimplies that sufficient training data are required to include thewithin-class variation of tree-crown features for each class toimprove the classification performance of the proposed method.
As shown in Fig. 8(b), the proposed filter increases the accuracy by 5%, whereas majority voting reduces the accuracy. Our
approach increases the accuracy by a total of approximately
9%, which proves the efficacy of the proposed methodology forpractical assessment. This work shows that hyperspectral datawith tree-crown features are effective for tree species classification in Japanese forests, in which conifers and broadleaf trees
result in complex biodiversity.

Thisarticlehasbeenacceptedforinclusioninafutureissueofthisjournal.Contentisfinalaspresented,withtheexceptionofpagination.
MATSUKI et al.: HYPERSPECTRAL TREE SPECIES CLASSIFICATION OF JAPANESE COMPLEX MIXED FOREST

9

Fig. 6. PA of 16 major tree species.

Fig. 7. Overall accuracy of tree species classification. (a) Comparison of six methods to examine effects of the use of shadow correction, tree-crown features, andfeature extraction from CHM in addition to shadow correction. (b) Effects of postprocessing.

V. C ONCLUSION
We presented a methodology to classify tree species using
hyperspectral data and a LiDAR-derived CHM. Shadow correction is applied to hyperspectral data, and information on

the tree size and shape is obtained after individual tree delineation of the CHM. The SVM classifier is used for pixel-levelclassification, which is followed by postprocessing using acrown-preserving smoothing filter. The proposed method wasThisarticlehasbeenacceptedforinclusioninafutureissueofthisjournal.Contentisfinalaspresented,withtheexceptionofpagination.
10

IEEE JOURNAL OF SELECTED TOPICS IN APPLIED EARTH OBSERVATIONS AND REMOTE SENSING

Fig. 8. Overall accuracy of tree species classification using reference polygons. (a) Comparison of six methods to examine effects of the use of shadow correction,tree-crown features, and feature extraction from CHM in addition to shadow correction. (b) Effects of postprocessing.

applied to a dataset taken over Tama Forest Science Gardenin Tokyo, Japan, to classify the data into 16 classes. Bothshadow correction and the use of tree-crown features derived
from the CHM markedly improved the classification accuracy.
In particular, individual tree-crown delineation, which is also
used in the postprocessing, made a major contribution to theaccurate classification of tree species in the complex mixed
forest, showing the importance of LiDAR data. Our results
indicate that the fusion of remote sensing data will be usefulfor forest management in Japan, one-third of which is covered
with forests. This methodology can be extended to very highresolution imagery by incorporating it in a DSM derived fromstereovision, which provides detailed tree-crown information.
Our future work includes transfer learning for tree speciesclassification considering more practical conditions.
R EFERENCES[1] S. E. Franklin, Remote Sensing for Sustainable Forest Management. Boca
Raton, FL, USA: CRC Press, 2001.
[2] D. Goodenough et al., “Processing hyperion and ALI for forest classification,” IEEE Trans. Geosci. Remote Sens., vol. 41, no. 6, pp. 1321–1331,
Jun. 2003.
[3] D. G. Leckie et al., “Issues in species classification of trees in old growthconifer stands,” Can. J. Remote Sens., vol. 31, no. 2, pp. 175–190, 2005.
[4] M. L. Clark, D. A. Roberts, and D. B. Clark, “Hyperspectral discrimination of tropical rain forest tree species at leaf to crown scales,” Remote
Sens. Environ., vol. 96, no. 3–4, pp. 375–398, 2005.
[5] M. Martin, S. Newman, J. Aber, and R. Congalton, “Determining forest
species composition using high spectral resolution remote sensing data,”
Remote Sens. Environ., vol. 65, no. 3, pp. 249–254, 1998.
[6] M. Dalponte, L. Bruzzone, L. Vescovo, and D. Gianelle, “The role ofspectral resolution and classifier complexity in the analysis of hyperspectral images of forest areas,” Remote Sens. Environ., vol. 113, no. 11,
pp. 2345–2355, 2009.
[7] M. Fauvel, J. A. Benediktsson, J. Chanussot, and J. R. Sveinsson,“Spectral and spatial classification of hyperspectral data using SVMs
and morphological profiles,” IEEE Trans. Geosci. Remote Sens., vol. 46,
no. 11, pp. 3804–3814, Nov. 2008.
[8] E. Næsset, “Estimating timber volume of forest stands using airborne
laser scanner data,” Remote Sens. Environ., vol. 61, no. 2, pp. 246–253,
1997.

[9] J. Hyyppä, O. Kelle, M. Lehikoinen, and M. Inkinen, “A segmentationbased method to retrieve stem volume estimates from 3-D tree heightmodels produced by laser scanners,” IEEE Trans. Geosci. Remote Sens.,
vol. 39, no. 5, pp. 969–975, May 2001.
[10] J. Holmgren and Å. Persson, “Identifying species of individual trees using
airborne laser scanner,” Remote Sens. Environ., vol. 90, pp. 415–423,
2004.
[11] W. Yao, P. Krzystek, and M. Heurich, “Tree species classification andestimation of stem volume and DBH based on single tree extraction by
exploiting airborne full-waveform LiDAR data,” Remote Sens. Environ.,
vol. 123, pp. 368–380, 2012.
[12] G. A. Blackburn, “Remote sensing of forest pigments using airborne
imaging spectrometer and LiDAR imagery,” Remote Sens. Environ.,
vol. 82, no. 2–3, pp. 311–321, 2002.
[13] R. Sugumaran and M. Voss, “Object-oriented classification of lidarfused hyperspectral imagery for tree species identification in an
urban environment,” in Proc. Urban Remote Sens. Joint Event, 2007,
pp. 1–6.
[14] G. P. Asner et al., “Invasive species detection in Hawaiian rainforests using airborne imaging spectroscopy and LiDAR,” Remote Sens.
Environ., vol. 112, pp. 1942–1955, 2008.
[15] M. Dalponte, L. Bruzzone, and D. Gianelle, “Fusion of hyperspectral
and lidar remote sensing data for classification of complex forest areas,”
IEEE Trans. Geosci. Remote Sens., vol. 46, no. 5, pp. 1416–1427, May
2008.
[16] M. Dalponte, L. Bruzzone, and D. Gianelle, “Tree species classificationin the Southern Alps based on the fusion of very high geometrical resolution multispectral/hyperspectral images and LiDAR data,” Remote Sens.
Environ., vol. 123, pp. 258–270, 2012.
[17] M. Dalponte, H. O. Ørka, L. T. Ene, T. Gobakken, and E. Naesset, “Treecrown delineation and tree species classification in boreal forests using
hyperspectral and ALS data,” Remote Sens. Environ., vol. 140, pp. 306–
317, Jan. 2014.
[18] T. G. Jones, N. C. Coops, and T. Sharma, “Assessing the utility of airborne hyperspectral and LiDAR data for species distribution mapping in
the coastal Pacific Northwest, Canada,” Remote Sens. Environ., vol. 114,
pp. 2841–2852, 2010.
[19] M. Alonzo, B. Bookhagen, and D. A. Roberts, “Urban tree species mapping using hyperspectral and LiDAR data fusion,” Remote Sens. Environ.,
vol. 148, pp. 70–83, 2014.
[20] Forestry and Forest Products Research Institute, Tama Forest Science
Garden [Online]. Available: http://www.ffpri.affrc.go.jp/tmk/en/index.
html, accessed on Feb. 2, 2015.
[21] S. M. Adler-Golden, M. W. Matthew, G. P. Anderson, G. W. Felde, andJ. A. Gardner, “An algorithm for de-shadowing spectral imagery,” inProc. 11th JPL Airborne Earth Sci. Workshop, 2002, pp. 1–8.
[22] R. Richter and A. Müller, “De-shadowing of satellite/airborne imagery,”Int. J. Remote Sens., vol. 26, no. 15, pp. 3137–3148, 2005.

Thisarticlehasbeenacceptedforinclusioninafutureissueofthisjournal.Contentisfinalaspresented,withtheexceptionofpagination.
MATSUKI et al.: HYPERSPECTRAL TREE SPECIES CLASSIFICATION OF JAPANESE COMPLEX MIXED FOREST[23] O. Friman, G. Tolt, and J. Ahlberg, “Illumination and shadow compensation of hyperspectral images using a digital surface model and non-linear
least squares estimation,” in Proc. SPIE 8180 Image Signal Process.
Remote Sens. XVII, vol. 81800Q, 2011, pp. 1–8.
[24] Y. Li, P. Gong, and T. Sasagawa, “Integrated shadow removal based on
photogrammetry and image analysis,” Int. J. Remote Sens., vol. 26, no. 18,
pp. 3911–3929, 2005.
[25] J. W. Boardman, “Automating spectral unmixing of AVIRIS data using
convex geometry concepts,” in Proc. Summaries 4th Annu. JPL Airborne
Geosci. Workshop, 1993, vol. 1, pp. 11–14. JPL Publication 93–26.
[26] J. M. P. Nascimento and J. M. B. Dias, “Vertex component analysis: A
fast algorithm to unmix hyperspectral data,” IEEE Trans. Geosci. Remote
Sens., vol. 43, no. 4, pp. 898–910, Apr. 2005.
[27] J. Bioucas-Dias and J. Nascimento, “Hyperspectral subspace identification,” IEEE Trans. Geosci. Remote Sens., vol. 46, no. 8, pp. 2435–2445,
Aug. 2008.
[28] D. Heinz and C.-I. Chang, “Fully constrained least squares linear spectral mixture analysis method for material quantification in hyperspectral
imagery,” IEEE Trans. Geosci. Remote Sens., vol. 39, no. 3, pp. 529–545,
Mar. 2001.
[29] Y. Ke and L. J. Quackenbush, “A review of methods for automatic individual treecrown detection and delineation from passive remote sensing,”
Int. J. Remote Sens., vol. 32, no. 17, pp. 4725–4747, 2011.
[30] M. Erikson and K. Olofsson, “Comparison of three individual tree crowndetection methods,” Mach. Vision Appl., vol. 16, no. 4, pp. 258–265,
2005.
[31] R. J. Pollock, “The automatic recognition of individual trees in aerial
images of forests based on a synthetic tree crown image model,” Ph.D.
dissertation, Dept. Comput. Sci., Univ. British Columbia, 1996.
[32] K. Olofsson, J. Wallerman, J. Holmgren, and H. Olsson, “Tree species
discrimination using Z/I DMC imagery and template matching of single
trees,” Scand. J. For. Res., vol. 21, no. S7, pp. 106–110, 2006.
[33] M. Erikson, “Segmentation of individual tree crowns in color aerial photographs using region growing supported by fuzzy rules,” Can. J. For.
Res., vol. 33, no. 8, pp. 1557–1563, 2003.
[34] M. Erikson, “Species classification of individually segmented tree crowns
in high resolution aerial images using radiometric and morphologic image
measures,” Remote Sens. Environ., vol. 91, no. 3–4, pp. 469–477, 2004.
[35] F. A. Gougeon, “A crown-following approach to the automatic delineation of individual tree crowns in high spatial resolution aerial images,”
Can. J. Remote Sens., vol. 21, no. 3, pp. 274–284, 1995.
[36] Z. Li, R. Hayward, J. Zhang, and Y. Liu, “Individual tree crown delineation techniques for vegetation management in power line corridor,” inProc. Digital Image Comput. Techn. Appl. (DICTA), 2008, pp. 148–154.
[37] B. Koch, U. Heyder, and H. Weinacker, “Detection of individual tree
crowns in airborne lidar data,” Photogramm. Eng. Remote Sens., vol. 72,
no. 4, pp. 357–363, 2006.
[38] S. Solberg, E. Naesset, and O. M. Bollandsas, “Single tree segmentation
using airborne laser scanner data in a structurally heterogeneous spruce
forest,” Photogramm. Eng. Remote Sens., vol. 72, no. 12, pp. 1369–1378,
2006.
[39] F. Melgani and L. Bruzzone, “Classification of hyperspectral remote sensing images with support vector machines,” IEEE Trans. Geosci. Remote
Sens., vol. 42, no. 8, pp. 1778–1790, Aug. 2004.
[40] C. M. Bishop, Pattern Recognition and Machine Learning (Information
Science and Statistics). New York, NY, USA: Springer-Verlag, 2006.

11[41] C. Chang and C. Lin, “LIBSVM: A library for support vector machines,”
ACM Trans. Intell. Syst. Technol., vol. 2, no. 3, pp. 27:1–27:27, 2011
[Online]. Available: http://www.csie.ntu.edu.tw/~cjlin/libsvm, accessedon Aug. 18, 2013.
[42] University of Houston, Outcome of the “Best Classification Challenge”
of the 2013 IEEE GRSS Data Fusion Contest [Online]. Available: http://
hyperspectral.ee.uh.edu/?page id=695, accessed on Sep. 25, 2014.
[43] C. Tomasi and R. Manduchi, “Bilateral filtering for gray and color
images,” in Proc. 6th Int. Conf. Comput. Vis., 1998, pp. 839–846.
[44] P. Choudhury and J. Tumblin, “The trilateral filter for high contrast images and meshes,” in Proc. ACM SIGGRAPH 2005 Courses(SIGGRAPH’05), New York, NY, USA, 2005, pp. 1–12.
[45] Z. Zhen, L. J. Quackenbush, S. V. Stehman, and L. Zhang, “Impactof training and validation sample selection on classification accuracy
and accuracy assessment when using reference polygons in object-basedclassification,” Int. J. Remote Sens., vol. 34, no. 19, pp. 6914–6930, 2013.

Tomohiro Matsuki received the M.Sc. degree inaerospace engineering from the University of Tokyo,
Tokyo, Japan, in 2014.
His research interests include hyperspectral data
analysis.

Naoto Yokoya (S’10–M’13) received the M.Sc. and
Ph.D. degrees in aerospace engineering from the
University of Tokyo, Tokyo, Japan, in 2010 and 2013,
respectively.
From 2012 to 2013, he was a Research Fellow
with the Japan Society for the Promotion of Science,
Tokyo, Japan. Currently, he is an Assistant Professor
with the University of Tokyo. His research interests
include image processing and pattern recognition for
remote sensing.

Akira Iwasaki received the M.Sc. degree inaerospace engineering and the Doctoral degree inengineering from the University of Tokyo, Tokyo,
Japan, in 1987 and 1996, respectively.
In 1987, he joined the Electrotechnical Laboratory,
where he was engaged in research on space technology and remote sensing system. Currently, he is a
Professor with the University of Tokyo.