IEEE JOURNAL OF SELECTED TOPICS IN APPLIED EARTH OBSERVATIONS AND REMOTE SENSING, VOL. 7, NO. 6, JUNE 2014

2419

Fusion of Hyperspectral and LiDAR Data forLandscape Visual Quality Assessment
Naoto Yokoya, Member, IEEE, Shinji Nakazawa, Tomohiro Matsuki, and Akira Iwasaki

Abstract—Landscape visual quality is an important factor associated with daily experiences and inﬂuences our quality of life. In
this work, the authors present a method of fusing airborne hyperspectral and mapping light detection and ranging (LiDAR) data forlandscape visual quality assessment. From the fused hyperspectral
and LiDAR data, classiﬁcation and depth images at any location canbe obtained, enabling physical features such as land-cover properties and openness to be quantiﬁed. The relationship between physical features and human landscape preferences is learned using least
absolute shrinkage and selection operator (LASSO) regression. The
proposed method is applied to the hyperspectral and LiDAR
datasets provided for the 2013 IEEE GRSS Data Fusion Contest.
The results showed that the proposed method successfully learned ahuman perception model that enables the prediction of landscapevisual quality at any viewpoint for a given demographic used fortraining. This work is expected to contribute to automatic landscapeassessment and optimal spatial planning using remote sensing data.
Index Terms—Hyperspectral data, landscape visual quality, least
absolute shrinkage and selection operator (LASSO) regression,light detection and ranging (LiDAR) data, multisensor classiﬁcation, openness.

I. INTRODUCTIONANDSCAPE quality assessment is becoming a major
component in spatial planning owing to increasing interestin the concept of sustainability including aspects related to thequality of life (QOL) and of landscapes [1]. Previous studies andtheories [2] suggest an inﬂuence of the landscape visual quality inurban areas on criminal [3], [4] and health behaviors [5], [6],which are signiﬁcant factors determining the QOL. Therefore,
evaluative maps of landscape visual quality based on humanperception or, in the case of this study, machine-modeled humanperception, are helpful for policy makers, space planners, andarchitects to plan and design the appearance of urban areas thatare attractive to inhabitants [7].
The assessment of landscape visual quality is a challenging
issue because there are many objective factors that affect subjective human perceptions. Psychophysical preference modelingis one of the quantitative holistic techniques of landscapeevaluation that mix subjective and objective methods [8], [9].

L

Manuscript received October 31, 2013; revised February 08, 2014; accepted
March 17, 2014. Date of publication April 03, 2014; date of current version
August 01, 2014.
N. Yokoya and A. Iwasaki are with the Department of Advanced Interdisciplinary Studies, The University of Tokyo, Tokyo 1538904, Japan (e-mail:yokoya@sal.rcast.u-tokyo.ac.jp; aiwasaki@sal.rcast.u-tokyo.ac.jp).
S. Nakazawa and T. Matsuki are with the Department of Aeronautics andAstronautics, The University of Tokyo, Tokyo 1138654, Japan (e-mail: shinji.
nakazawa@sal.rcast.u-tokyo.ac.jp; matsuki@sal.rcast.u-tokyo.ac.jp).
Color versions of one or more of the ﬁgures in this paper are available online at
http://ieeexplore.ieee.org.
Digital Object Identiﬁer 10.1109/JSTARS.2014.2313356

This approach relates psychological human preference data tophysical landscape component data in a mathematical manner,which mainly consists of three steps: 1) collection of humanperception data for a number of different landscapes; 2) physicalfeature extraction from these landscapes; and 3) learning therelationship between human perception and physical featuredatasets. Quantitative human perception data of landscape quality have commonly been collected by surveys that use groundbased color photographs and scoring methods, such as paired
comparisons, Likert scales, and ranking scales [4], [9], [10].
Physical characteristics of landscapes can be extracted fromcolor photographs by manual segmentation [9]. Multivariablelinear regression analysis have been widely used to ﬁnd psychophysical predictive models that relate physical features to landscape quality [1], [11], [12]. Even when a predictive model isobtained, the creation of evaluative maps of landscape visualquality is time-consuming, and it is not feasible to take photographs everywhere to extract physical features. Geographic
information systems (GISs) and computer vision are useful for
calculating key visual features from any viewpoint over a large
area to assess landscape quality [1], [13]. Wu et al. reported that
biological and physical characteristics deﬁned by classiﬁcation
have major inﬂuences on landscape visual quality [1]. Weitkamp
et al. proposed a GIS-based procedure for measuring landscapeopenness [13], [14] using depth information, which is an important part of our understanding and appreciation of landscapes[15]. Therefore, remotely sensed data, computer vision, andmachine learning techniques have major potential for extracting
the physical features of landscapes at a low cost.
Hyperspectral imaging is a promising remote sensing technology for generating classiﬁcation maps. A continuum spectrum enables the accurate identiﬁcation and classiﬁcation ofland-cover classes that are spectrally similar. Many researchers
have investigated hyperspectral classiﬁcation and shown its
validity in mineral mapping, tree species discrimination, andurban classiﬁcation [16]–[18]. Land-use and land-cover classiﬁcation maps are basic information of GISs. Light detection andranging (LiDAR) is an active remote sensing technique that uses
electromagnetic energy in the optical range to determine thedistance between the instrument and a target surface and deduce
physical properties of the target based on the interaction of theradiation with the target [19]. LiDAR has a wide range ofapplications, such as atmospheric monitoring and canopy height
analysis [20], [21]. Mapping LiDAR data is an accurate tool for
obtaining a digital surface model (DSM) and a digital terrain
model (DTM). Such data are rasterized 2.5D data and useful for
visible area analysis. Openness can be quantiﬁed using depth
information of sight [13], [14], which is calculated from a DSM.

1939-1404 © 2014 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.
See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.

2420

IEEE JOURNAL OF SELECTED TOPICS IN APPLIED EARTH OBSERVATIONS AND REMOTE SENSING, VOL. 7, NO. 6, JUNE 2014

Fig. 2. (a) Terrain and building datasets. (b) Visible space in 2-D with contour
lines. (c) Depth map from a viewpoint.
Fig. 1. Overview of the proposed method.

Moreover, the joint use of optical and LiDAR data is effective for
accurate classiﬁcation, e.g., tree species classiﬁcation [22]–[25].
Therefore, the fusion of hyperspectral and LiDAR data ispromising for extracting important physical features related tolandscape quality.
In this work, we present a new methodology that fuses anairborne hyperspectral image and a LiDAR-derived DSM forlandscape visual quality assessment. The main objective of thiswork is to propose a framework that relates landscape physicalfeatures to the human perception of such features, not to develop
a general psychophysical relationship. Landscape physical features, such as openness and proportions of land-cover classes fora given view, are extracted from the hyperspectral image and a
LiDAR-derived DSM. The relationship between physical features and human landscape quality assessment based on groundbased images is learned by least absolute shrinkage and selection
operator (LASSO) regression [26]. This framework enables theprediction of landscape quality from any viewpoint using largescale remote sensing observation.
This paper is organized as follows. Section II describes themethodology for predicting landscape visual quality by fusing anairborne hyperspectral image and a LiDAR-derived DSM. An
experimental study using the datasets provided for the 2013
IEEE GRSS Data Fusion Contest and images collected from
Google Street View is presented in Section III. The conclusion is
given in Section IV.
II. METHODOLOGY
The methodology presented in this paper is divided into threemain steps, as shown in Fig. 1. Each step is summarized as follows.
1) Viewpoint selection and human scoring: We prepare humanperception data by collecting landscape photographs taken
at various locations and view angles and asking people torate their visual quality, which is a common technique formeasuring landscape preferences [7], [9], [10]. Groundbased landscape photographs are acquired from Google
Street View [27].
2) Openness and classiﬁcation: We calculate physical features, such as openness, proportions, and average depths ofland-cover classes in a ﬁeld of view, at the collected
viewpoints in the ﬁrst step using the DSM and classiﬁcation map derived from hyperspectral and LiDAR data.
3) Regression and prediction: LASSO regression is applied
to the datasets using physical features and human preference scores as input and target variables, respectively [26].
Once the predictive model is learned by LASSO regression, the landscape visual quality can be predicted at any
viewpoint.
The three novel points of our method are summarized asfollows: 1) ground-based color photographs are collected using
Google Street View; 2) physical features are automaticallyextracted using remote sensing data and computer vision; and
3) LASSO regression is adopted to learn the psychophysicalmodel that relates physical features to human-perception-basedlandscape quality data. More details about the calculation of
openness and classiﬁcation using the hyperspectral image and
LiDAR-derived DSM and as well as LASSO regression are
given in Sections II-A, II-B, and II-C respectively.
A. Openness
Weitkamp et al. measured the visible space using the averageline of sight in a two-dimensional (2-D) space to assess landscapeopenness [13], [14]. Terrain and building datasets [Fig. 2(a)] aremerged to create contour lines, and a visible space is identiﬁedusing viewing limitations as shown in Fig. 2(b). The average
length of radials is highly correlated with perceived openness[14]. We extend this approach to a three-dimensional (3-D) spaceand deﬁne openness as the average value of a depth image

YOKOYA et al.: FUSION OF HYPERSPECTRAL AND LIDAR DATA FOR LANDSCAPE VISUAL QUALITY ASSESSMENT

obtained at a viewpoint [Fig. 2(c)]. Note that depth values are
assigned to nonsky objects to avoid including the inﬁnite depth of
the sky in the calculation of openness. When the location anddirection of an observer are given, the depth of the direction canbe calculated by computing the intersection of the DSM and theline-of-sight vector. Depth values are estimated on radials with
1 spacing in the vertical and horizontal directions. The maximum horizontal and vertical ranges of the viewing angle wereboth set to 90 considering the approximate ﬁeld of view of theGoogle Street View images. This ﬁeld of view can be an
approximation of human binocular vision that covers 120
horizontally and 60 vertically [28].
B. Classiﬁcation
In this work, we begin with the use of airborne data taken overurban areas with a small number of training samples. Shadows
contained in observed data have a major inﬂuence on classiﬁcation. Therefore, as a preprocessing step, shadows in the hyperspectral data need to be modiﬁed. Shadows caused by buildings
and clouds are modiﬁed using line-of-sight analysis and by
thresholding illumination distributions, respectively [29], [30].
When the DSM is directly used for classiﬁcation, various heightsof the terrain may result in classiﬁcation errors. Therefore, theDTM is estimated from the DSM by extracting buildings andtrees by thresholding local height information [31] and interpolating extracted segmentations. The normalized DSM (NDSM),
which is the height of features relative to the ground, is generated
by subtracting the DTM from the DSM.
After the preprocessing, we apply the nearest neighbor (NN)algorithm to a fused feature, i.e., the cosine similarity of spectra
and the NDSM difference. The NDSM difference is normalized
and fused with the cosine similarity using a weighted summation.
A class labelof a test pointis estimated byN
is a reference point belonging to class, withbeing the number of classes.
denotes the spectrum of point , and and indicate the DSM
and NDSM values, respectively. is the variance of a Gaussianfunction deﬁned by statistical analysis of the NDSM data and isa weight coefﬁcient determined by cross-validation. The ﬁnal
classiﬁcation result is obtained by applying iterative bilateralﬁltering [32] to the NN classiﬁcation map to reduce noise. We
extend the normal bilateral ﬁlter to higher dimensions, i.e., a 3-D
location for geometric closeness and a spectrum for photometricsimilarity. Finally, the test point is classiﬁed as

2421

N
is a normalization term deﬁned byN. An iterative bilateral ﬁlter with asmall window acts as an edge-preserving and noise-reducing
smoothing ﬁlter for the classiﬁcation map. By combining theclassiﬁcation map with the 3-D virtual view, proportions andaverage depths of land-cover classes occupied in a given ﬁeld ofview can be calculated, which are key physical features inﬂuencing
perceived landscape quality [1], [15]. In addition to land-coverclasses, we also consider the proportion of sky as a physical featureof landscapes. Here, the proportion of a class in a ﬁeld of view
indicates how much of the class exists in a 2-D view image. The
average depth of a class indicates how far pixels assigned to theclass in a 2-D view image are located from the viewpoint on
average, which can be interpreted as class-labeled openness.
C. LASSO Regression
The objective of the learning phase is to reveal the relationshipbetween physical features of a landscape and human landscapeperception and to construct a prediction model for a new input.
Multivariable linear regression is useful owing to its mathematical
simplicity and physical interpretability. Openness, the proportionof sky, and the proportions and average depths of land-coverclasses in a ﬁeld of view are selected as features and used as input
variables (R). and denote the numbers of samples
and variables, respectively. Human scores for landscape visualquality are used as target values (R). All input variables
are standardized with zero mean and unit variance to investigate
their impacts on landscape visual quality. The least-squares
method is the simplest method of regression; however, it suffersfrom overﬁtting. Regularized least-squares methods that add
penalty terms of regression parameters to the objective function
are useful to avoid this problem and produce robust results. We
adopt LASSO regression [26], which is one of the regularized
least-squares methods and uses the -norm of the parameter
vector as a penalty term. We can assume without loss ofgenerality that
. The LASSO estimate is formulated by

where

N

N

where
R
is the parameter vector and is the tuning
parameter. This problem may be solved by convex optimization.
In LASSO regression, more of the parameters become zero as thepenalty is increased, whereas in ridge regression, which uses the-norm for the penalty, the parameters are reduced but remainnonzero. The former property results in the straightforward physical interpretation of regression parameters, which is the reason
why we choose LASSO regression as the regression method. The
difference between least-squares and LASSO regression is discussed in Section III. The tuning parameter can be determined byleave-one-out cross-validation in which the tradeoff between thesparsity of parameters and the ﬁtting of the regression is examined.
III. EXPERIMENTAL STUDY
A. Datasets

whenbelongs to class
and
where
otherwise.
denotes the Euclidean distance and and are
the geometric and photometric spread parameters, respectively.

In this work, we used the hyperspectral image and LiDARderived DSM distributed by the 2013 IEEE GRSS Data Fusion
Contest. The two images have the same size (
)

2422

IEEE JOURNAL OF SELECTED TOPICS IN APPLIED EARTH OBSERVATIONS AND REMOTE SENSING, VOL. 7, NO. 6, JUNE 2014

TABLE I
R-SQUARED AND PEARSON’S CORRELATION VALUES OF LEAST SQUARES AND LASSO

Fig. 4. Scatter plot showing human-perceived and machine-predicted scores forthe test data with the corresponding views.

Fig. 3. Physical features obtained from hyperspectral and LiDAR data. (a) RGB.
(b) Classiﬁcation. (c) Depth and classiﬁcation images with views from top tobottom.

and ground sampling distance (GSD) (2.5 m). The hyperspectral
data have 144 spectral bands in the 380–1050 nm region. The
dataset was acquired over the University of Houston campus andthe neighboring urban area. The RGB image of the study area isshown in Fig. 3(a). The ground truth of 15 classes is provided by
the Best Classiﬁcation Challenge of the 2013 IEEE GRSS Data
Fusion Contest. Each class of the ground truth consists ofapproximately 200 pixels.
Parking lots and cars were merged, and synthetic grass, tennis
courts, and running tracks, which do not exist in the study area,

were excluded. Furthermore, we manually assigned the “walkway” class to 255 pixels from pavements on the campus into theground truth, so that they can be distinguished from spectrally
similar classes, such as “roadways.” The ﬁnal ground truth
consisted of 12 classes, i.e., healthy grass, stressed grass, trees,soil, water, residential, multistory buildings, roadways, highways, railways, parking lots, and walkways. Since we use
openness, the proportion of sky, and the proportions and average
depths of 12 classes as inputs, the number of variables is 26.
We collected 200 view images taken on the ground at 50
locations from four directions using Google Street View in thisarea. The 50 locations are shown in Fig. 3(a) as white crosses andthe four directions are north, south, east, and west. In thisexperiment, we ﬁxed the training data to 100 randomly chosenviews and used the remaining 100 views as test data. Subjective
human landscape preferences can be inﬂuenced by many factors,
such as culture, nationality, generation, and personality. The
main objective of this work is to present a framework that relates
physical features to human perception to construct a predictivemodel of landscape quality using remote sensing data, not toreveal a general psychophysical relationship. Therefore, wechose a speciﬁc group of people, i.e., eight students from the
University of Tokyo, Japan, and asked them to give preference
scores between 1 and 10 assuming that they have similar backgrounds owing to their common culture and generation. Theiraverage scores were used as target values.

YOKOYA et al.: FUSION OF HYPERSPECTRAL AND LIDAR DATA FOR LANDSCAPE VISUAL QUALITY ASSESSMENT

2423

Fig. 6. Regression coefﬁcients (blue: least squares and red: LASSO).

Fig. 5. Histograms of training samples for all input variables.

B. Results and Discussion
Fig. 3(b) shows the classiﬁcation map estimated by our
method. The presented method achieved 90.65% classiﬁcation
accuracy in the Best Classiﬁcation Challenge of the 2013 IEEE
GRSS Data Fusion Contest [33]. Fig. 3(c) shows the depth,classiﬁcation, and view images for the locations and directions
shown by four arrows in Fig. 3(a). Although the camera model ofthe depth images and classiﬁcation maps is different from that of
the subimages of Google Street View, Fig. 3(c)(i), (ii), and (iv)show the consistent appearance of the major-class distribution
between classiﬁcation images and ground-based photographs,
which indicates that the physical features at a viewpoint can beapproximately extracted by the proposed method. The advantageof the use of remote sensing data is that physical features can becomputed at any viewpoint of a large area. Some differences
between the photographs and the classiﬁcation images of views
are caused by the limitation of remote sensing images. In theclassiﬁcation image of Fig. 3(c)(ii), the walkway is misclassiﬁed
as grass owing to the mixed pixel spectra. A smaller GSD may
allow the more accurate extraction of features from a landscape.
Most attempts to create regression models of landscape quality
have been applied to rural landscapes [9], [10]. There are morebuildings in urban areas, resulting in a signiﬁcant difference inthe landscape at a smaller geometric scale depending on thelocation and viewing direction. Therefore, spatially detailed
information is more important for urban areas, whereas spatiallylarger datasets may be required for rural areas owing to there
being fewer obstacles and better viewing conditions. In Fig. 3(c)(iii), the visibility of the classiﬁcation view image under trees isdifferent from that of the photograph owing to the complicated
3-D structures of trees and the occlusion of the walkway. The
LiDAR-derived DTM and DSM have only 2.5D information,
meaning that a single value of elevation is provided for an entire
cell, which is the elevation of the perceived ground or theelevation of the ﬁrst detected return, respectively. If the full
3-D information contained in the LiDAR point cloud is used,better-matching synthetic images could have been obtained withrespect to the ground-based photographs used for the regression.
R-squared and Pearson’s correlation values were used toevaluate the agreement between the machine-predicted score,
which is the output of the regression model, and the humanpreference score. Table I shows a comparison between LASSO
regression and the least-squares method for both the training andthe test data. LASSO regression exhibits a more accurate androbust performance than the least-squares regression for the test
data. Fig. 4 shows a scatter plot of the human-preference andmachine-predicted scores obtained using the test data along withthe corresponding views. This plot intuitively visualizes thesimilarity between machine predictions and human perceptions
and reveals both the correlation and the misalignment obtainedby the proposed method. Fig. 5 shows histograms that represent
the distributions of the 100 training samples for all input variables. A wide range in the relative occurrence of different
variables can be observed. Fig. 6 shows the coefﬁcients of theleast-squares and LASSO regression, which were used to investigate physical factors that inﬂuence landscape visual quality. We
can ﬁnd differences between the two methods in terms of thescale and sparsity of the coefﬁcients as well as their signs for classproportions of human-made structures. In the least-squares
regression, it is more difﬁcult to interpret the coefﬁcients because
all the coefﬁcients of class proportions except for soil havepositive values. In LASSO regression, several coefﬁcients are
regularization, which results in a straightzero owing to theforward interpretation of the trained model. Openness and the
proportions of sky, trees, and water have positive impacts onlandscape quality, whereas those of most human-made structures
have negative impacts. This result is consistent with otherlandscape studies [1], [7], [9], [12]. In particular, the proportionof water has the largest positive impact; water exists in the formof fountains and a small natural lake on this campus. It is interesting that there are clear differences between the landscapequality impact of walkways and other human-made structures
and also between grass and trees, which are spectrally similarclasses. Detailed and accurate classiﬁcation by the joint use of a
hyperspectral image and a LiDAR-derived DSM enables ameaningful model of landscape quality prediction to be learned.

2424

IEEE JOURNAL OF SELECTED TOPICS IN APPLIED EARTH OBSERVATIONS AND REMOTE SENSING, VOL. 7, NO. 6, JUNE 2014

Fig. 7. Landscape assessment from the target building.

Finally, we show an application of the proposed method oflandscape assessment. Since the model is already learned and all
physical features can be calculated from the hyperspectral image
and LiDAR-derived DSM, we can automatically predict thelandscape visual quality from any 3-D location and direction.
We focused on one building, as shown on the left of Fig. 7, andpredicted the landscape visual quality scores for the views fromall ﬂoors. The upper-right images of Fig. 7 show the predictedscore maps for the views from the four sides of the building. At
this location, the landscape quality is high mainly because of thehigh openness and proportions of sky, trees, and water. Consequently, views from the higher ﬂoors, where a fountain can beseen and the openness and the proportion of sky are greater, havehigher landscape scores. The three images in the bottom-right ofFig. 7 show virtual RGB views from three representative points.
As described earlier, the proposed system can predict the landscape visual quality at any viewpoint in the study area. Thispredictive method will effectively predict the landscape visualquality for the given demographic that it was trained for.
IV. CONCLUSION
In this work, we presented a novel method of fusing a
hyperspectral image and a LiDAR-derived DSM for landscapevisual quality assessment. The physical properties of objects andthe openness in a view are assumed to be important factors
inﬂuencing landscape quality. The datasets of the 2013 IEEE
GRSS Data Fusion Contest were used in the experimental study,
and an accurate classiﬁcation map was obtained by fusing
spectral and 3-D spatial information. Using a LiDAR-derived
DSM and the classiﬁcation map, depth and classiﬁcation maps
from any viewpoint could be estimated. LASSO regression was
adopted to learn a psychophysical predictive model that relates
physical features to human preference scores with meaningful
regression coefﬁcients. Physical feature extraction by the joint
use of the hyperspectral image and LiDAR-derived DSM reveals
the differences in the landscape-quality impact between spectrally similar land-cover classes and enables the prediction oflandscape quality from any 3-D viewpoint over a large area. This
work is expected to contribute to automatic landscape assessment and optimal spatial planning using remote sensing data.
REFERENCES[1] Y. Wu, I. Bishop, H. Hossain, and V. Sposito, “Using GIS in landscapevisual quality assessment,” Appl. GIS, vol. 2, no. 3, pp. 18.1–18.20, 2006.

[2] J. Q. Wilson and G. L. Kelling, “Broken windows,” Atl. Mon., vol. 249,
no. 3, pp. 29–38, 1982.
[3] K. Keizer, S. Lindenberg, and L. Steg, “The spreading of disorder,” Science,
vol. 322, no. 5908, pp. 1681–1685, 2008.
[4] P. Salesses, K. Schechtner, and C. A. Hidalgo, “The collaborative image ofthe city: Mapping the inequality of urban perception,” PLoS One, vol. 8,
no. 7, pp. 1–12, 2013.
[5] D. Cohen, S. Spear, R. Scribner, P. Kissinger, K. Mason, and J. Wildgen,“‘Broken windows’ and the risk of gonorrhea,” Amer. J. Public Health,vol. 90, no. 2, pp. 230–236, 2000.
[6] C. E. Ross and J. Mirowsky, “Neighborhood disadvantage, disorder, andhealth,” J. Health Social Behav., vol. 42, pp. 258–276, 2001.
[7] J. K. Nasar, The Evaluative Image of the City. Newbury, CA, USA: Sage,
1997.
[8] E. L. Shafer, “Perception of natural environments,” Environ. Behav., vol. 1,
pp. 71–82, 1969.
[9] J. R. Wherrett, “Creating landscape preference models using internet survey
techniques,” Landscape Res., vol. 25, no. 1, pp. 76–96, 2000.
[10] H. Schroeder and T. C. Daniel, “Progress in predicting the perceived scenic
beauty of forest landscapes,” Forest Sci., vol. 27, no. 1, pp. 71–80, 1981.
[11] G. J. Buhyoff, P. A. Miller, J. W. Roach, D. Zhou, and L. G. Fuller, “An AI
methodology for landscape visual assessments,” AI Appl., vol. 8, no. 1,
pp. 1–13, 1994.
[12] M. Arriaza, J. F. Cañas-Ortega, J. A. Cañas-Madueño, and P. Ruiz-Aviles,“Assessing the visual quality of rural landscapes,” Landsc. Urban Plann.,
vol. 69, pp. 115–125, 2004.
[13] G. Weitkamp, A. Bregt, and R. V. Lammeren, “Measuring visible space toassess landscape openness,” Landsc. Res., vol. 36, no. 2, pp. 127–150, 2011.
[14] G. Weitkamp, “Mapping landscape openness with isovists,” Res. Urban.
Ser., vol. 2, no. 1, pp. 205–223, 2011.
[15] I. D. Bishop, J. R. Wherrett, and D. R. Miller, “Using image depth variables
as predictors of visual quality,” Environ. Plann. B: Plann. Des., vol. 27,
pp. 865–875, 2000.
[16] F. A. Kruse, J. W. Boardman, and J. F. Huntington, “Comparison of
airborne hyperspectral data and EO-1 hyperion for mineral mapping,”
IEEE Trans. Geosci. Remote Sens., vol. 41, no. 6, pp. 1388–1400,
Jun. 2003.
[17] M. L. Clark, D. A. Roberts, and D. B. Clark, “Hyperspectral discriminationof tropical rain forest tree species at leaf to crown scales,” Remote Sens.
Environ., vol. 96, pp. 375–398, 2005.
[18] J. A. Benediktsson, J. A. Palmason, and J. R. Sveinsson, “Classiﬁcation of
hyperspectral data from urban areas based on extended morphologicalproﬁles,” IEEE Trans. Geosci. Remote Sens., vol. 43, no. 3, pp. 480–491,
Mar. 2005.
[19] J. C. F. Diaz, W. E. Carter, R. L. Shrestha, and C. L. Glennie, “Lidar remotesensing,” in Handbook of Satellite Applications, 2013, pp. 757–808.
[20] F. Molero and F. Jaque, “The laser as a tool in environmental problems,”
Opt. Mater., vol. 13, pp. 167–173, 1999.
[21] M. A. Lefsky, W. B. Cohen, G. G. Parker, and D. J. Harding, “Lidar remotesensing for ecosystem studies,” Bioscience, vol. 52, no. 1, pp. 19–30, 2002.
[22] M. Dalponte, L. Bruzzone, and D. Gianelle, “Fusion of hyperspectral andlidar remote sensing data for classiﬁcation of complex forest areas,” IEEE
Trans. Geosci. Remote Sens., vol. 46, no. 5, pp. 1416–1427, May 2008.
[23] M. Pedergnana, P. R. Marpu, M. D. Mura, J. A. Benediktsson, and
L. Bruzzone, “Classiﬁcation of remote sensing optical and LiDAR datausing extended attribute proles,” IEEE J. Sel. Topics Signal Process., vol. 6,
no. 7, pp. 856–865, Nov. 2012.
[24] R. Dinuls, G. Erins, A. Lorencs, I. Mednieks, and J. Sinica-Sinavskis, “Treespecies identiﬁcation in mixed Baltic forest using LiDAR and multispectral
data,” IEEE J. Sel. Topics Appl. Earth Observ. Remote Sens., vol. 5, no. 2,
pp. 594–603, Apr. 2012.
[25] J. Vauhkonen, T. Hakala, J. Suomalainen, S. Kaasalainen, O. Nevalainen,
M. Vastaranta et al., “Classiﬁcation of spruce and pine trees using active
hyperspectral LiDAR,” IEEE Geosci. Remote Sens. Lett., vol. 10, no. 5,
pp. 1138–1141, Sep. 2013.
[26] R. Tibshirani, “Regression shrinkage and selection via the Lasso,” J. Roy.
Stat. Soc. Ser. B (Stat. Methodol.), vol. 58, no. 1, pp. 267–288, 1996.
[27] Google Maps, About Street View [Online]. Available: http://www.google.
com/maps/about/behind-the-scenes/streetview/ accessed on Jan. 14, 2014.
[28] D. B. Henson, Visual Fields. London, U.K.: Oxford Univ. Press, 1993.
[29] O. Friman, G. Tolt, and J. Ahlberg, “Illumination and shadow compensationof hyperspectral images using a digital surface model and non-linear leastsquares estimation,” Proc. SPIE, vol. 8180, pp. 1–8, 2011.
[30] B. D. Wemett, J. K. Riek, and R. A. Leathers, “Dynamic thresholding for
hyperspectral shadow detection using Levenberg-Marquardt minimization
on multiple Gaussian illumination distributions,” Proc. SPIE, vol. 7334,
pp. 1–9, 2009.

YOKOYA et al.: FUSION OF HYPERSPECTRAL AND LIDAR DATA FOR LANDSCAPE VISUAL QUALITY ASSESSMENT[31] A. Brunn and U. Weidner, “Extracting buildings from digital surface
models,” in Proc. IAPRS, Stuttgart, Germany, 1997, vol. 32, part 3–4w2.
[32] C. Tomasi and R. Manduchi, “Bilateral ﬁltering for gray and color images,”in Proc. IEEE Int. Conf. Comput. Vis. (ICCV’98), 1998, pp. 839–846.
[33] Outcome of the “Best Classiﬁcation Challenge” of the 2013 IEEE GRSS
Data Fusion Contest [Online]. Available: http://hyperspectral.ee.uh.edu/?
page_id=695, accessed on Jan. 14, 2014.

Naoto Yokoya (S’10–M’13) received the M.Sc. and
Ph.D. degrees in aerospace engineering from the
University of Tokyo, Tokyo, Japan, in 2010 and2013, respectively.
He was a Research Fellow with the Japan Society
for the Promotion of Science from 2012 to 2013. He iscurrently an Assistant Professor with the University ofTokyo. His research interests include image analysis
and data fusion in remote sensing.

Shinji Nakazawa received the M.Sc. degree in aerospace engineering from the University of Tokyo,
Tokyo, Japan, in 2014.
His research interests include image processing andoptical system.

2425

Tomohiro Matsuki received the M.Sc. degree inaerospace engineering from the University of Tokyo,
Tokyo, Japan, in 2014.
His research interests include hyperspectral data
analysis.

Akira Iwasaki received the M.Sc. degree in aerospace engineering and the Doctoral degree in engineering from the University of Tokyo, Tokyo, Japan,
in 1987 and 1996, respectively.
In 1987, he joined the Electrotechnical Laboratory,
where he was engaged in research on space technology and remote sensing system. He is currently aProfessor with the University of Tokyo.