# Automatic Fusion of Hyperspectral Images and Laser Scans Using Feature Points

Authors: Xiao Zhang; Aiwu Zhang; Xiangang Meng
Journal: Journal of Sensors
 (2015)

---

## Abstract

Automatic fusion of different kinds of image datasets is so intractable with diverse imaging principle. This paper presents a novel method for automatic fusion of two different images: 2D hyperspectral images acquired with a hyperspectral camera and 3D laser scans obtained with a laser scanner, without any other sensor. Only a few corresponding feature points are used, which are automatically extracted from a scene viewed by the two sensors. Extraction method of feature points relies on SURF algorithm and camera model, which can convert a 3D laser scan into a 2D laser image with the intensity of the pixels defined by the attributes in the laser scan. Moreover, Collinearity Equation and Direct Linear Transformation are used to create the initial corresponding relationship of the two images. Adjustment is also used to create corrected values to eliminate errors. The experimental result shows that this method is successfully validated with images collected by a hyperspectral camera and a laser scanner.


---

## Body

## 1. Introduction

Hyperspectral imaging technology can quickly detect hundreds or even thousands of different light frequencies and relative intensities of surface features, which is unlike regular cameras that are typically sensitive to only three different frequencies (red, green, and blue). Laser scanning technology can quickly obtain the accurate geometry information of surface features in despite of the adverse external circumstances. If the hyperspectral data and the laser data can be fused, the spectral information and the spatial information of the same location can be obtained at the same time, which can effectively make up for the deficiency of single data source.Currently, the registration and fusion of hyperspectral images and laser scans have been the research hotspots. However, because of the multiple different sensors and the different imaging modalities the fusion is very complicated. The most common approach is to perform registration using manual methods. However, this approach is very low precision in practice. Only several methods exist for aligning these two datasets of the same location. Nieto et al. [1] installed a digital camera on top of the laser to acquire the color point clouds and translated it into the 2D color image; then the registration is completed by the piecewise linear transform. Kurz et al. [2] used two sensors to detect the position of the target and then corrected them to complete the registration. Zachary and Juan [3] obtained initial position between two sensors through GPS and then used the mutual information to achieve the registration. In addition, some methods of aligning regular digital image with laser scan can be used for reference. For example, Tsai camera calibration method [4, 5] was used to obtain the 2D-3D homologous points and the unknown parameters to implement the registration. However, the precise corresponding points were difficultly found because of the difference of color structure. The stereo matching [6] was used to convert multi-images to 3D point clouds to realize the 2D-3D registration. However, this method did not realize the registration of single image and point cloud. Moreover, the mutual information [7–9] was also used to complete the 2D-3D registration. The collinear equation was used to construct 2D-3D correspondence [10, 11] to implement the registration of aerial images and laser scans.In summary, the attempt in this paper is to create a method that can correctly register and fuse the hyperspectral data and the laser data without the additional sensors. The remainder of this paper is organized as follows. Section2 provides the mathematical model of this algorithm. Section 3 in great detail describes the algorithm of automatic fusion. Section 4 simulates and verifies this automatic fusion methodology. Finally Section 5 makes the concluding remarks and maps out the directions for future work.


## 2. Derivation of Mathematical Model

### 2.1. Definition of Coordinate System

#### 2.1.1. Coordinate System of Hyperspectral Image

The data model of hyperspectral image, different from the model of remote sensing image and digital image, is the feature vector representation model and can be expressed by the data cube model, as shown in Figure1. x-axis and y-axis denote the space dimensions and λ-axis denotes the spectral band. X
O
Y-plane is the image information of a band or multiband of hyperspectral image; X
O
λ-plane and Y
O
λ-plane are the spectrum information of a hyperspectral image line, as shown in Figures 1(a) and 1(b). The cube model and the spectrum oscillogram of a hyperspectral pixel are shown in Figure 1(c).Figure 1
Hyperspectral Image cube model.Rotary broom hyperspectral camera scans imaging with the line array scanning mode and the hyperspectral image is the 2D image. Imaging geometric model is to create the mapping relationship between object space and image space, as shown in Figure2, where O
hyp is the coordinate origin; x
hyp is the abscissa, which is along the rotation direction of hyperspectral camera from the rotation starting location; y
hyp is the ordinate, which is along the rotation vertical direction. The horizontal resolution is determined with the rotation speed of turntable and the vertical resolution is determined by the scanning speed of hyperspectral camera.Figure 2
Coordinate system of Hyperspectral Image.


#### 2.1.2. Coordinate System of Laser Scans

Terrestrial 3D laser scans imaging with the line array scanning mode and the laser image is 3D point clouds, whose coordinate system is confirmed by the self-laser, as shown in Figure3, where O
las is the coordinate origin, which is the scanner electrooptical center; z
las is the scanner vertical rotation axis; x
las is along the optical axis scanner arbitrary horizontal angle, such as the first horizontal angle or the north direction of the built-in magnetic compass; y
las is orthogonal to x
las and z
las, which is formed by the right-hand system.Figure 3
Coordinate system of Laser Scan.


### 2.2. Definition of Camera Model

Rotary broom hyperspectral camera and Terrestrial 3D laser are both dependent on the cylindrical coordinate system, but their images are, respectively, 2D and 3D, so the camera model is formed to transform the 3D laser scan into the 2D laser image.This camera model is inferred by panoramic camera model [12, 13], as shown in Figure 4. The formula is shown in(1)
x
2
d-las
=
x
0
-
c
arctan
-
y
las
x
las
+
Δ
x
,
y
2
d-las
=
y
0
-
c
z
las
x
las
2
+
y
las
2
+
Δ
y
,where (
x
las
,
y
las
,
z
las
) is the point of laser scan; (
x
2
d-las
,
y
2
d-las
) is the point of 2D laser image; c is the principle distance of the cameral model; (
x
0
,
y
0
) is the principle point of 2D laser image; (
Δ
x
,
Δ
y
) is the correction parameter.Figure 4
Camera model.Through (1), the point of 2D laser image (
x
2
d-las
,
y
2
d-las
) is calculated, but the type of their values is double and the type of 2D image pixel is integer, so (
x
2
d-las
,
y
2
d-las
) must be changed into the integer value, as shown in (2)
x
2
d-las
′
=
int
x
2
d-las
-
min
⁡
⁡
x
2
d-las
max
⁡
⁡
x
2
d-las
-
min
⁡
⁡
x
2
d-las
×
int
max
⁡
⁡
x
2
d-las
-
min
⁡
⁡
x
2
d-las
A
m
,
y
2
d-las
′
=
int
y
2
d-las
-
min
⁡
⁡
y
2
d-las
max
⁡
⁡
y
2
d-las
-
min
⁡
⁡
y
2
d-las
×
int
max
⁡
⁡
y
2
d-las
-
min
⁡
⁡
y
2
d-las
A
n
,where (
x
2
d-las
′
,
y
2
d-las
′
) is the point position of the 2D image; A
m is the horizontal resolution; A
n is the vertical resolution.The pixels value of 2D laser image is defined by the attributes of the laser scan, which may be the information such as color, curvature, and normal, and so forth. The formula is as follows:(3)
color
=
col
-
min
⁡
⁡
y
max
⁡
⁡
y
-
min
⁡
⁡
y
×
255
,where color is the value of (
x
2
d-las
′
,
y
2
d-las
′
), ranged in [0–255]; col is the attribute value of (
x
las
,
y
las
,
z
las
); max
⁡
⁡
y and min
⁡
⁡
y are, respectively, the maximum and the minimum of this attribute.


## 2.1. Definition of Coordinate System

### 2.1.1. Coordinate System of Hyperspectral Image

The data model of hyperspectral image, different from the model of remote sensing image and digital image, is the feature vector representation model and can be expressed by the data cube model, as shown in Figure1. x-axis and y-axis denote the space dimensions and λ-axis denotes the spectral band. X
O
Y-plane is the image information of a band or multiband of hyperspectral image; X
O
λ-plane and Y
O
λ-plane are the spectrum information of a hyperspectral image line, as shown in Figures 1(a) and 1(b). The cube model and the spectrum oscillogram of a hyperspectral pixel are shown in Figure 1(c).Figure 1
Hyperspectral Image cube model.Rotary broom hyperspectral camera scans imaging with the line array scanning mode and the hyperspectral image is the 2D image. Imaging geometric model is to create the mapping relationship between object space and image space, as shown in Figure2, where O
hyp is the coordinate origin; x
hyp is the abscissa, which is along the rotation direction of hyperspectral camera from the rotation starting location; y
hyp is the ordinate, which is along the rotation vertical direction. The horizontal resolution is determined with the rotation speed of turntable and the vertical resolution is determined by the scanning speed of hyperspectral camera.Figure 2
Coordinate system of Hyperspectral Image.


### 2.1.2. Coordinate System of Laser Scans

Terrestrial 3D laser scans imaging with the line array scanning mode and the laser image is 3D point clouds, whose coordinate system is confirmed by the self-laser, as shown in Figure3, where O
las is the coordinate origin, which is the scanner electrooptical center; z
las is the scanner vertical rotation axis; x
las is along the optical axis scanner arbitrary horizontal angle, such as the first horizontal angle or the north direction of the built-in magnetic compass; y
las is orthogonal to x
las and z
las, which is formed by the right-hand system.Figure 3
Coordinate system of Laser Scan.


## 2.1.1. Coordinate System of Hyperspectral Image

The data model of hyperspectral image, different from the model of remote sensing image and digital image, is the feature vector representation model and can be expressed by the data cube model, as shown in Figure1. x-axis and y-axis denote the space dimensions and λ-axis denotes the spectral band. X
O
Y-plane is the image information of a band or multiband of hyperspectral image; X
O
λ-plane and Y
O
λ-plane are the spectrum information of a hyperspectral image line, as shown in Figures 1(a) and 1(b). The cube model and the spectrum oscillogram of a hyperspectral pixel are shown in Figure 1(c).Figure 1
Hyperspectral Image cube model.Rotary broom hyperspectral camera scans imaging with the line array scanning mode and the hyperspectral image is the 2D image. Imaging geometric model is to create the mapping relationship between object space and image space, as shown in Figure2, where O
hyp is the coordinate origin; x
hyp is the abscissa, which is along the rotation direction of hyperspectral camera from the rotation starting location; y
hyp is the ordinate, which is along the rotation vertical direction. The horizontal resolution is determined with the rotation speed of turntable and the vertical resolution is determined by the scanning speed of hyperspectral camera.Figure 2
Coordinate system of Hyperspectral Image.


## 2.1.2. Coordinate System of Laser Scans

Terrestrial 3D laser scans imaging with the line array scanning mode and the laser image is 3D point clouds, whose coordinate system is confirmed by the self-laser, as shown in Figure3, where O
las is the coordinate origin, which is the scanner electrooptical center; z
las is the scanner vertical rotation axis; x
las is along the optical axis scanner arbitrary horizontal angle, such as the first horizontal angle or the north direction of the built-in magnetic compass; y
las is orthogonal to x
las and z
las, which is formed by the right-hand system.Figure 3
Coordinate system of Laser Scan.


## 2.2. Definition of Camera Model

Rotary broom hyperspectral camera and Terrestrial 3D laser are both dependent on the cylindrical coordinate system, but their images are, respectively, 2D and 3D, so the camera model is formed to transform the 3D laser scan into the 2D laser image.This camera model is inferred by panoramic camera model [12, 13], as shown in Figure 4. The formula is shown in(1)
x
2
d-las
=
x
0
-
c
arctan
-
y
las
x
las
+
Δ
x
,
y
2
d-las
=
y
0
-
c
z
las
x
las
2
+
y
las
2
+
Δ
y
,where (
x
las
,
y
las
,
z
las
) is the point of laser scan; (
x
2
d-las
,
y
2
d-las
) is the point of 2D laser image; c is the principle distance of the cameral model; (
x
0
,
y
0
) is the principle point of 2D laser image; (
Δ
x
,
Δ
y
) is the correction parameter.Figure 4
Camera model.Through (1), the point of 2D laser image (
x
2
d-las
,
y
2
d-las
) is calculated, but the type of their values is double and the type of 2D image pixel is integer, so (
x
2
d-las
,
y
2
d-las
) must be changed into the integer value, as shown in (2)
x
2
d-las
′
=
int
x
2
d-las
-
min
⁡
⁡
x
2
d-las
max
⁡
⁡
x
2
d-las
-
min
⁡
⁡
x
2
d-las
×
int
max
⁡
⁡
x
2
d-las
-
min
⁡
⁡
x
2
d-las
A
m
,
y
2
d-las
′
=
int
y
2
d-las
-
min
⁡
⁡
y
2
d-las
max
⁡
⁡
y
2
d-las
-
min
⁡
⁡
y
2
d-las
×
int
max
⁡
⁡
y
2
d-las
-
min
⁡
⁡
y
2
d-las
A
n
,where (
x
2
d-las
′
,
y
2
d-las
′
) is the point position of the 2D image; A
m is the horizontal resolution; A
n is the vertical resolution.The pixels value of 2D laser image is defined by the attributes of the laser scan, which may be the information such as color, curvature, and normal, and so forth. The formula is as follows:(3)
color
=
col
-
min
⁡
⁡
y
max
⁡
⁡
y
-
min
⁡
⁡
y
×
255
,where color is the value of (
x
2
d-las
′
,
y
2
d-las
′
), ranged in [0–255]; col is the attribute value of (
x
las
,
y
las
,
z
las
); max
⁡
⁡
y and min
⁡
⁡
y are, respectively, the maximum and the minimum of this attribute.


## 3. Algorithm of Automatic Fusion

In this automatic fusion algorithm, first of all, the hyperspectral gray image(
image
hyp
) is extracted from the hyperspectral image and the 2D laser image (
image
las
) from the laser scan is created by the camera model. And then, the feature points of (
image
hyp
) and (
image
las
) are produced with SURF and SC-RANSAC, and the feature points of hyperspectral image and laser scan are generated with the inverse operation of camera model. The initial registration is achieved by Collinearity Equation and Direct Linear Transformation; the precision registration is completed through creating corrected values to eliminate errors with Adjustment. At length, the automatic fusion is accomplished by the registration result. This algorithm flowing chart is shown in Figure 5.Figure 5
Algorithm flowing chart.


### 3.1. Initial Registration

#### 3.1.1. Extraction of Feature Points

Feature Points between(
image
hyp
) and (
image
las
) are extracted by SURF algorithm [14], which is superior to SIFT algorithm in every aspect [15–17]. In order to eliminate the error feature points, SC-RANSAC [18] is used, which is the currently fastest RANSAC extension coalesced with RANSAC and spatial consistency check from the literature [19]. Moreover, the searching method, an important role to improve the speed, is the stochastic KD tree algorithm [20], which searches by multiple stochastic KD tree to advance searching nodes and whose accuracy and matching speed are better for high dimensional data search [21, 22].After completing the extraction of feature points, according to the inversion operation of camera model, the corresponding 3D feature points of(
image
las
) is found. If P
i (i
=
1
,
…
,
n) are the feature points of hyperspectral image and Q
i (i
=
1
,
…
,
n) are the corresponding feature points of laser scan, the corresponding feature points between them are found, too.


#### 3.1.2. Collinearity Equation

The Collinearity Equation, the basic equation of photogrammetry, is used to set up the mapping relationship between the 2D coordinate and the 3D coordinate. IfP
x
hyp
,
y
hyp in the hyperspectral coordinate system corresponds to Q
x
las
,
y
las
,
z
las in the laser coordinate system, the corresponding relation between them can be expressed as (4)
x
hyp
-
x
0
+
δ
x
=
-
f
a
1
x
las
+
b
1
y
las
+
c
1
z
las
a
3
x
las
+
b
3
y
las
+
c
3
z
las
y
hyp
-
y
0
+
δ
y
=
-
f
a
2
y
las
+
b
2
y
las
+
c
2
z
las
a
3
y
las
+
b
3
y
las
+
c
3
z
las
,where (a
i
,
b
i
,
c
i) i
=
1,2
,
3 is the direction cosine of rotation matrix; (
δ
x
,
δ
y
) is the system errors correction; f is the principal distance.Direct Linear Transformation (DLT) is the solution of direct linear relationship between the photo point coordinate and the corresponding object point coordinate, which is essentially a kind of space resection and space intersection method. This algorithm is applicable to a variety of no metric cameras without the known internal orientation elements and is also suitable for the close range photogrammetry of the large angle without the initial external orientation elements. According to DLT algorithm, (4) is translated into the following formula:(5)
x
las
1
y
las
1
z
las
1
1
0
0
0
0
x
hyp
1
x
las
1
x
hyp
1
y
las
1
x
hyp
1
z
las
1
0
0
0
0
x
las
1
y
las
1
z
las
1
1
y
hyp
1
x
las
1
y
hyp
1
y
las
1
y
hyp
1
z
las
1
x
las
2
y
las
2
z
las
2
1
0
0
0
0
x
hyp
2
x
las
2
x
hyp
2
y
las
2
x
hyp
2
z
las
2
0
0
0
0
x
las
2
y
las
1
z
las
2
1
y
hyp
2
x
las
2
y
hyp
2
y
las
2
y
hyp
2
z
las
2
⋮
⋮
⋮
x
las
n
y
las
n
z
las
n
1
0
0
0
0
x
hyp
n
x
las
n
x
hyp
n
y
las
n
x
hyp
n
z
las
n
0
0
0
0
x
las
n
y
las
1
n
z
las
n
1
y
hyp
n
x
las
n
y
hyp
n
y
las
n
y
hyp
n
z
las
n
l
1
l
2
l
3
l
4
l
5
l
6
l
7
l
8
l
9
l
10
l
11
=
-
x
hyp
1
-
y
hyp
1
-
x
hyp
2
-
y
hyp
2
⋮
-
x
hyp
n
-
y
hyp
n
,where l
i (i
=
1
,
…
,
11) are the coefficients. According to these coefficients, the interior orientation elements and the exterior orientation elements between hyperspectral coordinate and laser coordinate are calculated. The corresponding relation between P
(
x
hyp
,
y
hyp
) and Q
(
x
las
,
y
las
,
z
las
) is also calculated. However, because of the great errors of DTL, l
i (i
=
1
,
…
,
11) can only be considered the approximate results, and the corresponding relation results are the initial registration results.


### 3.2. Precise Registration

In order to accurately determine the corresponding relation between hyperspectral coordinate and laser coordinate, Adjustment is executed using the redundant observation values of the hyperspectral pixels. If the correction of the hyperspectral pixel observed value is(
v
x
,
v
y
), the corresponding relation between P
(
x
hyp
,
y
hyp
) and Q
(
x
las
,
y
las
,
z
las
) is shown in (6)
x
hyp
-
x
0
+
v
x
+
δ
x
=
-
f
a
1
x
las
+
b
1
y
las
+
c
1
z
las
a
3
x
las
+
b
3
y
las
+
c
3
z
las
,
y
hyp
-
y
0
+
v
y
+
δ
y
=
-
f
a
2
x
las
+
b
2
y
las
+
c
2
z
las
a
3
x
las
+
b
3
y
las
+
c
3
z
las
.IfA
=
l
9
x
las
+
l
10
y
las
+
l
11
z
las
+
1, the error equation of (
v
x
,
v
y
) is shown in (7)
v
x
=
-
1
A
x
las
l
1
+
y
las
l
2
+
z
las
l
3
+
l
4
x
hyp
-
x
0
r
2
+
x
hyp
x
las
l
9
+
x
hyp
y
las
l
10
+
x
hyp
z
las
l
11
+
A
x
hyp
-
x
0
r
2
k
1
+
x
hyp
v
y
=
-
1
A
x
las
l
5
+
y
las
l
6
+
z
las
l
7
+
l
8
A
y
hyp
-
y
0
r
2
k
1
+
y
hyp
x
las
l
9
+
y
hyp
y
las
l
10
+
y
hyp
z
las
l
11
+
A
y
hyp
-
y
0
r
2
k
1
+
y
hyp
,where k
1 is the symmetric radial distortion coefficient; r
=
(
x
hyp
-
x
0
)
2
+
(
y
hyp
-
y
0
)
2 is the radius vector of hyperspectral pixel.The operation is executed by least squares method, whose iterative condition is that the interpolation of adjacentf
x in f
x
=
A
cos
⁡
⁡
d
β is less than 0.01 mm. The calculating process of A value is also the iterative process, and each iterative A value is calculated by control point. Thus, the precise values of the coefficients (
l
1
,
l
2
,
…
,
l
11
,
k
1
) are calculated.


### 3.3. Data Fusion

IfHype
(
x
,
y
) is the arbitrary point of hyperspectral image and Point
(
X
,
Y
,
Z
) in laser scan is the corresponding point of Hype
(
x
,
y
), according to l
i (i
=
1
,
…
,
11), the corresponding relation between Hype
(
x
,
y
) and Point
(
X
,
Y
,
Z
) is calculated by (8)
x
=
-
l
1
X
+
l
2
Y
+
l
3
Z
+
l
4
l
9
X
+
l
10
Y
+
l
11
Z
+
1
,
y
=
-
l
5
X
+
l
6
Y
+
l
7
Z
+
l
8
l
9
X
+
l
10
Y
+
l
11
Z
+
1
.Therefore, based on (8), each point of laser data corresponds to each point of hyperspectral data.In caseHype
(
p
1
,
…
,
p
n
) is the spectral information of Hype
(
x
,
y
) in hyperspectral image, and Point
(
X
,
Y
,
Z
,
q
1
,
…
,
q
m
) is the property of Point
(
X
,
Y
,
Z
) in laser scan, in which q
1
,
…
,
q
m is the feature except the spatial coordinates (
X
,
Y
,
Z
), such as intensity, amplitude, and so forth, the hyperspectral data and the laser data are fused, and the property of Point
(
X
,
Y
,
Z
) is expressed by Point
(
X
,
Y
,
Z
,
q
1
,
…
,
q
m
,
p
1
,
…
,
p
n
) which includes the spatial information of laser data and the spectral information of hyperspectral data, based on the corresponding relationship between Point
(
X
,
Y
,
Z
) and Hype
(
x
,
y
).


## 3.1. Initial Registration

### 3.1.1. Extraction of Feature Points

Feature Points between(
image
hyp
) and (
image
las
) are extracted by SURF algorithm [14], which is superior to SIFT algorithm in every aspect [15–17]. In order to eliminate the error feature points, SC-RANSAC [18] is used, which is the currently fastest RANSAC extension coalesced with RANSAC and spatial consistency check from the literature [19]. Moreover, the searching method, an important role to improve the speed, is the stochastic KD tree algorithm [20], which searches by multiple stochastic KD tree to advance searching nodes and whose accuracy and matching speed are better for high dimensional data search [21, 22].After completing the extraction of feature points, according to the inversion operation of camera model, the corresponding 3D feature points of(
image
las
) is found. If P
i (i
=
1
,
…
,
n) are the feature points of hyperspectral image and Q
i (i
=
1
,
…
,
n) are the corresponding feature points of laser scan, the corresponding feature points between them are found, too.


### 3.1.2. Collinearity Equation

The Collinearity Equation, the basic equation of photogrammetry, is used to set up the mapping relationship between the 2D coordinate and the 3D coordinate. IfP
x
hyp
,
y
hyp in the hyperspectral coordinate system corresponds to Q
x
las
,
y
las
,
z
las in the laser coordinate system, the corresponding relation between them can be expressed as (4)
x
hyp
-
x
0
+
δ
x
=
-
f
a
1
x
las
+
b
1
y
las
+
c
1
z
las
a
3
x
las
+
b
3
y
las
+
c
3
z
las
y
hyp
-
y
0
+
δ
y
=
-
f
a
2
y
las
+
b
2
y
las
+
c
2
z
las
a
3
y
las
+
b
3
y
las
+
c
3
z
las
,where (a
i
,
b
i
,
c
i) i
=
1,2
,
3 is the direction cosine of rotation matrix; (
δ
x
,
δ
y
) is the system errors correction; f is the principal distance.Direct Linear Transformation (DLT) is the solution of direct linear relationship between the photo point coordinate and the corresponding object point coordinate, which is essentially a kind of space resection and space intersection method. This algorithm is applicable to a variety of no metric cameras without the known internal orientation elements and is also suitable for the close range photogrammetry of the large angle without the initial external orientation elements. According to DLT algorithm, (4) is translated into the following formula:(5)
x
las
1
y
las
1
z
las
1
1
0
0
0
0
x
hyp
1
x
las
1
x
hyp
1
y
las
1
x
hyp
1
z
las
1
0
0
0
0
x
las
1
y
las
1
z
las
1
1
y
hyp
1
x
las
1
y
hyp
1
y
las
1
y
hyp
1
z
las
1
x
las
2
y
las
2
z
las
2
1
0
0
0
0
x
hyp
2
x
las
2
x
hyp
2
y
las
2
x
hyp
2
z
las
2
0
0
0
0
x
las
2
y
las
1
z
las
2
1
y
hyp
2
x
las
2
y
hyp
2
y
las
2
y
hyp
2
z
las
2
⋮
⋮
⋮
x
las
n
y
las
n
z
las
n
1
0
0
0
0
x
hyp
n
x
las
n
x
hyp
n
y
las
n
x
hyp
n
z
las
n
0
0
0
0
x
las
n
y
las
1
n
z
las
n
1
y
hyp
n
x
las
n
y
hyp
n
y
las
n
y
hyp
n
z
las
n
l
1
l
2
l
3
l
4
l
5
l
6
l
7
l
8
l
9
l
10
l
11
=
-
x
hyp
1
-
y
hyp
1
-
x
hyp
2
-
y
hyp
2
⋮
-
x
hyp
n
-
y
hyp
n
,where l
i (i
=
1
,
…
,
11) are the coefficients. According to these coefficients, the interior orientation elements and the exterior orientation elements between hyperspectral coordinate and laser coordinate are calculated. The corresponding relation between P
(
x
hyp
,
y
hyp
) and Q
(
x
las
,
y
las
,
z
las
) is also calculated. However, because of the great errors of DTL, l
i (i
=
1
,
…
,
11) can only be considered the approximate results, and the corresponding relation results are the initial registration results.


## 3.1.1. Extraction of Feature Points

Feature Points between(
image
hyp
) and (
image
las
) are extracted by SURF algorithm [14], which is superior to SIFT algorithm in every aspect [15–17]. In order to eliminate the error feature points, SC-RANSAC [18] is used, which is the currently fastest RANSAC extension coalesced with RANSAC and spatial consistency check from the literature [19]. Moreover, the searching method, an important role to improve the speed, is the stochastic KD tree algorithm [20], which searches by multiple stochastic KD tree to advance searching nodes and whose accuracy and matching speed are better for high dimensional data search [21, 22].After completing the extraction of feature points, according to the inversion operation of camera model, the corresponding 3D feature points of(
image
las
) is found. If P
i (i
=
1
,
…
,
n) are the feature points of hyperspectral image and Q
i (i
=
1
,
…
,
n) are the corresponding feature points of laser scan, the corresponding feature points between them are found, too.


## 3.1.2. Collinearity Equation

The Collinearity Equation, the basic equation of photogrammetry, is used to set up the mapping relationship between the 2D coordinate and the 3D coordinate. IfP
x
hyp
,
y
hyp in the hyperspectral coordinate system corresponds to Q
x
las
,
y
las
,
z
las in the laser coordinate system, the corresponding relation between them can be expressed as (4)
x
hyp
-
x
0
+
δ
x
=
-
f
a
1
x
las
+
b
1
y
las
+
c
1
z
las
a
3
x
las
+
b
3
y
las
+
c
3
z
las
y
hyp
-
y
0
+
δ
y
=
-
f
a
2
y
las
+
b
2
y
las
+
c
2
z
las
a
3
y
las
+
b
3
y
las
+
c
3
z
las
,where (a
i
,
b
i
,
c
i) i
=
1,2
,
3 is the direction cosine of rotation matrix; (
δ
x
,
δ
y
) is the system errors correction; f is the principal distance.Direct Linear Transformation (DLT) is the solution of direct linear relationship between the photo point coordinate and the corresponding object point coordinate, which is essentially a kind of space resection and space intersection method. This algorithm is applicable to a variety of no metric cameras without the known internal orientation elements and is also suitable for the close range photogrammetry of the large angle without the initial external orientation elements. According to DLT algorithm, (4) is translated into the following formula:(5)
x
las
1
y
las
1
z
las
1
1
0
0
0
0
x
hyp
1
x
las
1
x
hyp
1
y
las
1
x
hyp
1
z
las
1
0
0
0
0
x
las
1
y
las
1
z
las
1
1
y
hyp
1
x
las
1
y
hyp
1
y
las
1
y
hyp
1
z
las
1
x
las
2
y
las
2
z
las
2
1
0
0
0
0
x
hyp
2
x
las
2
x
hyp
2
y
las
2
x
hyp
2
z
las
2
0
0
0
0
x
las
2
y
las
1
z
las
2
1
y
hyp
2
x
las
2
y
hyp
2
y
las
2
y
hyp
2
z
las
2
⋮
⋮
⋮
x
las
n
y
las
n
z
las
n
1
0
0
0
0
x
hyp
n
x
las
n
x
hyp
n
y
las
n
x
hyp
n
z
las
n
0
0
0
0
x
las
n
y
las
1
n
z
las
n
1
y
hyp
n
x
las
n
y
hyp
n
y
las
n
y
hyp
n
z
las
n
l
1
l
2
l
3
l
4
l
5
l
6
l
7
l
8
l
9
l
10
l
11
=
-
x
hyp
1
-
y
hyp
1
-
x
hyp
2
-
y
hyp
2
⋮
-
x
hyp
n
-
y
hyp
n
,where l
i (i
=
1
,
…
,
11) are the coefficients. According to these coefficients, the interior orientation elements and the exterior orientation elements between hyperspectral coordinate and laser coordinate are calculated. The corresponding relation between P
(
x
hyp
,
y
hyp
) and Q
(
x
las
,
y
las
,
z
las
) is also calculated. However, because of the great errors of DTL, l
i (i
=
1
,
…
,
11) can only be considered the approximate results, and the corresponding relation results are the initial registration results.


## 3.2. Precise Registration

In order to accurately determine the corresponding relation between hyperspectral coordinate and laser coordinate, Adjustment is executed using the redundant observation values of the hyperspectral pixels. If the correction of the hyperspectral pixel observed value is(
v
x
,
v
y
), the corresponding relation between P
(
x
hyp
,
y
hyp
) and Q
(
x
las
,
y
las
,
z
las
) is shown in (6)
x
hyp
-
x
0
+
v
x
+
δ
x
=
-
f
a
1
x
las
+
b
1
y
las
+
c
1
z
las
a
3
x
las
+
b
3
y
las
+
c
3
z
las
,
y
hyp
-
y
0
+
v
y
+
δ
y
=
-
f
a
2
x
las
+
b
2
y
las
+
c
2
z
las
a
3
x
las
+
b
3
y
las
+
c
3
z
las
.IfA
=
l
9
x
las
+
l
10
y
las
+
l
11
z
las
+
1, the error equation of (
v
x
,
v
y
) is shown in (7)
v
x
=
-
1
A
x
las
l
1
+
y
las
l
2
+
z
las
l
3
+
l
4
x
hyp
-
x
0
r
2
+
x
hyp
x
las
l
9
+
x
hyp
y
las
l
10
+
x
hyp
z
las
l
11
+
A
x
hyp
-
x
0
r
2
k
1
+
x
hyp
v
y
=
-
1
A
x
las
l
5
+
y
las
l
6
+
z
las
l
7
+
l
8
A
y
hyp
-
y
0
r
2
k
1
+
y
hyp
x
las
l
9
+
y
hyp
y
las
l
10
+
y
hyp
z
las
l
11
+
A
y
hyp
-
y
0
r
2
k
1
+
y
hyp
,where k
1 is the symmetric radial distortion coefficient; r
=
(
x
hyp
-
x
0
)
2
+
(
y
hyp
-
y
0
)
2 is the radius vector of hyperspectral pixel.The operation is executed by least squares method, whose iterative condition is that the interpolation of adjacentf
x in f
x
=
A
cos
⁡
⁡
d
β is less than 0.01 mm. The calculating process of A value is also the iterative process, and each iterative A value is calculated by control point. Thus, the precise values of the coefficients (
l
1
,
l
2
,
…
,
l
11
,
k
1
) are calculated.


## 3.3. Data Fusion

IfHype
(
x
,
y
) is the arbitrary point of hyperspectral image and Point
(
X
,
Y
,
Z
) in laser scan is the corresponding point of Hype
(
x
,
y
), according to l
i (i
=
1
,
…
,
11), the corresponding relation between Hype
(
x
,
y
) and Point
(
X
,
Y
,
Z
) is calculated by (8)
x
=
-
l
1
X
+
l
2
Y
+
l
3
Z
+
l
4
l
9
X
+
l
10
Y
+
l
11
Z
+
1
,
y
=
-
l
5
X
+
l
6
Y
+
l
7
Z
+
l
8
l
9
X
+
l
10
Y
+
l
11
Z
+
1
.Therefore, based on (8), each point of laser data corresponds to each point of hyperspectral data.In caseHype
(
p
1
,
…
,
p
n
) is the spectral information of Hype
(
x
,
y
) in hyperspectral image, and Point
(
X
,
Y
,
Z
,
q
1
,
…
,
q
m
) is the property of Point
(
X
,
Y
,
Z
) in laser scan, in which q
1
,
…
,
q
m is the feature except the spatial coordinates (
X
,
Y
,
Z
), such as intensity, amplitude, and so forth, the hyperspectral data and the laser data are fused, and the property of Point
(
X
,
Y
,
Z
) is expressed by Point
(
X
,
Y
,
Z
,
q
1
,
…
,
q
m
,
p
1
,
…
,
p
n
) which includes the spatial information of laser data and the spectral information of hyperspectral data, based on the corresponding relationship between Point
(
X
,
Y
,
Z
) and Hype
(
x
,
y
).


## 4. Experiment and Analysis

In order to verify the effectiveness of the algorithm, the experiments are conducted. The dataset of the hyperspectral data and the laser data were obtained from the electronic display board of the playground of Capital Normal University by our own laboratory. The 3D laser used was a Riegl LSM-420i and the hyperspectral imager was integrated by our laboratory, which main parameters are shown in Table1. The setup of data acquisition is shown in Figure 6. The algorithm code was written in Matlab with mex files written in Visual C++. The code was run on a dell computer with Inter i5CPU and 4 G RAM.Table 1
Hyperspectral camera parameters.
Name
Parameter
Spectrum range
400–1000 nm
Spectral resolution
2.8 nm
FOV
27.2°
Spatial resolution
1600 pixel
Spectrometer entrance slit width
30μm
Spectrometer entrance slit length
11.84 mm
Camera lens focal length
23 mm
Spectral band numbers
840Figure 6
Data acquisition map.The initial RGB image data of hyperspectral data is shown in Figure7(a). The initial laser scan is shown in Figure 7(b), whose horizontal and vertical angular resolution are 1° and whose horizontal and vertical spacing are about 7 cm, and the different colors show the intensity of laser scan.Figure 7
Initial data.
(a)
Hyperspectral Image
(b)
Laser Scan


### 4.1. Experiment

In this paper, because the geometry of electronic display board is very flat and the normal vectors are similar, so the point-cloud image is generated by the intensity as a pixel value and the image resolution is determined by the point-cloud distance. Then the points of hyperspectral image and laser scan are found by 2D-3D registration algorithm, and six pairs of feature points are chosen, as shown in Table2.Table 2
Feature points of Hyperspectral Image and Laser Scan.
Hyperspectral Image
Laser Scan
(78, 110)
(36.524, −2.535, 5.610)
(64, 748)
(35.819, −10.884, 5.804)
(369, 750)
(35.805, −11.016, 1.834)
(356, 292)
(36.378, −4.905, 2.058)
(600, 261)
(36.318, −4.564, −1.019)
(582, 667)
(35.895, −9.884, −0.898)According to Collinearity Equation and DLT in initial registration, the approximate coefficientsL
1 (l
1
,
l
2
,
…
,
l
11) are obtained, as shown in Table 3.Table 3
Approximate coefficients.
Approximate coefficients
L
1
(
l
1
,
l
2
,
…
,
l
11
)
l
1
l
2
l
3
l
4
l
5
l
6
l
7
l
8
l
9
l
10
l
11
11.7084
−0.9069
2.1177
−444.3699
7.4423
1.6245
−0.0975
−270.4678
−0.0265
0.0019
0.0002According to Adjustment in precise registration, the precise coefficientsL
2 (l
1
,
l
2
,
…
,
l
11) are obtained, as shown in Table 4.Table 4
Precise coefficients.
Precise coefficients
L
2
(
l
1
,
l
2
,
…
,
l
11
)
l
1
l
2
l
3
l
4
l
5
l
6
l
7
l
8
l
9
l
10
l
11
11.5679
−0.9045
1.6447
−436.0369
7.4790
1.1372
−0.0844
−272.3880
−0.0267
0.0020
0.0002The fusions of hyperspectral data and laser data are, respectively, executed byL
1 (l
1
,
l
2
,
…
,
l
11) and L
2 (l
1
,
l
2
,
…
,
l
11), as shown in Figure 8. The initial fusion image has basically been fused; nevertheless there are more great errors, as shown in Figure 8(a). For instance, the texts of electronic board have obvious deviation. The blue points represent no corresponding points, so the base of electronic board does not have the corresponding points. The precise fusion image has greatly been fused, as shown in Figure 8(b). The texts and the base of electronic board have been the corresponding fusion.Figure 8
Fusion images.
(a)
Fusion image of initial registration
(b)
Fusion image of precise registration


### 4.2. Evaluation of Precision

To further verify the effectiveness of this algorithm, other feature points are selected as the check points to verify its accuracy. Firstly, the internal orientation elements and the external orientation elements are calculated by the approximate solution algorithm, and the corresponding hyperspectral pixels of laser scan are calculated. Then, the corresponding hyperspectral pixels of laser scan are calculated by the precise solution algorithm. The comparison is shown in Table5. “Hyperspectral Image” is the hyperspectral coordinate, and “Laser Scan” is the laser coordinate. “Hyperspectral Image
L
1” is the hyperspectral coordinate calculated by the approximate solution algorithm, and “Hyperspectral Image
(
L
2
)” is the hyperspectral coordinate calculated by the precise solution algorithm.Table 5
The corresponding coordinates of check points.
Hyperspectral Image
Laser Scan
Hyperspectral Image (L
1)
Hyperspectral Image (L
2)
1
(105, 175)
(36.524, −3.241, 5.219)
(101.510, 163.671)
(104.060, 172.998)
2
(96, 752)
(35.819, −10.884, 5.804)
(90.316, 707.821)
( 99.710, 754.157)
3
(369, 772)
(35.805, −11.016, 1.834)
(368.462, 721.168)
(396.928, 73.661)
4
(425, 130)
(36.482, −2.656, 1.663)
(396.083, 120.457)
(423.808, 28.865)
5
(365, 317)
(36.325, −4.961, 2.052)
(360.131, 295.554)
(386.726, 316.734)
6
(120, 313)
(36.325, −4.900, 5.351)
(112.758, 295.324)
(121.541,314.931)
7
(430, 73)
(36.698, −2.672, 1.609)
(389.679, 81.230)
(426.318,72.521)
8
(385, 316)
(36.378, −4.905, 2.058)
(355.868, 291.805)
(383.162, 314.289)
9
(670, 755)
(36.133, −9.203, −0.839)
(593.862, 661.657)
(666.248, 752.267)
10
(609, 126)
(36.158, −1.873, −1.577)
(597.238, 112.234)
(611.544, 129.725)To verify the errors of the check points, the residual errors of the horizontal and vertical direction are, respectively, calculated based on the distance between “Hyperspectral Image” and “HyperspectralImage
(
L
1
)” and “Hyperspectral Image
(
L
2
),” as shown in Table 6. “d
x
1” and “d
y
1” are, respectively, the residual errors of the horizontal and vertical direction between “Hyperspectral Image” and “Hyperspectral Image
(
L
1
).” “d
x
2” and “d
y
2” are, respectively, the residual errors of the horizontal and vertical direction between “Hyperspectral Image” and “Hyperspectral Image
(
L
2
).” From Table 6, the horizontal residual mean errors are decreased from −20.8094 to 4.6046 and the vertical residual mean errors are decreased from −27.8079 to 0.1148. Precision is greatly improved.Table 6
Residual errors and residual mean errors of check points (unit: pixel).
d
x
1
d
x
2
d
y
1
d
y
2
1
−3.4900
−0.9400
−11.3293
−2.0016
2
−5.6836
3.7100
−44.1793
2.1570
3
−0.5383
27.9281
−50.8319
1.6609
4
−28.9171
−1.1921
−9.5426
−1.1347
5
−4.8696
21.7266
−21.4457
−0.2665
6
−7.2416
1.5411
−17.6757
1.9312
7
−40.3209
−3.6824
8.2298
−0.4789
8
−29.1323
−1.8383
−24.1954
−1.7114
9
−76.1384
−3.7516
−93.3432
−2.7327
10
−11.7618
2.5444
−13.7657
3.7246
Residual mean errors
−20.8094
4.6046
−27.8079
0.1148Moreover, the twenty check points are selected to verify this algorithm and the residual errors of the thirty check points are shown in Figure9. In agreement with the above analysis, the residual errors of precise registration shapely reduce, and the horizontal residual mean errors are decreased from −18.3751 to 1.5820 and the vertical residual mean errors are decreased from −16.3553 to −0.11167. To sum up, this method has reached a more satisfactory accuracy.Figure 9
Analysis of residual errors of check points.
(a)
Residual errors of check points ofx-axis
(b)
Residual errors of check points ofy-axisFor the dataset used with natural images, very few methods are appropriate to attempt comparison with. We implemented the other approaches. However no images were successfully aligned by these methods. This failure was expected. The following are the reasons. Firstly, the methods are very few and the initial conditions are so many. For example, the method in Nieto et al. [1] needs that color point clouds are created by the calibration of laser scanner and digital camera. The method in Zachary and Juan [3] needs that the initial site of the two sensors is obtained by GPS. However, the method in this paper does not need the initial site of the two sensors and the additional device such as digital camera and GPS. Moreover, there are some comparisons of the different registration methods of images and laser scans. For example, the method in Zhang et al. [10] applies the inspection line and collinear equation, but this algorithm is more suitable for the aerial image and airborne data. The method in Liu [23] needs to select manually 2D-3D the same points, which causes the great error of human factor.


## 4.1. Experiment

In this paper, because the geometry of electronic display board is very flat and the normal vectors are similar, so the point-cloud image is generated by the intensity as a pixel value and the image resolution is determined by the point-cloud distance. Then the points of hyperspectral image and laser scan are found by 2D-3D registration algorithm, and six pairs of feature points are chosen, as shown in Table2.Table 2
Feature points of Hyperspectral Image and Laser Scan.
Hyperspectral Image
Laser Scan
(78, 110)
(36.524, −2.535, 5.610)
(64, 748)
(35.819, −10.884, 5.804)
(369, 750)
(35.805, −11.016, 1.834)
(356, 292)
(36.378, −4.905, 2.058)
(600, 261)
(36.318, −4.564, −1.019)
(582, 667)
(35.895, −9.884, −0.898)According to Collinearity Equation and DLT in initial registration, the approximate coefficientsL
1 (l
1
,
l
2
,
…
,
l
11) are obtained, as shown in Table 3.Table 3
Approximate coefficients.
Approximate coefficients
L
1
(
l
1
,
l
2
,
…
,
l
11
)
l
1
l
2
l
3
l
4
l
5
l
6
l
7
l
8
l
9
l
10
l
11
11.7084
−0.9069
2.1177
−444.3699
7.4423
1.6245
−0.0975
−270.4678
−0.0265
0.0019
0.0002According to Adjustment in precise registration, the precise coefficientsL
2 (l
1
,
l
2
,
…
,
l
11) are obtained, as shown in Table 4.Table 4
Precise coefficients.
Precise coefficients
L
2
(
l
1
,
l
2
,
…
,
l
11
)
l
1
l
2
l
3
l
4
l
5
l
6
l
7
l
8
l
9
l
10
l
11
11.5679
−0.9045
1.6447
−436.0369
7.4790
1.1372
−0.0844
−272.3880
−0.0267
0.0020
0.0002The fusions of hyperspectral data and laser data are, respectively, executed byL
1 (l
1
,
l
2
,
…
,
l
11) and L
2 (l
1
,
l
2
,
…
,
l
11), as shown in Figure 8. The initial fusion image has basically been fused; nevertheless there are more great errors, as shown in Figure 8(a). For instance, the texts of electronic board have obvious deviation. The blue points represent no corresponding points, so the base of electronic board does not have the corresponding points. The precise fusion image has greatly been fused, as shown in Figure 8(b). The texts and the base of electronic board have been the corresponding fusion.Figure 8
Fusion images.
(a)
Fusion image of initial registration
(b)
Fusion image of precise registration


## 4.2. Evaluation of Precision

To further verify the effectiveness of this algorithm, other feature points are selected as the check points to verify its accuracy. Firstly, the internal orientation elements and the external orientation elements are calculated by the approximate solution algorithm, and the corresponding hyperspectral pixels of laser scan are calculated. Then, the corresponding hyperspectral pixels of laser scan are calculated by the precise solution algorithm. The comparison is shown in Table5. “Hyperspectral Image” is the hyperspectral coordinate, and “Laser Scan” is the laser coordinate. “Hyperspectral Image
L
1” is the hyperspectral coordinate calculated by the approximate solution algorithm, and “Hyperspectral Image
(
L
2
)” is the hyperspectral coordinate calculated by the precise solution algorithm.Table 5
The corresponding coordinates of check points.
Hyperspectral Image
Laser Scan
Hyperspectral Image (L
1)
Hyperspectral Image (L
2)
1
(105, 175)
(36.524, −3.241, 5.219)
(101.510, 163.671)
(104.060, 172.998)
2
(96, 752)
(35.819, −10.884, 5.804)
(90.316, 707.821)
( 99.710, 754.157)
3
(369, 772)
(35.805, −11.016, 1.834)
(368.462, 721.168)
(396.928, 73.661)
4
(425, 130)
(36.482, −2.656, 1.663)
(396.083, 120.457)
(423.808, 28.865)
5
(365, 317)
(36.325, −4.961, 2.052)
(360.131, 295.554)
(386.726, 316.734)
6
(120, 313)
(36.325, −4.900, 5.351)
(112.758, 295.324)
(121.541,314.931)
7
(430, 73)
(36.698, −2.672, 1.609)
(389.679, 81.230)
(426.318,72.521)
8
(385, 316)
(36.378, −4.905, 2.058)
(355.868, 291.805)
(383.162, 314.289)
9
(670, 755)
(36.133, −9.203, −0.839)
(593.862, 661.657)
(666.248, 752.267)
10
(609, 126)
(36.158, −1.873, −1.577)
(597.238, 112.234)
(611.544, 129.725)To verify the errors of the check points, the residual errors of the horizontal and vertical direction are, respectively, calculated based on the distance between “Hyperspectral Image” and “HyperspectralImage
(
L
1
)” and “Hyperspectral Image
(
L
2
),” as shown in Table 6. “d
x
1” and “d
y
1” are, respectively, the residual errors of the horizontal and vertical direction between “Hyperspectral Image” and “Hyperspectral Image
(
L
1
).” “d
x
2” and “d
y
2” are, respectively, the residual errors of the horizontal and vertical direction between “Hyperspectral Image” and “Hyperspectral Image
(
L
2
).” From Table 6, the horizontal residual mean errors are decreased from −20.8094 to 4.6046 and the vertical residual mean errors are decreased from −27.8079 to 0.1148. Precision is greatly improved.Table 6
Residual errors and residual mean errors of check points (unit: pixel).
d
x
1
d
x
2
d
y
1
d
y
2
1
−3.4900
−0.9400
−11.3293
−2.0016
2
−5.6836
3.7100
−44.1793
2.1570
3
−0.5383
27.9281
−50.8319
1.6609
4
−28.9171
−1.1921
−9.5426
−1.1347
5
−4.8696
21.7266
−21.4457
−0.2665
6
−7.2416
1.5411
−17.6757
1.9312
7
−40.3209
−3.6824
8.2298
−0.4789
8
−29.1323
−1.8383
−24.1954
−1.7114
9
−76.1384
−3.7516
−93.3432
−2.7327
10
−11.7618
2.5444
−13.7657
3.7246
Residual mean errors
−20.8094
4.6046
−27.8079
0.1148Moreover, the twenty check points are selected to verify this algorithm and the residual errors of the thirty check points are shown in Figure9. In agreement with the above analysis, the residual errors of precise registration shapely reduce, and the horizontal residual mean errors are decreased from −18.3751 to 1.5820 and the vertical residual mean errors are decreased from −16.3553 to −0.11167. To sum up, this method has reached a more satisfactory accuracy.Figure 9
Analysis of residual errors of check points.
(a)
Residual errors of check points ofx-axis
(b)
Residual errors of check points ofy-axisFor the dataset used with natural images, very few methods are appropriate to attempt comparison with. We implemented the other approaches. However no images were successfully aligned by these methods. This failure was expected. The following are the reasons. Firstly, the methods are very few and the initial conditions are so many. For example, the method in Nieto et al. [1] needs that color point clouds are created by the calibration of laser scanner and digital camera. The method in Zachary and Juan [3] needs that the initial site of the two sensors is obtained by GPS. However, the method in this paper does not need the initial site of the two sensors and the additional device such as digital camera and GPS. Moreover, there are some comparisons of the different registration methods of images and laser scans. For example, the method in Zhang et al. [10] applies the inspection line and collinear equation, but this algorithm is more suitable for the aerial image and airborne data. The method in Liu [23] needs to select manually 2D-3D the same points, which causes the great error of human factor.


## 5. Conclusion

A method for fusing the hyperspectral data with the laser data suitable for surface features is presented. This method operates by creating a 2D laser image using a camera model and extracting feature points of hyperspectral image and laser scan. The collinearity equation is used to create the correspondence to find correct alignment of hyperspectral image with laser scan. The adjustment is used to improve the registration accuracy of hyperspectral image with laser scan. The method was demonstrated to successfully work to fuse a hyperspectral image and a laser scan. In future work a dataset with nature environments will be obtained and the features will be more complex; therefore the feature extraction and the accuracy advance need further strengthening.



---

*Source: 415361-2015-07-28.xml*