# Image Mosaic Method Based on SIFT Features of Line Segment

Authors: Jun Zhu; Mingwu Ren
Journal: Computational and Mathematical Methods in Medicine
 (2014)

---

## Abstract

This paper proposes a novel image mosaic method based on SIFT (Scale Invariant Feature Transform) feature of line segment, aiming to resolve incident scaling, rotation, changes in lighting condition, and so on between two images in the panoramic image mosaic process. This method firstly uses Harris corner detection operator to detect key points. Secondly, it constructs directed line segments, describes them with SIFT feature, and matches those directed segments to acquire rough point matching. Finally, Ransac method is used to eliminate wrong pairs in order to accomplish image mosaic. The results from experiment based on four pairs of images show that our method has strong robustness for resolution, lighting, rotation, and scaling.


---

## Body

## 1. Introduction

Recently image mosaic [1–4] has been an important subject in image processing researches. Image mosaic technologies hold extensive potential applications in remote sensing image processing, computer recognition, medical image analysis, artificial intelligence, and other fields. And also there are a number of techniques for capturing panoramic images of real world scenes [5]. Since, in real word application, the input images are taken at varying orientations and exposures, a feature-based registration technique similar to the pieces of literature [2, 6] is used to automatically align the input images. The image matching accuracy will have a direct influence on quality of panoramic image. Currently, there are two types of methods for image matching: one is the grayscale-based method that uses the correlation of grayscale in overlapping regions of two images to obtain optimal matching through correlation maximizing. The grayscale-based method is easy to implement, but it is relatively sensitive to grayscale changes in images, especially under variable lighting. The other matching methods based on image features use image pixel values to extract features. Because these features are partially invariant to lighting changes, matching ambiguity would be excellently resolved in the process of image matching. As for the extraction of image feature points, there already have been many proved methods, for example, Harris method [3], Susan method [7], and Shi-Tomasi method [8]. This feature-based image mosaic method has two main advantages as follows: (1) the computation complexity of image matching will be significantly reduced for the reason that the image feature points are far less than pixels; (2) the feature points have strong robustness for unbalance lighting and noises; as a result, the quality of image mosaic would be improved.The methods to describe point features are mainly dependent on the description of image blocks [9], such as SIFT (Scale Invariant Feature Transform) method [10]. The pieces of literature in recent years [11–14] indicated that the researchers attached more and more importance to the improvement of SIFT-based matching accuracy while limiting the computation volume. The method of image feature description plays a vital role in the quality of image mosaic. And the performance is evaluated by robustness and speed. There have been several researches on how to improve the robustness and reduce the computation time [11, 15]. Inspired by the pieces of literature [16–18], we have known that the mesh feature of images has relatively strong robustness for image rotation and scaling. This paper proposes a matching method based on SIFT features of directed line segment in images. In order to improve the robustness and efficiency, similar to the pieces of literature [19, 20], we set our method as follows. It firstly uses Harris corner detection operator to extract key points and then constructs directed graph of extracted points. Secondly, it describes directed line segments with SIFT feature and matches them to attain rough matching of points. Finally, it adjusts matching points and eliminates wrong pairs through Ransac method to accomplish collage of images. The whole framework of our method can be seen from Figure 1. The method proposed here has the following major advantages: (1) the description based on features of mesh has strong robustness for image rotation, distortion, and scaling; (2) the description of directed line segment with SIFT features has certain robustness for image lighting influences and rotation; (3) rough point matching by statistical method could improve matching accuracy.Figure 1
Flowchart of our method.The remainder of our paper is structured as follows. Section2 reviews the Harris method and SIFT feature description. Section 3 describes our novel image mosaic method based on SIFT features of line segment. Experiments and analysis are demonstrated in Section 4. Conclusions are made in Section 5.


## 2. Harris Corner Detection and SIFT Feature Description

When SIFT method is adopted to detect feature points, the computation in the procedure of image pyramid construction, key point location determination by extreme value detection and others, will be of very time consuming. With the advantages of good performance of Harris operator to detect corner points, and so the combination of feature points detection by Harris method and SIFT description, the image mosaic will be accelerated.


### 2.1. Harris Corner Detection

Harris operator is a sort of signal-based point feature extraction operator proposed by Chris Harris [21] that is characterized with simple computation, homogeneous and reasonable extracted corner features, available quantitative extraction, and stable operator. Harris corner detection method applies the self-correlation function theory for signal processing that defines the point with both high row curvature and high line curvature. Harris method can be expressed as
(1)
M
=
G
(
s
~
)
⊗
(
g
x
2
g
x
g
y
g
x
g
y
g
y
2
)
,
in which g
x means the gradient at x direction, g
y means the gradient at y direction, G
(
s
~
) means the Gaussian template, and ⊗ means the convolution between Gaussian template and the function.On this basis, the response function of Harris corner detection method is(2)
R
=
det
⁡
⁡
(
M
)
-
k
(
trace
(
M
)
)
2
,
in which det
⁡
⁡
(
M
) means the matrix determinant, trace
(
M
) means the matrix trace, and k means a default constant, which generally is 0.04~0.06. In practical application, a point will be determined to be a corner point if its corner response function is greater than the given threshold value T. Different images have very largely different structures and textural features, which leads to their appropriate threshold value T which will differ in a large range. Later, Shi and Tomasi brought forward an improved method: if the smaller between two eigenvalues is greater than the minimum threshold value, it will get strong corners. The method proposed by Shi and Tomasi is relatively perfect and could obtain better results under many conditions.


### 2.2. SIFT Feature Description

SIFT [10] extracts invariant feature based on invariant descriptor that was proposed by Lowe in 2004. SIFT feature fundamentally remains invariant to image translation, rotation, scaling, brightness variation, and noises. SIFT feature description mainly includes two steps: (1) determine direction parameter of feature points; (2) use graphic information around feature points to construct 128-dimensional descriptor.


#### 2.2.1. The Determination of Direction Parameter

To ensure rotated invariance of descriptor of feature points, it shall calculate the main direction of feature points and create SIFT feature descriptor at this main direction. For the detected feature points, finite difference calculation will be applied to figure out pixel gradient modulem and angle of gradient amplitude θ in the region with the feature point as center. The formalists are as follows:
(3)
m
(
x
,
y
)
=
L
1
2
+
L
2
2
,
L
1
=
L
(
x
+
1
,
y
)
-
L
(
x
-
1
,
y
)
,
L
2
=
L
(
x
,
y
+
1
)
-
L
(
x
,
y
-
1
)
,
θ
(
x
,
y
)
=
arctan
⁡
(
L
(
x
,
y
+
1
)
-
L
(
x
,
y
-
1
)
L
(
x
+
1
,
y
)
-
L
(
x
-
1
,
y
)
)
,
in which L
(
x
,
y
) means pyramid image grayscale of the feature point at (
x
,
y
) on its scale. Then use histogram to statistically state pixel gradient module and direction in this region. The abscissa axis of histogram is the angle of amplitude of gradient direction, and the ordinate axis is the accumulated value of gradient module corresponding to gradient direction angle. The graphic of gradient direction is divided into 36 columns according to the range of 0°~360° that is each 10° is for a column. The peak value of histogram represents the direction of image gradient in neighborhood of this feature point, which is the main direction of this point, and selects 80% of peak value as auxiliary direction value. Therefore, one feature point could be set with many directions to enhance the robustness of matching.


#### 2.2.2. The Construction of SIFT Descriptor

Rotate local region around feature point byθ; the angle is represented by the main direction to maintain its rotational invariance. In the rotated region, equally divide the 16
×
16 rectangular window with the feature point as center into 4
×
4 subregions, and in each subregion, figure out gradient histogram of eight directions (0°, 45°, 90°, 135°, 180°, 225°, 270°, and 315°). In the same way, it is necessary to conduct Gaussian weighting processing to each pixel’s gradient magnitude. Therefore, each feature point will generate a 128-dimensional eigenvector.


## 2.1. Harris Corner Detection

Harris operator is a sort of signal-based point feature extraction operator proposed by Chris Harris [21] that is characterized with simple computation, homogeneous and reasonable extracted corner features, available quantitative extraction, and stable operator. Harris corner detection method applies the self-correlation function theory for signal processing that defines the point with both high row curvature and high line curvature. Harris method can be expressed as
(1)
M
=
G
(
s
~
)
⊗
(
g
x
2
g
x
g
y
g
x
g
y
g
y
2
)
,
in which g
x means the gradient at x direction, g
y means the gradient at y direction, G
(
s
~
) means the Gaussian template, and ⊗ means the convolution between Gaussian template and the function.On this basis, the response function of Harris corner detection method is(2)
R
=
det
⁡
⁡
(
M
)
-
k
(
trace
(
M
)
)
2
,
in which det
⁡
⁡
(
M
) means the matrix determinant, trace
(
M
) means the matrix trace, and k means a default constant, which generally is 0.04~0.06. In practical application, a point will be determined to be a corner point if its corner response function is greater than the given threshold value T. Different images have very largely different structures and textural features, which leads to their appropriate threshold value T which will differ in a large range. Later, Shi and Tomasi brought forward an improved method: if the smaller between two eigenvalues is greater than the minimum threshold value, it will get strong corners. The method proposed by Shi and Tomasi is relatively perfect and could obtain better results under many conditions.


## 2.2. SIFT Feature Description

SIFT [10] extracts invariant feature based on invariant descriptor that was proposed by Lowe in 2004. SIFT feature fundamentally remains invariant to image translation, rotation, scaling, brightness variation, and noises. SIFT feature description mainly includes two steps: (1) determine direction parameter of feature points; (2) use graphic information around feature points to construct 128-dimensional descriptor.


### 2.2.1. The Determination of Direction Parameter

To ensure rotated invariance of descriptor of feature points, it shall calculate the main direction of feature points and create SIFT feature descriptor at this main direction. For the detected feature points, finite difference calculation will be applied to figure out pixel gradient modulem and angle of gradient amplitude θ in the region with the feature point as center. The formalists are as follows:
(3)
m
(
x
,
y
)
=
L
1
2
+
L
2
2
,
L
1
=
L
(
x
+
1
,
y
)
-
L
(
x
-
1
,
y
)
,
L
2
=
L
(
x
,
y
+
1
)
-
L
(
x
,
y
-
1
)
,
θ
(
x
,
y
)
=
arctan
⁡
(
L
(
x
,
y
+
1
)
-
L
(
x
,
y
-
1
)
L
(
x
+
1
,
y
)
-
L
(
x
-
1
,
y
)
)
,
in which L
(
x
,
y
) means pyramid image grayscale of the feature point at (
x
,
y
) on its scale. Then use histogram to statistically state pixel gradient module and direction in this region. The abscissa axis of histogram is the angle of amplitude of gradient direction, and the ordinate axis is the accumulated value of gradient module corresponding to gradient direction angle. The graphic of gradient direction is divided into 36 columns according to the range of 0°~360° that is each 10° is for a column. The peak value of histogram represents the direction of image gradient in neighborhood of this feature point, which is the main direction of this point, and selects 80% of peak value as auxiliary direction value. Therefore, one feature point could be set with many directions to enhance the robustness of matching.


### 2.2.2. The Construction of SIFT Descriptor

Rotate local region around feature point byθ; the angle is represented by the main direction to maintain its rotational invariance. In the rotated region, equally divide the 16
×
16 rectangular window with the feature point as center into 4
×
4 subregions, and in each subregion, figure out gradient histogram of eight directions (0°, 45°, 90°, 135°, 180°, 225°, 270°, and 315°). In the same way, it is necessary to conduct Gaussian weighting processing to each pixel’s gradient magnitude. Therefore, each feature point will generate a 128-dimensional eigenvector.


## 2.2.1. The Determination of Direction Parameter

To ensure rotated invariance of descriptor of feature points, it shall calculate the main direction of feature points and create SIFT feature descriptor at this main direction. For the detected feature points, finite difference calculation will be applied to figure out pixel gradient modulem and angle of gradient amplitude θ in the region with the feature point as center. The formalists are as follows:
(3)
m
(
x
,
y
)
=
L
1
2
+
L
2
2
,
L
1
=
L
(
x
+
1
,
y
)
-
L
(
x
-
1
,
y
)
,
L
2
=
L
(
x
,
y
+
1
)
-
L
(
x
,
y
-
1
)
,
θ
(
x
,
y
)
=
arctan
⁡
(
L
(
x
,
y
+
1
)
-
L
(
x
,
y
-
1
)
L
(
x
+
1
,
y
)
-
L
(
x
-
1
,
y
)
)
,
in which L
(
x
,
y
) means pyramid image grayscale of the feature point at (
x
,
y
) on its scale. Then use histogram to statistically state pixel gradient module and direction in this region. The abscissa axis of histogram is the angle of amplitude of gradient direction, and the ordinate axis is the accumulated value of gradient module corresponding to gradient direction angle. The graphic of gradient direction is divided into 36 columns according to the range of 0°~360° that is each 10° is for a column. The peak value of histogram represents the direction of image gradient in neighborhood of this feature point, which is the main direction of this point, and selects 80% of peak value as auxiliary direction value. Therefore, one feature point could be set with many directions to enhance the robustness of matching.


## 2.2.2. The Construction of SIFT Descriptor

Rotate local region around feature point byθ; the angle is represented by the main direction to maintain its rotational invariance. In the rotated region, equally divide the 16
×
16 rectangular window with the feature point as center into 4
×
4 subregions, and in each subregion, figure out gradient histogram of eight directions (0°, 45°, 90°, 135°, 180°, 225°, 270°, and 315°). In the same way, it is necessary to conduct Gaussian weighting processing to each pixel’s gradient magnitude. Therefore, each feature point will generate a 128-dimensional eigenvector.


## 3. Directed Line Segment Matching

The description method based on features of line segment not only can acquire local information of images, such as textures and gradients, but also can be able to obtain image content between line segments and other information. Our method has two creative aspects: (1) it describes image features through the description of connecting line between key points, not through image blocks; (2) the description method based on line segment can reflect topological structures of image and therefore it has relatively high robustness for nonlinearly distorted and rotated images.


### 3.1. Line Segment Features

Given two imagesI and I
′ to be matched, we use Harris operator to detect key points in these two images and utilize detected key points to construct two directed graphics, G
=
(
V
,
E
) and G
′
=
(
V
′
,
E
′
). Define V
=
{
a
1
,
a
2
,
…
,
a
n
} and V
′
=
{
b
1
,
b
2
,
…
,
b
m
} that mean key points extracted from images I and I
′, respectively, and E and E
′ are the edge sets of directed graphics G and G
′, respectively, where E
=
{
(
a
i
,
a
j
)
,
i
≠
j
}, E
′
=
{
(
b
i
,
b
j
)
,
i
≠
j
}. Take the directed line segment between two key points as the object of feature description. Set one edge of graph G, e
i
j
∈
E (the starting point is a
i, and the end point is a
j). In order to reduce computation time of describing features of the edge e
i
j, we equidistantly sample five points from the edge e
i
j, {
p
1
,
p
2
,
p
3
,
p
4
,
p
5
}, in which p
k
=
p
i
+
(
(
k
-
1
)
/
4
)
(
p
j
-
p
i
), k
=
1
,
…
,
5; p
i refers to the image coordinate of point a
i. As to the feature points {
p
1
,
p
2
,
p
3
,
p
4
,
p
5
} sampled from the directed line segment e
i
j, extract their SIFT feature, respectively, S
=
(
s
1
,
s
2
,
s
3
,
s
4
,
s
5
). Each column in S, s
k, k
=
1,2
,
…
,
5, is a 128-dimensional vector, which represents SIFT feature of point p
k. Those feature points sampled uniformly from line segment have strong robustness for image scaling, rotation, and changes in resolution.


### 3.2. Nearest-Neighbor Matching of Line Segments

With the purpose of rough matching, a line segment-matching method based on nearest-neighbor criterion is proposed. It is assumed that imageI has n
1 directed line segments, L
=
[
l
1
,
l
2
,
…
,
l
n
1
], and image I
′ has n
2 directed line segments, L
′
=
[
l
1
′
,
l
2
′
,
…
,
l
n
1
′
], and a static matrix, K
∈
R
n
1
×
n
2, could be defined as
(4)
K
(
i
,
j
)
=
{
1
l
j
′
is
the
nearest
neighbor
points
of
l
i
0
otherwise
.
Because this paper uses a matrix to describe features of line segments, it takes F-norm of the matrix as measurement in computation of nearest-neighbor points; that is, d
(
l
i
,
l
j
′
)
=
∥
S
i
-
S
j
′
∥
F and S
i means features of line segment l
i; S
j
′ means features of line segment l
j
′. Computation procedure of matrix K is as shown in Procedure 1.Procedure 1:Method 1: computation procedure of matrix K.
(1) InitializeK
∈
R
n
1
×
n
2 into null matrix
(2) Fori
=
1,2
,
…
,
n
1
If l
j
′ is k neighboring point of l
i, K
(
i
,
j
)
=
1
(3) OutputK


### 3.3. Matching of Points

In last section, we obtain matching of directed line segments through the rule of nearest-neighborhood. It is necessary to get more accuracy of point matching to exact mosaic of image. With the sets of key points in two given images,V
=
{
a
1
,
a
2
,
…
,
a
n
} and V
′
=
{
b
1
,
b
2
,
…
,
b
m
}, we use the method based on statistical voting to obtain the frequency of point matching. As we know, if two straight lines match each other, their starting point and end point should match each other. In the first place, initiate a statistical matrix G
∈
R
n
×
m into null matrix; the computation procedure of G is as shown in Procedure 2. In matrix G, the larger element value indicates higher probability for corresponding point matching.Procedure 2:Method 2: computation procedure of matrix G.
(1) InitializeG
∈
R
n
×
m into null matrix
(2) Fori
=
1,2
,
…
,
n
1,  j
=
1,2
,
…
,
n
2
(i) IfK
(
i
,
j
)
=
1
(ii) According to the directed line segmentl
i is a
l
→
a
m, the directed line segment l
j
′ is b
p
→
b
q
(iii)G
(
l
,
p
)
=
G
(
l
,
p
)
+
1, G
(
m
,
q
)
=
G
(
m
,
q
)
+
1
(3) Output matrixGThe criteria to select matching points are:(1)
G
(
i
,
j
)
>
σ, in which σ is a proper positive number;
(2)
selecting the point corresponding to maximum elements in each row and each column as the matching point,G
(
i
,
j
)
>
σ, in which σ is a proper positive number, such as in G
(
i
,
j
), if the elements in row i and column j are maximum, a
i and b
j match each other; it will set all elements in row i and column j to be null;
(3)
if the maximum element in rowi and the maximum element in column j are not the same, it will randomly select one of them; select for example the maximum element in row i, G
(
i
,
q
), a
i and b
q match each othering; it will set all elements in row i and column q to be null.


## 3.1. Line Segment Features

Given two imagesI and I
′ to be matched, we use Harris operator to detect key points in these two images and utilize detected key points to construct two directed graphics, G
=
(
V
,
E
) and G
′
=
(
V
′
,
E
′
). Define V
=
{
a
1
,
a
2
,
…
,
a
n
} and V
′
=
{
b
1
,
b
2
,
…
,
b
m
} that mean key points extracted from images I and I
′, respectively, and E and E
′ are the edge sets of directed graphics G and G
′, respectively, where E
=
{
(
a
i
,
a
j
)
,
i
≠
j
}, E
′
=
{
(
b
i
,
b
j
)
,
i
≠
j
}. Take the directed line segment between two key points as the object of feature description. Set one edge of graph G, e
i
j
∈
E (the starting point is a
i, and the end point is a
j). In order to reduce computation time of describing features of the edge e
i
j, we equidistantly sample five points from the edge e
i
j, {
p
1
,
p
2
,
p
3
,
p
4
,
p
5
}, in which p
k
=
p
i
+
(
(
k
-
1
)
/
4
)
(
p
j
-
p
i
), k
=
1
,
…
,
5; p
i refers to the image coordinate of point a
i. As to the feature points {
p
1
,
p
2
,
p
3
,
p
4
,
p
5
} sampled from the directed line segment e
i
j, extract their SIFT feature, respectively, S
=
(
s
1
,
s
2
,
s
3
,
s
4
,
s
5
). Each column in S, s
k, k
=
1,2
,
…
,
5, is a 128-dimensional vector, which represents SIFT feature of point p
k. Those feature points sampled uniformly from line segment have strong robustness for image scaling, rotation, and changes in resolution.


## 3.2. Nearest-Neighbor Matching of Line Segments

With the purpose of rough matching, a line segment-matching method based on nearest-neighbor criterion is proposed. It is assumed that imageI has n
1 directed line segments, L
=
[
l
1
,
l
2
,
…
,
l
n
1
], and image I
′ has n
2 directed line segments, L
′
=
[
l
1
′
,
l
2
′
,
…
,
l
n
1
′
], and a static matrix, K
∈
R
n
1
×
n
2, could be defined as
(4)
K
(
i
,
j
)
=
{
1
l
j
′
is
the
nearest
neighbor
points
of
l
i
0
otherwise
.
Because this paper uses a matrix to describe features of line segments, it takes F-norm of the matrix as measurement in computation of nearest-neighbor points; that is, d
(
l
i
,
l
j
′
)
=
∥
S
i
-
S
j
′
∥
F and S
i means features of line segment l
i; S
j
′ means features of line segment l
j
′. Computation procedure of matrix K is as shown in Procedure 1.Procedure 1:Method 1: computation procedure of matrix K.
(1) InitializeK
∈
R
n
1
×
n
2 into null matrix
(2) Fori
=
1,2
,
…
,
n
1
If l
j
′ is k neighboring point of l
i, K
(
i
,
j
)
=
1
(3) OutputK


## 3.3. Matching of Points

In last section, we obtain matching of directed line segments through the rule of nearest-neighborhood. It is necessary to get more accuracy of point matching to exact mosaic of image. With the sets of key points in two given images,V
=
{
a
1
,
a
2
,
…
,
a
n
} and V
′
=
{
b
1
,
b
2
,
…
,
b
m
}, we use the method based on statistical voting to obtain the frequency of point matching. As we know, if two straight lines match each other, their starting point and end point should match each other. In the first place, initiate a statistical matrix G
∈
R
n
×
m into null matrix; the computation procedure of G is as shown in Procedure 2. In matrix G, the larger element value indicates higher probability for corresponding point matching.Procedure 2:Method 2: computation procedure of matrix G.
(1) InitializeG
∈
R
n
×
m into null matrix
(2) Fori
=
1,2
,
…
,
n
1,  j
=
1,2
,
…
,
n
2
(i) IfK
(
i
,
j
)
=
1
(ii) According to the directed line segmentl
i is a
l
→
a
m, the directed line segment l
j
′ is b
p
→
b
q
(iii)G
(
l
,
p
)
=
G
(
l
,
p
)
+
1, G
(
m
,
q
)
=
G
(
m
,
q
)
+
1
(3) Output matrixGThe criteria to select matching points are:(1)
G
(
i
,
j
)
>
σ, in which σ is a proper positive number;
(2)
selecting the point corresponding to maximum elements in each row and each column as the matching point,G
(
i
,
j
)
>
σ, in which σ is a proper positive number, such as in G
(
i
,
j
), if the elements in row i and column j are maximum, a
i and b
j match each other; it will set all elements in row i and column j to be null;
(3)
if the maximum element in rowi and the maximum element in column j are not the same, it will randomly select one of them; select for example the maximum element in row i, G
(
i
,
q
), a
i and b
q match each othering; it will set all elements in row i and column q to be null.


## 4. Experiment and Analysis

The experiments select four pairs of images taken by ordinary camera. In order to prove the effectiveness of our method, the selected four pairs of images vary largely in lighting, rotation, scaling, and resolution. In Figure2, the left image and the right image are different in resolution. Figure 7 shows that the objects in the two images are different in orientation. The two images in Figure 12 are taken under different lighting conditions, and the left one is exposed more time. Moreover, in Figure 17, the building in the left image is larger than the one in the right image. All images to be matched in the experiment are with the size of 461
×
346 in pixel.Images with different resolutions.
(a)
Low-resolution image
(b)
High-resolution image


### 4.1. Experiment on Images with Different Resolutions

In this experiment, we choose two images taken by an ordinary camera. The two images are preprocessed that the two images have different resolutions. Figure2(a) is a low-resolution image, so it looks blurred. In addition, Figure 2(b) looks more clean since it has high resolution. Figures 3–5, show the matching results from different methods. Figure 6 gives the last panoramic image stitched by our method.Figure 3
Results of Harris corner matching based on grayscale feature.Figure 4
Results of matching based on SIFT feature.Figure 5
Results of matching by the method proposed in our paper.Figure 6
Mosaic results by the method proposed in this paper.Object-rotated image; the building in both (a) and (b) is rotated.
(a)
(b)


### 4.2. Experiment on Rotated Images

In this experiment, the two images to be stitched in Figure7 were taken by ordinary camera. A building has different orientations in the two images because the position of camera was changed. Results of matching by different methods are shown in Figures 8–10. Moreover, Figure 11 gives the last panoramic image stitched by our method.Figure 8
Results of Harris corner matching based on grayscale feature.Figure 9
Results of matching based on SIFT feature.Figure 10
Results of matching by the method proposed in this paper.Figure 11
Mosaic results by the method proposed in this paper.Different lighting images.
(a)
Strong lighting image
(b)
Weak lighting image


### 4.3. Experiments on Different Lighting Condition Images

In this experiment, we choose two images sampled from original camera. The lighting conditions in the two images are largely different. Figure12(a) has longer exposure time. Figures 13–15 are the results of matching by different methods. And Figure 16 gives the panoramic image stitched by our method.Figure 13
Results of Harris corner matching based on grayscale feature.Figure 14
Results of matching based on SIFT feature.Figure 15
Results of matching by the method proposed in this paper.Figure 16
Mosaic results by the method proposed in this paper.Different scale images; the building in (a) is enlarged compared with the one in (b).
(a)
(b)


### 4.4. Experiment on Object Scaling Images

In this experiment, we choose two images that have different scale. The same building in the two images has different scales. Figure17(a) is taken when the lens of camera is zoomed relative to Figure 17(b). Therefore, the building in the left image is larger than the one in the right. Figures 18, 19, and 20 are the results of matching by several kinds of methods. Figure 21 gives the last panoramic image stitched by our method.Figure 18
Results of Harris corner matching based on grayscale feature.Figure 19
Results of matching based on SIFT feature.Figure 20
Results of matching by the method proposed in this paper.Figure 21
Mosaic results by the method proposed in this paper.


### 4.5. Experimental Observations and Discussion

Based on the four different experiments presented previously, a significant advantage of our method should be highlighted, that is, the accuracy of matching. According to the experimental results, we can draw the following conclusions.(1)
From Figures3–6, our method significantly outperforms both traditional method based on grayscale feature and method based on SIFT feature in precision for the matching. The performance of the method based on grayscale feature is so serious that it even cannot accomplish the last step. It may be that the grayscale feature will change largely as the resolution changes.
(2)
From Figures4 and 5, it can be demonstrated that the SIFT feature is robust to variance of resolution to some extent. It may be that the SIFT feature describes the local path of image. And our method outperforms the method based on the SIFT, since it extracts feature by describing the line in the image by SIFT feature.
(3)
From Figures8–10, both SIFT and our method can obtain good result, as SIFT feature is invariant to rotation. And the grayscale is sensitive to rotation.
(4)
From Figures13–15, both SIFT and our method can obtain good results, as SIFT feature is invariant to different light conditions. And the grayscale is sensitive to uneven illumination.
(5)
From Figures9, 10, 14, and 15, our method has a higher accuracy, it may be that our method includes more information and the statistical voting strategy could acquire more accurate matching pairs.
(6)
From Figures18–21, only the method proposed in the paper can obtain a good performance, as it could describe the lines of images. And the mean to describe the line is robust to object scaling. The methods based on the grayscale and SIFT cannot obtain good matching results; they even cannot accomplish the last stitching step.In summary, the method proposed in this paper has a remarkable performance in image matching, since it is robust to difference of resolution, image scaling, rotation, and lighting.


## 4.1. Experiment on Images with Different Resolutions

In this experiment, we choose two images taken by an ordinary camera. The two images are preprocessed that the two images have different resolutions. Figure2(a) is a low-resolution image, so it looks blurred. In addition, Figure 2(b) looks more clean since it has high resolution. Figures 3–5, show the matching results from different methods. Figure 6 gives the last panoramic image stitched by our method.Figure 3
Results of Harris corner matching based on grayscale feature.Figure 4
Results of matching based on SIFT feature.Figure 5
Results of matching by the method proposed in our paper.Figure 6
Mosaic results by the method proposed in this paper.Object-rotated image; the building in both (a) and (b) is rotated.
(a)
(b)


## 4.2. Experiment on Rotated Images

In this experiment, the two images to be stitched in Figure7 were taken by ordinary camera. A building has different orientations in the two images because the position of camera was changed. Results of matching by different methods are shown in Figures 8–10. Moreover, Figure 11 gives the last panoramic image stitched by our method.Figure 8
Results of Harris corner matching based on grayscale feature.Figure 9
Results of matching based on SIFT feature.Figure 10
Results of matching by the method proposed in this paper.Figure 11
Mosaic results by the method proposed in this paper.Different lighting images.
(a)
Strong lighting image
(b)
Weak lighting image


## 4.3. Experiments on Different Lighting Condition Images

In this experiment, we choose two images sampled from original camera. The lighting conditions in the two images are largely different. Figure12(a) has longer exposure time. Figures 13–15 are the results of matching by different methods. And Figure 16 gives the panoramic image stitched by our method.Figure 13
Results of Harris corner matching based on grayscale feature.Figure 14
Results of matching based on SIFT feature.Figure 15
Results of matching by the method proposed in this paper.Figure 16
Mosaic results by the method proposed in this paper.Different scale images; the building in (a) is enlarged compared with the one in (b).
(a)
(b)


## 4.4. Experiment on Object Scaling Images

In this experiment, we choose two images that have different scale. The same building in the two images has different scales. Figure17(a) is taken when the lens of camera is zoomed relative to Figure 17(b). Therefore, the building in the left image is larger than the one in the right. Figures 18, 19, and 20 are the results of matching by several kinds of methods. Figure 21 gives the last panoramic image stitched by our method.Figure 18
Results of Harris corner matching based on grayscale feature.Figure 19
Results of matching based on SIFT feature.Figure 20
Results of matching by the method proposed in this paper.Figure 21
Mosaic results by the method proposed in this paper.


## 4.5. Experimental Observations and Discussion

Based on the four different experiments presented previously, a significant advantage of our method should be highlighted, that is, the accuracy of matching. According to the experimental results, we can draw the following conclusions.(1)
From Figures3–6, our method significantly outperforms both traditional method based on grayscale feature and method based on SIFT feature in precision for the matching. The performance of the method based on grayscale feature is so serious that it even cannot accomplish the last step. It may be that the grayscale feature will change largely as the resolution changes.
(2)
From Figures4 and 5, it can be demonstrated that the SIFT feature is robust to variance of resolution to some extent. It may be that the SIFT feature describes the local path of image. And our method outperforms the method based on the SIFT, since it extracts feature by describing the line in the image by SIFT feature.
(3)
From Figures8–10, both SIFT and our method can obtain good result, as SIFT feature is invariant to rotation. And the grayscale is sensitive to rotation.
(4)
From Figures13–15, both SIFT and our method can obtain good results, as SIFT feature is invariant to different light conditions. And the grayscale is sensitive to uneven illumination.
(5)
From Figures9, 10, 14, and 15, our method has a higher accuracy, it may be that our method includes more information and the statistical voting strategy could acquire more accurate matching pairs.
(6)
From Figures18–21, only the method proposed in the paper can obtain a good performance, as it could describe the lines of images. And the mean to describe the line is robust to object scaling. The methods based on the grayscale and SIFT cannot obtain good matching results; they even cannot accomplish the last stitching step.In summary, the method proposed in this paper has a remarkable performance in image matching, since it is robust to difference of resolution, image scaling, rotation, and lighting.


## 5. Conclusions

This paper proposed a new image mosaic method based on SIFT feature of directed line segment. This method has strong robustness for resolution, rotation, lighting, and scaling. The line-based description method proposed here has much robustness for image rotation and scaling; the description of directed line segment with SIFT feature can better avoid uneven lighting; and rough matching on the basis of statistical voting can acquire more accurate matching pairs and improve the quality of image mosaic.



---

*Source: 926312-2014-01-06.xml*