# Study on Application of Scale Invariant Feature Transform Algorithm on Automated Geometric Correction of Remote Sensing Images

Hui Deng, Limin Wang, Jia Liu, Dandan Li, Zhongxin Chen, Qingbo Zhou

► To cite this version:

Hui Deng, Limin Wang, Jia Liu, Dandan Li, Zhongxin Chen, et al. Study on Application of Scale Invariant Feature Transform Algorithm on Automated Geometric Correction of Remote Sensing Images. 6th Computer and Computing Technologies in Agriculture (CCTA), Oct 2012, Zhangjiajie, China. pp.352-358, 10.1007/978-3-642-36137-1_41. hal-01348251

HAL Id: hal-01348251

https://inria.hal.science/hal-01348251

Submitted on 22 Jul 2016

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
Study on Application of Scale Invariant Feature
Transform Algorithm on Automated Geometric
Correction of Remote Sensing Images

Hui Deng¹,², Limin Wang¹,², Jia Liu¹,², Dandan Li¹,², Zhongxin Chen¹,², Qingbo Zhou¹,²

¹ Key Lab. of Resources Remote Sensing & Digital Agriculture, Ministry of Agriculture, 10081, Beijing, P. R. China

² Institute of Agriculture Resources and Regional Planning, Chinese Academy of Agricultural Sciences, 10081,Beijing, P. R. China

**Abstract.** In recent years, data collected from remote sensing satellite and aerophotography have been showing a geometric sequence increase. A method of Scale Invariant Feature Transform (SIFT) algorithm could be employed for the automatic geometric fine correction. This method could avoid the impact of the rotation and zooming of template matching during the image matching process, and it can also save the labor during the image processing operation. Based on the SIFT algorithm, this paper proposes a two-step method, which firstly conducts coarse match on feature points, and then further conducts fine correction on the coarsely matched feature points by using the least squares technique. The result indicates that, this method is an effective automatic matching method for remote sensing images.

**Keywords:** remote sensing image, automatic geometric correction, SIFT algorithm, least squares

# 1 Introduction

Along with the rapid increase of huge amount of remote sensing data, the workload of geometric fine correction on manual image matching for the image data collected from different sources and in different time period is huge. Image matching is the preliminary work of a lot of remote sensing applications (such as multi-source data fusion, variation detection, image mosaic, motion detection, and object identification, etc, and its result directly affects the quality of the follow-up works[1]. Currently, the matching technology for the satellite remote sensing images from the same sensor, at different time period, with small angular variation has been relatively mature, and some classical feature extraction algorithms, such as arithmetic operators[2] ,

¹ Foundation: The National High Technology Research and Development Program ("863" Program) of China (No. 2012BAH29B02, No. NC2010FB0002) and National Non-Profit Scientific Institution, Ministry of Finance of China(202-15)

First Author: Key Lab. of Resources Remote Sensing & Digital Agriculture, Ministry of Agriculture, Beijing 10081, PR China. E-mail address: denghuizxy@126.com.
arithmetic operators[3], are extracted. They proposes a multiscale registration technique using robust Scale Invariant Feature Transform (SIFT) features in Steerable-Domain[4]. The matching the images from multi-sensor with different resolutions is the focus and the difficulty of current researches[5]. Scholars both at home and abroad have conducted extensive in-depth research on this topic, but no perfect method, which could correctly match the multi-source image under different radiation distribution, dimension, rotation, and translation, has been established.

In recent years, scale invariant Feature Transform (SIFT) feature has achieved great success in image matching. The matching method based on the SIFT descriptor has been successfully applied in many fields, such as object identification [6], motion panorama[7], and Structure From Motion (SFM)[8]. SIFT algorithm was introduced [6] and [9] in 1999 and was well established in 2004. They also conducted experiments on the many representative descriptors under six circumstances, such as different scenes, change in illumination, and geometry deformation of images, resolution difference, rotation, blurring, and image compression. The results indicate that SIFT descriptor has the best performance [10].

## 2. Introduction to the algorithm

The extraction and matching algorithm based on SIFT point features mainly includes three steps: extraction of point features, computation of feature descriptors, and feature matching from [6] and [11].

The Gaussian smoothing is applied on original image with different standard deviation $σ$, to obtain the Gaussian differential image. The maximum or minimum gray-scale values on differential images are taken as the feature points. Formula (1) is gaussian smoothing formula, and the extraction process of feature points is illustrated in the Figure 1. In the Figure, each pixel on the Gaussian differential image (expressed by crosshair) is compared with 26 adjacent pixels (expressed by dot) within the 3X3 area of current and upper and lower adjacent dimensions. If the average values of Difference of Gaussian of a pixel is larger or smaller than the adjacent pixels, the pixel is taken as the local extreme point. Then, some local extreme points are eliminated according to the preset threshold value of Difference of Gaussian and edge effect standard, and remaining extreme points are taken as the final SIFT feature points.

$$G(x, y, \sigma) = \frac{1}{2\pi\sigma^2} e^{-(x^2+y^2)/2\rho^2} \quad (1)$$

By taking feature point as the center, the image area with specified height and width is taken, to calculate the gradient direction and gradient strength of each pixel within the area. The total gradient strength of different gradient directions are calculated and are taken as the eigen vector. The calculation process of the eigen vector is illustrated in the Figure 3. In the Figure, the arrow in the left stands for the gradient strength and direction. The total gradient strengths of 8 directions within the 2×2 sub-areas are calculated separately, so as to obtain an eigen vector with 32 dimensions. The calculation of Lowe's study (Lowe, 2004) was on 8 directions within a 4×4 sub-area, so the feature descriptor of each SIFT point feature is the eigen vector of 4×4×8=128 dimensions.
<figure><img src="image_2.png" /><figcaption>**Fig. 1.** SIFT point feature extraction process (Lowe, 2004)</figcaption></figure>

Euclidean distance of the eigen vector of different feature points of image to be matched and Reference image is calculated, and the feature point with minimum distance is taken as the initial matching point. The error matching points are eliminated according to the nearest neighbor and second nearest neighbor Euclidean distance ratio.

<figure><img src="image_3.png" /><figcaption>**Figure 2.** Feature descriptor calculation process (Lowe, 2004)</figcaption></figure>

**3. Analysis and Conclusion**

Firstly, feature point and eigen vector information of images to be corrected and Reference images is calculated. The partitioning technology is used to locate feature point pairs, so as to improve the feature point search efficiency, and then the coarse matching is conducted based on both images to find out the matched feature point pairs on both images. After the coarse matching, the fine matching on feature points
by using least squares algorithm, to ensure that the total error of the feature points
after fine matching is less than 0.5 pixel. Finally, the geometry fine correction based
on finely matched feature points is conducted.

<figure><img src="image_4.png" /><figcaption>**Figure 3.** Flow Chart of geometry fine correction</figcaption></figure>

Feature points and eigenvector refer to the local features of detected images (e.g. edge, angular point, outline, etc). Then it is necessary to combine and convert the features based on the need of matching targets, so as to form eigen vectors which are easy to be matched with good stability, to convert the image geometry correction issue into feature matching issue, and further to convert the feature matching issue into clustering issue of eigenvector of feature space. Eigen vector information is a set
of vectors with 128 dimensions. The following Figure illustrates part information of feature points and eigen vector of the Reference image.

The matching is conducted according to the similarity of eigen vector. Usually, various distance functions are taken as the similarity measurement of features, such as Euclidean distance and Mahalanobis distance, etc. The following figure is the schematic diagram of feature point location of two images after coarse matching. The SIFT algorithm is used to extract the 435 feature point pairs of two images.

A total of 50 pairs of feature points are randomly selected from 435 pairs of feature points and they are taken as the ground control points input into the ERDAS geometric correction procedure, to eventually get the error result, which is illustrated in the following Figure. In the figure, the error of the X direction is 4.9251, error of Y direction is 5.0334, and the total error is 7.0422.

<figure><img src="image_5.png" /><figcaption>Figure 4. Error result of coarse matching feature points as GCP</figcaption></figure>

The fine matching on the feature points of the coarse matching is conducted by using least squares algorithm, and the final matching accuracy is optimized. The specific operation process is as follows:

* To establish the dualistic quadratic regression equation between the coarsely matched feature points.

* The feature points are calculated based on the regression equation of each feature point of image to be corrected.

* The error between simulation feature points and the feature points on Reference image is calculated and the maximum error is calculated.

* When the maximum errors are larger than the system preset default parameter of 0.5, the feature point is deleted and the procedure return to
step 1 to recalculate; otherwise, the fine matching of feature point pairs is completed. Then exit the calculation.

The following figure is the schematic diagram of feature point location after fine matching. The number of the feature point pairs after fine matching is 134.

<figure><img src="image_6.png" /><figcaption>**Figure 5.** Schematic diagram of feature point pair location after fine matching</figcaption></figure>

A total of 134 pairs of feature points are input into the ERDAS geometric correction module, to eventually get the error result, which is illustrated in the following Figure. The error of X direction is 2.4689, error of Y direction is 2.1418, and the total error is 3.2685.

Resampling of corrected image is conducted by using the feature points after fine matching, to finish the final geometry fine correction.

The following figure is the schematic diagram of the location of the image after fine correction and the original Reference image.

A total of 20 pairs of control points are randomly selected from the image after fine correction and the original Reference images, to conduct the accuracy analysis, and the final error result is obtained, which is illustrated in the following Figure. The error of the X direction is 1.1018, the error of Y direction is 0.8426, and the total error is 1.3870.
The paper analyses the limitation of the conventional SIFT algorithm in the matching of remote sensing images, and the extraction time and matching accuracy of feature points is optimized. Optimization algorithm can extract more matched feature points with higher accuracy. Based on the SIFT algorithm improved by this paper, and by firstly conducting coarse matching and then fine correction, the error is reduced from 7.0422 to 1.3870. The accuracy of the automatic matching of remote sensing images is improved, which is suitable for the business operation. However, the expenses for the system calculation time is relatively high. The further optimization on algorithm in the aspect of parallel computation could be conducted so as to reduce the operation time.

## References

1. Zhang M Q. 2006. Research on the Line-Based Registration of Multi-Source Remote Sensing Imagery. Nanjing: Hohai University

2. Smith S M, Brady J M. SUSAN-A New Approach to Low Level Image Processing [J]. International Journal of Computer Vision, 1997, 23(1): 45-78

3. Harris C, Stephens M. A Combined Corner and Edge Detector[C]. Fourth Alvey Vision Conference. Manchester, UK, 1988:147-151

4. Xiangzeng Liu, Zheng Tian, Chunyan Chai, Huijing Fua. Multiscale registration of remote sensing image using robust SIFT features in Steerable-Domain[J]. 2011, 14(2): 63-72

5. Yves Dufournaud. Matching Images with Different Resolutions. Theory in Computer Vision[M]. Kluwer Academic Publishers, 2004

6. Lowe D G. Distinctive Image Features from Scale-Invariant Keypoints[ J ]. International Journal of Computer Vision, 2004,60(2) : 91—110.

7. Brown M, Lowe D G. Recognising Panoramas [A]. In Proceedings of the 9th

8. Schaffalitzky F, Zisserman A. Multi-view Matching for Unordered Image Sets, or How do I Organize my Holiday Snaps [ A ]Proceedings of the 7th European Conference on Computer Vision(ECCV02) [ C ]. 2002.

9. Lowe D G. Object Recognition from Local Scale-Invariant Features [C]. International Conference on Computer Vision, Corfu, Greece, 1999

10. Mikolajczyk K, Schmid C. A Performance Evaluation of Local Descriptors [J]. IEEE Trans Pattern Analysis and Machine Intelligence, 2005, 27 (10):161521 630

11. Li Q L, Wang G Y, Liu J G and Chen S B. Robust scale-invariant feature matching for remote sensing image registration[J], IEEE Geoscience and Remote Sensing Letters, 2009, 6(2): 287-291