# Homogenized ORB Algorithm Using Dynamic Threshold and Improved Quadtree

Authors: Chaoqun Ma; Xiaoguang Hu; Jin Xiao; Guofeng Zhang
Journal: Mathematical Problems in Engineering
 (2021)

---

## Abstract

The Oriented FAST and Rotated BRIEF (ORB) algorithm has the problem that the extracted feature points are overconcentrated or even overlapped, leading to information loss of local image features. A homogenized ORB algorithm using dynamic thresholds and improved quadtree method is proposed in this paper, named Quadtree ORB (QTORB). In the feature point extraction stage, a new dynamic local threshold calculation method is proposed to enhance the algorithm’s ability to extract feature points at homogeneous regions. Then, a quadtree method is improved and adopted to manage and optimize feature points to eliminate those excessively concentrated and overlapping feature points. Meanwhile, in the feature points optimization process, different quadtree depths are set at different image pyramid levels to prevent excessive splitting of the quadtree and increase calculation speed. In the feature point description stage, local gray difference value information is introduced to enhance the saliency of the feature description. Finally, the Hamming distance is used to match points and RANSAC is used to avoid mismatches. Two datasets, namely, the optical image dataset and SAR image dataset, are used in the experiment. The experimental result shows that, considering accuracy and real-time efficiency, the QTORB can effectively improve the distribution uniformity of feature points.


---

## Body

## 1. Introduction

Image registration plays an important role in computer vision. Image registration is widely used in many aspects such as image matching [1–7], change detection [8, 9], 3D reconstruction [10–12], guidance [13–15], mapping sciences [16–21], and mobile robot [22, 23]. In general, image registration methods can be mainly divided into two kinds: gray-scale matching methods and feature-based matching methods. The image gray-scale matching method uses similarity measures such as correlation function, covariance function, and absolute judgement to determine the correspondence between two images. The image feature-based matching method mainly compares two images by analyzing the spatial location, color, texture, and shape to achieve the correct matching. Feature-based methods are widely concerned and adopted. Many improvements are also based on such algorithms, based on the reason that they are robust against variations in geometry and illumination [24]. Feature-based techniques are usually regarded as an issue of matching process between points due to the generality of point representation and easy extraction. This paper is also based on point-based methods.Currently, there are many matching methods using image features. Feature-based description methods are mainly made up of two groups considering the different data types used to form the description: floating feature methods and binary feature methods. As for feature extraction algorithms based on floating-point description, there are many famous algorithms proposed in the past years. Among them, Scale-Invariant Feature Transform (SIFT) [25] is an excellent and outstanding algorithm. David Lowe firstly proposed the SIFT in 1999 and improved it furthermore in 2004. SIFT has strong matching ability, stable extracted feature points, and strong robustness under many conditions. SIFT adds a heavy burden to the estimate, however. The computational efficiency of SIFT is severely restricted by feature point detection method using extreme-scale space. Meanwhile, the process of feature description on gradient histogram also causes a high time cost. Without compromising its efficiency too much, this weakness has led to comprehensive efforts to optimize its speed [26]. The Speeded-Up Robust Features, shortly named as SURF, algorithm was proposed in 2008 by Bay et al. [27] with reference to the SIFT algorithm. SURF has a high degree of appreciation for feature points, and it has well robustness under the conditions of illumination, viewing angle, and scale changes. At the same moment, to a considerable degree, it solves the shortcomings of high calculation complexity and long calculation time of the SIFT algorithm.On the other hand, in recent years, numerous binary descriptor algorithms have been proposed [28–33]. The key to improving the performance of image matching algorithms is to generate descriptors with higher discrimination and less computation. Since binary descriptors are compact and can use bit operations to speed up matching, the required memory and calculations of binary descriptors are greatly reduced compared to nonbinary descriptors. Rublee et al. firstly [34] proposed an Oriented FAST and Rotated BRIEF (ORB) algorithm in 2011, which is essentially a combination of the Features from Accelerated Segment Test (FAST) [35] corner detection algorithm and the improvement of the Binary Robust Independent Elementary Features (BRIEF) [36] binary feature description algorithm. The ORB algorithm generates binary descriptors according to the point pairs selected by machine learning and introduces direction information to enhance the rotation invariance of the descriptor. ORB has fast calculation speed and low storage space consumption. Its calculation time is about one percent of SIFT and one tenth of SURF. However, the traditional ORB algorithm has poor matching quality and anti-interference ability. Based on the ORB algorithm, researchers have proposed many improved image matching algorithms. At the IEEE International Conference on Computer Vision (ICCV) conference in 2011, Leutenegger et al. proposed the Binary Robust Invariant Scalable Keypoints (BRISK) algorithm [29] which has better rotation invariance, scale invariance, and robustness. The BRISK algorithm mainly uses FAST9-16 for feature points detection and constructs an image pyramid to obtain scale invariance. For feature points description, the BRISK algorithm uses a circular sampling pattern to obtain binary descriptors. In 2015, Yang et al. proposed a local difference binary feature description method called Local Difference Binary (LDB) [37]. The LDB algorithm is improved based on the classic binary descriptor (just considering the regional intensity information), adding the gradient information in the horizontal and vertical directions to construct the descriptor. Meanwhile, LDB divides the feature point area into subblocks of different sizes, and the small subblocks can obtain enough local detailed information to improve the discrimination of the descriptor. Large subblocks can effectively remove noise, but are not sensitive to small changes, reducing the uniqueness of the descriptor. In 2017, Levi et al. proposed the Learned Arrangements of Three Patch Codes (LATCH) [38]. As an improvement on the original local binary feature description, the LATCH algorithm uses multi-image block F norm comparison replacing the comparison of pixels’ gray levels in ORB, which enhances the antinoise performance of the algorithm. In 2017, Mur-Arta [39, 40] uses the quadtree method to achieve the extraction of ORB feature homogenization. This method obviously improves the uniformity of the feature point distribution, but the algorithm operation time is obviously increased, and the efficiency needs to be improved. In 2019, Xin-Nan et al. used an adaptive threshold to replace the fixed threshold artificially set in the original ORB algorithm and were able to extract more feature points. However, this method requires the calculation of threshold value for each pixel, which results in the increase of algorithm operation time [41]. In 2020, Ma et al. proposed a feature description method combining multiple image block comparison method and added local gray difference information based on the original gray intensity information to improve the precision of the ORB algorithm [42].This paper researched the feature points detection and feature points description stage. Through analyzing the principle of ORB algorithm, the following shortcomings were concluded from ORB algorithm:(1)
For feature points detection, the ORB algorithm adopts the FAST point extraction algorithm. FAST compares the gray values of pixels with surrounding pixels to select the more prominent pixels in the image as feature points. The FAST algorithm, however, uses an arbitrarily set global fixed threshold, and the effect is that the detected feature points are too clustered or even overlapping. In areas where the image changes are not obvious, only a few or even no feature points can be extracted. Generally speaking, the more feature points, the more accurate the image matching obtained, but the dense distribution of feature points is very unfavorable for the subsequent feature description and will affect the accuracy of the system when applied to certain scenes [43], such as video tracking and image navigation.(2)
The ORB adopts the enhanced BRIEF description method to describe the feature points. The BRIEF algorithm generates binary feature descriptions by randomly selecting pixels in the image window of feature points to compare gray values. But in the process of comparing the gray value of pixels, only the relationship between the pixel points gray value is used. However, the image information contained in the pixel points gray value difference itself is not used, which will cause the loss of image information.This paper focuses on the problems of the ORB algorithm mentioned above and proposes an enhanced ORB algorithm. The following contributions are principally made:(1)
Mur-Arta [39, 40] adopted the FAST feature point extraction method using double threshold and the feature point management and optimization method using quadtree. The algorithm improved in Mur-Arta [39, 40] significantly improves the distribution uniformity of the feature points. However, this algorithm still uses artificially set thresholds and does not consider the specific conditions of the pixel neighborhood image, and it cannot truly achieve adaptive extraction. This paper makes an improvement, based on the FAST, by using dynamic local threshold. This method calculates the local threshold based on the neighborhood image block to improve the feature point extraction ability of the original ORB algorithm in the area where the image change is not obvious.(2)
Mur-Arta [39, 40] used a quadtree-based feature point management and optimization method to eliminate excessively concentrated and overlapping feature points. However, there is no constraint on the depth of the quadtree when the quadtree nodes are split, which creates too many splits and decreases the computational efficiency of the algorithm. This paper sets a maximum depth of the quadtree for different pyramid levels. By this way, the estimation of redundant feature points is thus decreased, and the performance of feature extraction is increased.(3)
In the feature point description stage, this paper converts the gray value difference information between pixel point pairs into a binary code string through a specified threshold. Then this binary code string and the binary code string generated by the original ORB algorithm are fused to obtain a combined binary final feature description. By this method, each descriptor obtains more spatially supported visual information to make feature description less sensitive to noise.The structure of the remaining chapters of this article is shown below. The feature points detection and description used in ORB are explained in Section2. The improved feature point detection method, feature point optimization method, and feature point description technique proposed in this paper are illustrated in Section 3. The experimental indicators and outcomes on the optical image dataset and SAR image dataset are discussed in Section 4. The topic is discussed in Section 5. Section 6 is the conclusion.


## 2. Feature Point Detection and Description of ORB Algorithm

The ORB algorithm, respectively, uses the FAST method and the BRIFE method in the feature point extraction and feature point description stages. And ORB algorithm has, respectively, made improvements to the defects of the FAST method and the BRIEF method.In the feature point extraction stage, the theory of the FAST is that the gray difference values between a pixel point and sufficient pixel points in its neighborhood are measured separately. Count the number of the gray difference values that exceeds the selected thresholdt from these gray differences. If this number exceeds a specific number, the central point is regarded as a feature point. On the one hand, FAST has the disadvantage of being sensitive to noise. Therefore, FAST cannot calculate the direction information that is required for the description of the feature. It is necessary to assign a direction to each feature point in advance. Therefore, ORB calculates the main direction for the feature point by the gray centroid process. The gray centroid method assumes that, from the feature point, the gray intensity center of the feature point neighborhood patch is offset. The vector from the feature point to its neighborhood window gray intensity center is defined to calculate an orientation [25]. On the other hand, FAST algorithm cannot extract feature points with scale invariance. To overcome this shortcoming, the ORB algorithm first constructs a scale space image pyramid and then uses the FAST to generate feature points at every layer image of the pyramid. The feature points selected by this method have scale invariance.In the feature point description stage, the main idea of the BRIEF method is making use of random selected pixels comparison to describe a point. A binary string can be obtained from the comparison between the gray values of selected pixel pairs. Considering the operation speed, the significance and discrimination, the ORB algorithm uses a 256-dimensional descriptor. In general, the BRIEF method is very noise sensitive. Some improvement has been made by the ORB algorithm to enhance the antinoise ability of the BRIEF. First, define an image window centered on the feature point. The size of the window is set to31×31 pixels. Then, randomly select enough number of 5×5 pixels size image blocks pairs in the image window. In the generation of the binary description vector, the gray integral of these 5×5 pixels image size block pairs are used instead of pixel pairs. Meanwhile, the BRIEF algorithm does not have rotational invariance. The gray centroid method is adopted in ORB algorithm to measure the directions of the feature point. Since that both the FAST method and the BRIEF method have the advantage of fast computing speed, the ORB algorithm is very prominent in computing speed.


## 3. Homogenized ORB Algorithm Based on Dynamic Threshold and Improved Quadtree

Our work mainly improves the feature point extraction and feature description stage. First, this paper proposes to calculate thresholds based on the gray values of different pixel neighborhoods instead of manually set thresholds, to improve the ability of our algorithm to extract feature points in homogeneous image regions. Secondly, an improved quadtree method for feature points management and optimization is adopted to improve the distribution uniformity of the detected feature points. Meanwhile, this paper set a specific limit quadtree splitting depth for each image pyramid to prevent excessive node splitting. In the feature description stage, the image information contained in the gray value difference is adopted to obtain more information support to enhance the discrimination of description.


### 3.1. FAST Feature Point Extraction Based on Dynamic Local Threshold

In feature point extraction, the FAST makes use of the gray difference values between a pixel point and its surrounding 16- pixels. The number of points extracted largely depends on the thresholdt mentioned in Section 2. Therefore, for different image regions, different t values should be selected based on concrete image environment. The threshold set by the ORB algorithm for FAST is an artificially set global threshold, without considering the specific situation of the local neighborhood of the pixel points. Therefore, the feature points detected by the ORB algorithm are concentrated in the areas where the image changes are obvious, and there are a lot of redundant feature points. For the uniform area of the image, the FAST algorithm can only extract a few feature points because of the similar gray-scale features of the image. Based on the ORB algorithm, Mur-Artal [39, 40] adopted a double-threshold for the FAST corner detection method. A high threshold is used in image areas with obvious features, while a low threshold is used in image areas where feature information is not obvious. By this method, the algorithm proposed by Mur-Artal [39. 40] can extract more feature points than ORB. However, its two thresholds are also artificially set, which has certain limitations and cannot be admirably adapted to all areas of the image.In this paper, we adopt the dynamic local threshold method. First, the entire image is divided into grids according to the set grid size. Each grid serves as a separate image area. In each grid, the thresholdt is calculated as follows:(1)t=α×1m∑i=1mfQi−fQ¯,where α is a scale factor, m is the number of pixels points in the grid, fQi is the gray value of the pixel point Qi in the grid, and fQ¯ is the average gray value of all pixel points in the grid. This paper verifies that when the value of α is 1.2, our algorithm can make a better matching effect.


### 3.2. Feature Point Management and Optimization Using the Improved Quadtree Method

The FAST algorithm based on dynamic local threshold proposed in Section3.1 is first used to complete the feature point extraction. To further homogenize the feature points in the image, the quadtree method is used to divide the two-dimensional space of the image into blocks. Then, the feature points in each block are processed. The detailed steps are as follows:Step 1. Take the entire image as the initial node of the quadtree segmentation to get the initial quadtree structure.Step 2. Judge all nodes in the image. Assume that Nkp is the number of feature points contained in the node. If Nkp is equal to 0, delete this node. If Nkp is equal to 1, the node is no longer divided. If Nkp is greater than 1, the node continues to split into four child nodes.Step 3. Repeat the second step until the number of nodes reaches the set number of required feature points, then the split of the quadtree ends.Step 4. Select the feature points contained in each node. If Nkp is greater than 1, the feature point with the largest Harris response value is selected to present this node.However, in the above steps, since the depth of the quadtree is not specifically set, the number of splits is too much, which brings a heavy computing load. This paper sets a different splitting depth for the quadtree based on the expected feature points in each pyramid image layer, to reduce the calculation of redundant features. The detailed process is shown in Algorithm1. In Algorithm 1, d refers to the of depth of the current quadtree; dmax refers to the maximum depth specifically set for current pyramid layer. Nkp refers to the number of feature points in the node. Nstore refers to the number of stored nodes, and Nset refers to the expected number of feature points for the current pyramid layer. The maximum depth of the quadtree dmax should satisfy the following formula:(2)4dmax≥Nset.Algorithm 1: Algorithm for improved quadtree method.
(1) Initialize node(2)
Split child node(3)ifd<dmax then(4)ifNkp>0 then(5)ifNkp>1 then(6)   Split child node(7)else then(8)   Store child node(9)else then(10)  Store child node(11)else then(12)ifNstore>Nset then(13) Store the point which has the largest response in each node(14)end


### 3.3. Feature Description Method Fused with Gray Difference Information

As for the feature point description stage, the ORB adopts the steer BRIEF algorithm. As described in Section2, in the feature point neighborhood image block, the BRIEF uses the gray value relationship of random pixel pairs to decide if the corresponding bit of the final binary feature definition is 0 or 1. But this method only uses the size of the gray value, and the quantization information of the difference between the gray values is not used, resulting in the loss of part of the image information. This paper proposes to use both gray value size information and gray value difference quantization information to describe feature points. The specific principle is expressed in Figure 1.Figure 1
Binary description combined with gray difference value information.In Figure1, this paper is built on basis of the improved BRIEF in ORB. We introduce the image information contained in the gray difference value of the pixel block pairs. Define a feature point Q; according to the description method in Section 2, the ORB feature description method selects T groups of pixel block pairs for gray value comparison and obtains a binary code string, which is assumed to be DW. Then, the gray difference value of each blocks pair is further recorded to generate another binary code string. A dataset G is defined to record the gray difference values:(3)G=Gtt=1…T=fAt−fBtt=1…T,where f is the mean gray value of the image block.For each feature point,T gray difference values can be obtained. Since the gray difference value is a floating-point value, it cannot be directly converted to a binary code. Therefore, the average value Gaverage of these T gray difference values is calculated as a threshold to binary-code these gray difference values:(4)Gaverage=GT=1T∑1≤t≤TtGt.Compare theT gray difference values with the average Gaverage:(5)hQ,Gt=1,Gaverage<Gt,0,otherwise.Based on equation (5), a binary string Dw∧ is generated:(6)Dw∧=∑1≤t≤Tt2thQ,Gt.Finally, as the process in Figure1, DW and Dw∧ are combined to generate a new binary description:(7)Descriptionkeypoint=DW+Dw∧.


### 3.4. The Overall Workflow of the Improved ORB Algorithm

QTORB algorithm proposed in this work still follows the relevant steps of the ORB algorithm in the overall process as shown in Figure2.Figure 2
The overall workflow of the QTORB.The detailed steps of QTORB algorithm are as follows in Algorithm2. T is set as 128. f is the gray mean function.Algorithm 2: Detailed operation steps of QTORB.
Input: Image dataOutput1: Feature pointsOutput2: Feature points descriptions(1)
Build Gauss pyramid bases on input image. The number of layers is set to 8 and the scale is 1.2(2)  Gridding pyramid image. The grid size is set to30×30. In each grid, calculate threshold using Equation (1). Traverse all meshes of all pyramid images to detect feature points setQ=Q1,Q2,…,Qi,…,QN(3)  Optimize the feature points using improved quadtree method(4)fori=1,2,…,Ndo(5)  Define image blockM of 31×31pixels size around point Qi(6)  InM, randomly select Tpatches pairs with the centers of patches are RP=RPtt=1…T=xt1,yt1,xt2,yt2t=1...T(7)  Define5×5 size patches Wt1,Wt2 centered on point xt1,yt1and xt2,yt2, S=Wtt=1...T=Wt1,Wt2t=1...T(8)fort=1,2,…,Tdo(9)iffWt1<fWt2 then Corresponding bit = 1(10)else then Corresponding bit = 0(11)end if(12)end for (generate binary code DW)(13)  Calculate the gray difference based on all pairsG=Gtt=1…T=fWt1−fWt2t=1…T, calculate the mean Gaverage of all differences(14)fort=1,2,…,Tdo(15)ifGaverage<Gt then Corresponding bit = 1(16)else then Corresponding bit = 0(17)end if(18)end for (generate binary code Dw∧)(19)DW and Dw∧ are combined to obtain final Descriptionkeypoint for each feature point(20)end for


## 3.1. FAST Feature Point Extraction Based on Dynamic Local Threshold

In feature point extraction, the FAST makes use of the gray difference values between a pixel point and its surrounding 16- pixels. The number of points extracted largely depends on the thresholdt mentioned in Section 2. Therefore, for different image regions, different t values should be selected based on concrete image environment. The threshold set by the ORB algorithm for FAST is an artificially set global threshold, without considering the specific situation of the local neighborhood of the pixel points. Therefore, the feature points detected by the ORB algorithm are concentrated in the areas where the image changes are obvious, and there are a lot of redundant feature points. For the uniform area of the image, the FAST algorithm can only extract a few feature points because of the similar gray-scale features of the image. Based on the ORB algorithm, Mur-Artal [39, 40] adopted a double-threshold for the FAST corner detection method. A high threshold is used in image areas with obvious features, while a low threshold is used in image areas where feature information is not obvious. By this method, the algorithm proposed by Mur-Artal [39. 40] can extract more feature points than ORB. However, its two thresholds are also artificially set, which has certain limitations and cannot be admirably adapted to all areas of the image.In this paper, we adopt the dynamic local threshold method. First, the entire image is divided into grids according to the set grid size. Each grid serves as a separate image area. In each grid, the thresholdt is calculated as follows:(1)t=α×1m∑i=1mfQi−fQ¯,where α is a scale factor, m is the number of pixels points in the grid, fQi is the gray value of the pixel point Qi in the grid, and fQ¯ is the average gray value of all pixel points in the grid. This paper verifies that when the value of α is 1.2, our algorithm can make a better matching effect.


## 3.2. Feature Point Management and Optimization Using the Improved Quadtree Method

The FAST algorithm based on dynamic local threshold proposed in Section3.1 is first used to complete the feature point extraction. To further homogenize the feature points in the image, the quadtree method is used to divide the two-dimensional space of the image into blocks. Then, the feature points in each block are processed. The detailed steps are as follows:Step 1. Take the entire image as the initial node of the quadtree segmentation to get the initial quadtree structure.Step 2. Judge all nodes in the image. Assume that Nkp is the number of feature points contained in the node. If Nkp is equal to 0, delete this node. If Nkp is equal to 1, the node is no longer divided. If Nkp is greater than 1, the node continues to split into four child nodes.Step 3. Repeat the second step until the number of nodes reaches the set number of required feature points, then the split of the quadtree ends.Step 4. Select the feature points contained in each node. If Nkp is greater than 1, the feature point with the largest Harris response value is selected to present this node.However, in the above steps, since the depth of the quadtree is not specifically set, the number of splits is too much, which brings a heavy computing load. This paper sets a different splitting depth for the quadtree based on the expected feature points in each pyramid image layer, to reduce the calculation of redundant features. The detailed process is shown in Algorithm1. In Algorithm 1, d refers to the of depth of the current quadtree; dmax refers to the maximum depth specifically set for current pyramid layer. Nkp refers to the number of feature points in the node. Nstore refers to the number of stored nodes, and Nset refers to the expected number of feature points for the current pyramid layer. The maximum depth of the quadtree dmax should satisfy the following formula:(2)4dmax≥Nset.Algorithm 1: Algorithm for improved quadtree method.
(1) Initialize node(2)
Split child node(3)ifd<dmax then(4)ifNkp>0 then(5)ifNkp>1 then(6)   Split child node(7)else then(8)   Store child node(9)else then(10)  Store child node(11)else then(12)ifNstore>Nset then(13) Store the point which has the largest response in each node(14)end


## 3.3. Feature Description Method Fused with Gray Difference Information

As for the feature point description stage, the ORB adopts the steer BRIEF algorithm. As described in Section2, in the feature point neighborhood image block, the BRIEF uses the gray value relationship of random pixel pairs to decide if the corresponding bit of the final binary feature definition is 0 or 1. But this method only uses the size of the gray value, and the quantization information of the difference between the gray values is not used, resulting in the loss of part of the image information. This paper proposes to use both gray value size information and gray value difference quantization information to describe feature points. The specific principle is expressed in Figure 1.Figure 1
Binary description combined with gray difference value information.In Figure1, this paper is built on basis of the improved BRIEF in ORB. We introduce the image information contained in the gray difference value of the pixel block pairs. Define a feature point Q; according to the description method in Section 2, the ORB feature description method selects T groups of pixel block pairs for gray value comparison and obtains a binary code string, which is assumed to be DW. Then, the gray difference value of each blocks pair is further recorded to generate another binary code string. A dataset G is defined to record the gray difference values:(3)G=Gtt=1…T=fAt−fBtt=1…T,where f is the mean gray value of the image block.For each feature point,T gray difference values can be obtained. Since the gray difference value is a floating-point value, it cannot be directly converted to a binary code. Therefore, the average value Gaverage of these T gray difference values is calculated as a threshold to binary-code these gray difference values:(4)Gaverage=GT=1T∑1≤t≤TtGt.Compare theT gray difference values with the average Gaverage:(5)hQ,Gt=1,Gaverage<Gt,0,otherwise.Based on equation (5), a binary string Dw∧ is generated:(6)Dw∧=∑1≤t≤Tt2thQ,Gt.Finally, as the process in Figure1, DW and Dw∧ are combined to generate a new binary description:(7)Descriptionkeypoint=DW+Dw∧.


## 3.4. The Overall Workflow of the Improved ORB Algorithm

QTORB algorithm proposed in this work still follows the relevant steps of the ORB algorithm in the overall process as shown in Figure2.Figure 2
The overall workflow of the QTORB.The detailed steps of QTORB algorithm are as follows in Algorithm2. T is set as 128. f is the gray mean function.Algorithm 2: Detailed operation steps of QTORB.
Input: Image dataOutput1: Feature pointsOutput2: Feature points descriptions(1)
Build Gauss pyramid bases on input image. The number of layers is set to 8 and the scale is 1.2(2)  Gridding pyramid image. The grid size is set to30×30. In each grid, calculate threshold using Equation (1). Traverse all meshes of all pyramid images to detect feature points setQ=Q1,Q2,…,Qi,…,QN(3)  Optimize the feature points using improved quadtree method(4)fori=1,2,…,Ndo(5)  Define image blockM of 31×31pixels size around point Qi(6)  InM, randomly select Tpatches pairs with the centers of patches are RP=RPtt=1…T=xt1,yt1,xt2,yt2t=1...T(7)  Define5×5 size patches Wt1,Wt2 centered on point xt1,yt1and xt2,yt2, S=Wtt=1...T=Wt1,Wt2t=1...T(8)fort=1,2,…,Tdo(9)iffWt1<fWt2 then Corresponding bit = 1(10)else then Corresponding bit = 0(11)end if(12)end for (generate binary code DW)(13)  Calculate the gray difference based on all pairsG=Gtt=1…T=fWt1−fWt2t=1…T, calculate the mean Gaverage of all differences(14)fort=1,2,…,Tdo(15)ifGaverage<Gt then Corresponding bit = 1(16)else then Corresponding bit = 0(17)end if(18)end for (generate binary code Dw∧)(19)DW and Dw∧ are combined to obtain final Descriptionkeypoint for each feature point(20)end for


## 4. Experiments and Results

A desktop computer with 64-bit Windows 10 system was used. The specific parameters are as follows: i7-7700, 3.60 GHz, and 16 GB. The simulation environment is Microsoft Visual Studio 2019. Simulation language adopts C++ combined with OpenCV 3.4.0. The test image datasets are the Oxford dataset [44] and the SAR dataset.The comparative test algorithms include two types of matching algorithms: floating-point feature description and binary feature description. Algorithms based on floating-point feature description include SIFT [25] and SURF [27]. Algorithms based on binary feature description include BRISK [29], LATCH [38], LDB [37], TPLGD proposed in [42], ORB [34], the algorithm proposed in [39, 40] (in this paper, the symbol MRA refers to this algorithm), and the QTORB proposed in this paper. The detailed steps information of each algorithm is shown in Table 1. To further avoid mismatches, the RANSAC [45] algorithm is used for all tested algorithms. If the true match error is within 3 pixels, the keypoint pair is deemed to be the right match.Table 1
The detailed related steps information of each algorithm.
Tested algorithmsDetection methodDescription methodMatching distanceDescription sizeError thresholdMismatch eliminationSIFTDOGSIFTEuclidean128-dimension3 pixelsRANSACSURFHessianSURFEuclidean64-dimension3 pixelsRANSACBRISKBRISKBRISKHamming64-byte3 pixelsRANSACLATCHFASTLATCHHamming32-byte3 pixelsRANSACLDBFASTLDBHamming64-byte3 pixelsRANSACTPLGDFASTTPLGDHamming64-byte3 pixelsRANSACORBFASTBRIEFHamming32-byte3 pixelsRANSACMRAFASTBRIEFHamming32-byte3 pixelsRANSACQTORBFASTQTORBHamming64-byte3 pixelsRANSAC


### 4.1. Evaluation Metrics

Four measurement criteria are used in experiment. Evenness is the first criterion. The Evenness [46] calculation setups are shown in Figure 3. Its basic principle is to use the mathematical characteristics between the number of feature points in the same area of various shapes of the image to measure the distribution of feature points. First, the image is divided into ten blocks from five directions: vertical, horizontal, 45 degrees, 135 degrees, internal, and external. By the division method in Figure 3, ten different regions can be obtained.Figure 3
Divide the image in multiple directions.Count the number of feature points in each area as a dataset. Calculate the varianceV of this dataset. The calculation of Evenness is shown in equation (8). The smaller the Evenness value, the better the uniform distribution of feature points.(8)Evenness=101×logV.The second criterion is Precision [44]. The calculation formula is equation (9). Numcorrect matches refers to the number of correct matches while Numall matches refers to the number of all matched points.(9)Precision=Numcorrect matchesNumall matches×100%.The third criterion is the Root-Mean-Square Error (RMSE). RMSE represents the positional distance between the feature point obtained by the matching algorithm and the real feature point. Assume that feature pointApxap,yap and Bqxbq,ybq is a matched point pair obtained by the image matching algorithm. Feature point Apxap,yap is located in image A, while Bqxbq,ybq is located in image B. The true position of point Apxap,yap on image B is Bp∧xbp∧,ybp∧. Then, the true position distance between point Bqxbq,ybq and point Bp∧xbp∧,ybp∧ is as follows:(10)RMSE=xbp−xbp∧2+ybq−ybp∧2.Image operation time is the fourth criterion. In this paper, algorithm operation time includes feature point extraction time, description time, and matching time. Input process time and indicators calculation time are not considered.


### 4.2. Oxford Optical Image Dataset

Provided by Mikolajczyk and Schmid [44], the Oxford dataset is a publicly measurable dataset. There are eight image classes in the dataset including image blur (Trees group and Bikes group), light (Leuven group), JPG compression (Ubc group), rotation scale conversion (Bark group, Boat group), and visual improvements (Wall group and Graf group). As shown in Figure 4, we will pick six image classes from the Oxford dataset.Figure 4
Optical images from Oxford dataset. (a) Bikes (blur). (b) Boat (rotation and scale). (c) Graf (viewpoint). (d) Leuven (illumination). (e) Trees (blur). (f) Ubc (JPG compression).
(a)(b)(c)(d)(e)(f)There are six images prepared for experimentation in each image group. As an input image pair, the first image can be paired with the other five images. The homography matrix between two images in each image pair is provided. The ground truth of the two images can be calculated based on this homography matrix.


### 4.3. SAR Dataset

For tests, a SAR image dataset acquired by the UAV-borne SAR platform is used. These SAR images include several typical scenes of objects on the ground, including cities, lakes, fields, roads, hills, and rivers, in Bedfordshire, southeast England. For a SAR image, rotation and zoom operations are used to create an image to be matched. The SAR image and its image to be matched are matched as an input image pair. As for the ground truth for each image pair, the homography matrix between two images in each image pair can be calculated according to the specific angle degree and scale ratio. In Figure5, some SAR images are shown.Figure 5
Eight SAR images. (a) Hills. (b) Lakes. (c) Town. (d) Road. (e) River. (f) Field. (g) Field. (h) Field.
(a)(b)(c)(d)(e)(f)(g)(h)


### 4.4. Experiment Result on Oxford Dataset

Six image groups from the Oxford dataset are selected for experiment including Bikes (blur), Boat (rotation and scale), Graf (viewpoint), Leuven (illumination), Trees (blur), and Ubc (JPG compression). To ensure the generality of the test results, each image pair is tested thirty times. The average value of each data is used as the final experimental data.


#### 4.4.1. Feature Points Evenness and Matching Effect on Oxford Dataset

The SIFT, SURF, BRISK, LATCH, LDB, TPLGD, ORB, MRA, and QTORB algorithms performed feature point extraction and image matching on the images selected from the Oxford dataset. From the distribution of feature points before and after matching, the performance difference of uniform distribution of feature points of the nine algorithms tested can be intuitively found.Figure6 shows us the feature point extraction results of the tested algorithms using the Bikes images group from the Oxford dataset. It can be intuitively found that the feature points extracted by the LATCH, LDB, TPLGD, and ORB algorithms are concentrated in image areas including more prominent edges while almost no feature points are extracted in the homogeneous area. Although the SIFT, SURF, and BRISK algorithms can extract feature points in the homogeneous area of the image, the number of feature points in the homogeneous area is not much. Most of the feature points are still concentrated in the edge area of the image, resulting in a large amount of overlap. As for the MRA and QTORB, the feature points are obviously more uniform than the other tested algorithms. The uniform difference between the MRA and QTORB algorithm is not obvious, but the QTORB algorithm can extract some feature points that the MRA cannot extract in some homogeneous area.Figure 6
The distribution of feature points extracted by the (a) SIFT, (b) SURF, (c) BRISK, (d) LATCH, (e) LDB, (f) TPLGD, (g) ORB, (h) MRA, and (i) QTORB algorithms on the Bikes group image from Oxford dataset
(a)(b)(c)(d)(e)(f)(g)(h)(i)Based on the extracted feature points above, we continue to complete the image matching process. Figure7 shows us the image matching effect of the tested algorithms using the Bikes images group from the Oxford dataset. The feature point pair connected by the green line is a correct matching pair, and the feature point pair connected by the red line is a wrong matching pair. Consistent with the distribution of feature points shown in Figure 6, Figure 7 shows that the correct matching feature points implemented by SIFT, SURF, BRISK, LATCH, LDB, TPLGD, and ORB algorithms also have a relatively serious centralized distribution. The matching results achieved by the MRA and QTORB algorithms are very uniform, and there are correct matches in both the homogeneous area and the edge area. The matching results achieved by the MRA and QTORB are relatively similar, making it difficult to distinguish the difference intuitively. To analyze the performance differences of the tested algorithms in more detail, the feature distribution uniformity of the selected six images groups of Oxford dataset is counted, as shown in Table 2.Figure 7
Image matching effect of the (a) SIFT, (b) SURF, (c) BRISK, (d) LATCH, (e) LDB, (f) TPLGD, (g) ORB, (h) MRA, and (i) QTORB algorithms on the Bikes group image from Oxford dataset.
(a)(b)(c)(d)(e)(f)(g)(h)(i)Table 2
Mean evenness of feature points using Oxford dataset images.
MethodBikesBoatGrafLeuvenTreesUbcSIFT192.5424203.3804159.8065234.0446207.2986253.872SURF254.0986201.3406159.55948273.6258231.2498269.886BRISK196.2948234.982170.94194251.4328255.885289.5048LATCH173.5252137.4798116.26842174.9244140.3472181.9418LDB179.6288143.53562120.0294177.5232144.188185.8752TPLGD174.0654141.31146116.88112175.7902151.1302180.313ORB171.8528148.43458120.23718175.1806146.18394180.0906MRA141.277111.0517484.45352158.6698112.15136168.1602QTORB140.647692.4308484.3705168.1638113.32876166.0178From Table2, under the six scene transformations provided by Oxford dataset, both MRA and QTORB can achieve more uniform feature point extraction than the other tested algorithms and provide a more reasonable distribution of feature points for subsequent image processing. Meanwhile, compared with MRA, the QTORB can better adapt to more scenarios.


#### 4.4.2. Matching Precision Using Oxford Dataset Images

This paper has carried out image matching tests on the six images groups provided by Oxford dataset. In each group, five images pairs (from 1–2 to 1–6, the more backward the image pairs, the greater the change between images) need to be matched. The precision of each matching process is shown in Figure8. The symbol “Mean” represents the average precision of the five matching processes in each group. The average precision data is counted in Table 3.Figure 8
Matching precision based on the Oxford dataset. (a) Bikes (blur). (b) Boat (rotation and scale). (c) Graf (viewpoint). (d) Leuven (illumination). (e) Trees (blur). (f) Ubc (JPG compression).
(a)(b)(c)(d)(e)(f)Table 3
Mean precision using Oxford dataset images (%).
MethodBikesBoatGrafLeuvenTreesUbcMeanSIFT81.799583.628844.512190.141473.138997.649778.4784SURF76.279558.699332.377881.073757.160681.430064.5035BRISK80.606974.971845.069093.724761.264594.118874.9593LATCH79.083666.133947.575384.302168.510196.426873.6720LDB75.150860.604543.558684.862765.383796.333070.9822TPLGD79.305979.459447.245387.008569.584396.741676.5575ORB78.047777.954042.680983.906967.359396.874974.4706MRA74.554174.318649.452087.847058.113894.970873.2094QTORB76.523376.394547.181288.902863.397695.317374.6195From Figure8 and Table 3, for the image groups Bikes, Boat, Trees, and Ubc, SIFT is the best performer. The best performance on the Graf and Leuven image groups, respectively, is the MRA algorithm and the BRISK algorithm. The QTORB algorithm proposed in this paper has acceptable performance in terms of precision. The SIFT algorithm is the highest. The QTORB outperforms the SURF, LATCH, LDB, ORB, and MRA.


#### 4.4.3. Matching RMSE Using Oxford Dataset Images

Figure9 shows the RMSE indicators of the SIFT, SURF, BRISK, LATCH, LDB, TPLGD, ORB, MRA, and QTORB in different images change scenarios. In Table 4, among all the tested algorithms, the RMSE of the SIFT algorithm is the smallest, which means that the positioning is the most accurate. The performances of other algorithms are at the same level. The QTORB is inferior to the SIFT, SURF, and BRISK algorithms, while still better than LATCH, LDB, TPLGD, ORB, and MRA.Figure 9
Mean RMSE of the SIFT, SURF, BRISK, LATCH, LDB, TPLGD, ORB, MRA, and QTORB algorithms based on Oxford dataset.Table 4
Mean RMSE using Oxford dataset images (pixel).
MethodBikesBoatGrafLeuvenTreesUbcMeanSIFT0.89910.80270.84510.51641.28370.58390.8218SURF1.03741.23991.28510.71721.28890.67971.0414BRISK1.19181.12041.25780.73831.32970.73631.0624LATCH1.24781.24611.23501.10411.49920.29991.1054LDB1.28721.21301.35151.06431.37770.33211.1043TPLGD1.26371.18171.27281.08981.49650.31221.1028ORB1.30301.09761.14181.10091.43860.31261.0658MRA1.28521.10611.17661.00501.50090.32951.0672QTORB1.26231.24871.09490.99531.43160.35151.0641


#### 4.4.4. Operation Time Using Oxford Dataset Images

Figure10 visually shows the difference in operation time of SIFT, SURF, BRISK, LATCH, LDB, TPLGD, ORB, MRA, and QTORB. Table 5 records the mean operation time and shows that the running time of SIFT algorithm is the longest while ORB is the least time-consuming. The QTORB algorithm takes slightly longer than ORB, but still much shorter than MRA. At the same time, it also can be found that the real-time performance of the image matching algorithm based on the floating-point feature description is far lower than that based on the binary feature description.Figure 10
Mean operation time of the SIFT, SURF, BRISK, LATCH, LDB, TPLGD, ORB, MRA, and QTORB algorithms based on Oxford dataset.Table 5
Mean operation time using Oxford dataset images (s).
MethodBikesBoatGrafLeuvenTreesUbcMeanSIFT1.26262.93511.33661.07575.44732.17282.3717SURF1.44412.01381.68161.28083.18291.85141.9091BRISK0.39211.87340.68030.55252.28661.84381.2715LATCH0.33770.50610.37990.32950.73370.45050.4562LDB0.54740.71770.59170.52750.93400.66660.6642TPLGD0.28380.47180.35180.24720.73560.50930.4333ORB0.17370.27730.19320.16440.40790.23390.2417MRA0.39680.58230.37690.33000.87780.46230.5044QTORB0.28040.34370.21670.26090.48070.34000.3204


### 4.5. Experiment on SAR Dataset

To check the performance on remote sensing SAR images of the tested algorithms, scaling, rotation, and mixing (scaling and rotation occur simultaneously) were evaluated for three types of image transformations. The scale ratio in the experiment is 0.9 and the angle of rotation is 10 degrees. In the experiment, a total of one hundred SAR images were used.


#### 4.5.1. Feature Points Evenness and Matching Effect on SAR Dataset

This section tests the distribution uniformity of the feature points extracted by the SIFT, SURF, BRISK, LATCH, LDB, TPLGD, ORB, MRA, and QTORB, applied to SAR images. In Figure11, a SAR image containing a township scene is selected to display the distribution of feature points. It is intuitive to find that the feature points extracted by the SIFT, SURF, BRISK, LATCH, LDB, TPLGD, and ORB algorithms have obvious concentration in the town area at the bottom-left of the image. For other uniform regions, although SIFT, SURF, and BRISK can also extract feature points, most of the feature points are also concentrated on the edge. The LATCH, LDB, TPLGD, and ORB basically cannot extract feature points in other uniform regions. Most of the feature points are concentrated in the towns and roads. Compared with the images provided by the Oxford dataset, the differences between the scenes contained in the SAR images are more significant, which makes the concentrated distribution of feature points more obvious. The MRA and QTORB algorithms greatly alleviate the concentrated distribution of feature points.Figure 11
The distribution of points extracted by the tested algorithms using the selected SAR image containing a township scene. (a) SIFT. (b) SURF. (c) BRISK. (d) LATCH. (e) LDB. (f) TPLGD. (g) ORB. (h) MRA. (i) QTORB.
(a)(b)(c)(d)(e)(f)(g)(h)(i)Figure12 shows the image matching effect of SIFT, SURF, BRISK, LATCH, LDB, TPLGD, ORB, MRA, and QTORB using SAR images. The feature point pair connected by the green line is a correct matching pair, and the point pair connected by the red line is a wrong matching pair.Figure 12
Matching result of the (a) SIFT, (b) SURF, (c) BRISK, (d) LATCH, (e) LDB, (f) TPLGD, (g) ORB, (h) MRA, and (i) QTORB algorithms using the selected SAR image containing a township scene.
(a)(b)(c)(d)(e)(f)(g)(h)(i)From Figure12, the distribution of the correct matched feature points by the SIFT, SURF, BRISK, LATCH, LDB, TPLGD, ORB, MRA, and QTORB is still consistent with that of the feature points extracted in Figure 11. The performance of the MRA and QTORB algorithms is confirmed again. In SAR images experiment, three image transformation scenarios are set according to the scale ratio and rotation angle including scaling, rotation, and mixing (scaling and rotation exist simultaneously). From Table 6, the QTORB algorithm can achieve the maximum uniform distribution of feature points in all these three image transformation scenarios.Table 6
Mean evenness of feature points on SAR dataset.
MethodScale = 0.9Rotation = 10Scale = 0.9Rotation = 10MeanSIFT259.9164257.2572255.6382257.6039SURF242.1884246.6370236.4556241.7603BRISK263.5500267.2996263.2762264.7086LATCH182.0922177.5690185.4978181.7197LDB180.8172179.3682173.2460177.8105TPLGD184.1988170.0662167.3348173.8666ORB179.6610171.1720185.9520178.9283MRA162.9324157.5000146.1532155.5285QTORB159.4978153.2828141.2876151.3561


#### 4.5.2. Matching Precision on SAR Dataset Images

In experiment, one hundred SAR images were used. Five SAR images were randomly selected to visually display the test results. Figure13 shows the matching precision of SIFT, SURF, BRISK, LATCH, LDB, TPLGD, ORB, MRA, and QTORB using the selected five SAR images. Figure 13 intuitively shows that the QTORB algorithm can achieve image matching well in the given three image scenarios. In most cases, it can achieve better matching precision than ORB and MRA.Figure 13
Matching precision based on the selected SAR images. Three transformations are used: (a) scaling (scale = 0.9), (b) rotation (rotation = 10), and (c) mixing (scale = 0.9, rotation = 10).
(a)(b)(c)Table7 is calculated from all the one hundred tested SAR images. Under the three image transformation conditions specifically set, the highest matching precision is the SIFT. According to the mean data in Table 7, the QTORB algorithm is lower than the SIFT, BRISK, and TPLGD algorithms and higher than SURF, LATCH, LDB, ORB, and MRA.Table 7
Mean precision based on SAR dataset (%).
MethodScale = 0.9Rotation = 10Scale = 0.9Rotation = 10MeanSIFT99.727999.761699.726899.7388SURF96.193296.397794.397895.6629BRISK97.311697.939798.329897.8604LATCH96.739896.272896.703996.5721LDB97.239595.980395.846596.3554TPLGD97.965596.566696.513397.0151ORB94.757595.736695.587695.3606MRA95.496495.851897.232696.1936QTORB95.832195.927298.113696.6243


#### 4.5.3. Matching RMSE on SAR Dataset Images

Figure14 and Table 8 show the RMSE indicators of the SIFT, SURF, BRISK, LATCH, LDB, TPLGD, ORB, MRA, and QTORB tested on SAR images. Under the three image transformation conditions specifically set, the RMSE of the SIFT is the lowest, which means that the positioning accuracy of the SIFT is the most accurate. According to the mean data of the three cases in Table 8, the RMSE of SIFT and SURF is significantly lower than that of BRISK, LATCH, LDB, TPLGD, ORB, MRA, and QTORB. The RMSE of the QTORB algorithm is lower than that of the ORB algorithm.Figure 14
Mean RMSE of SIFT, SURF, BRISK, LATCH, LDB, TPLGD, ORB, MRA, and QTORB using the SAR dataset images.Table 8
Mean RMSE based on SAR dataset (pixel).
MethodScale = 0.9Rotation = 10Scale = 0.9Rotation = 10MeanSIFT0.10900.11230.12420.1151SURF0.50880.47280.62190.5345BRISK0.69280.49940.60460.5989LATCH0.90150.87520.88370.8868LDB0.80250.81480.81540.8109TPLGD0.77400.91510.88070.8566ORB0.87760.89650.92010.8980MRA0.87800.90980.85190.8799QTORB0.87480.92140.88240.8929


#### 4.5.4. Operation Time on SAR Dataset Images

Figure15 and Table 9 record the operation time of SIFT, SURF, BRISK, LATCH, LDB, TPLGD, ORB, MRA, and QTORB. It can be found that among all the algorithms tested, the SIFT algorithm takes the longest time and ORB algorithm takes the least time. The LATCH, LDB, TPLGD, MRA, and QTORB algorithms proposed based on the improvement of the ORB algorithm cost more time than ORB algorithm to different degrees. Compared with the MRA algorithm, the time-consuming increase of the QTORB algorithm is smaller. The operation time of QTORB is only slightly longer than that of ORB and still much less than that of MRA. It can also be concluded through experiments that the image matching algorithm based on binary feature description is much faster than the image matching algorithm based on floating-point feature description.Figure 15
Mean operation time of the SIFT, SURF, BRISK, LATCH, LDB, TPLGD, ORB, MRA, and QTORB based on the SAR dataset.Table 9
Mean operation time based on SAR dataset (s).
MethodScale = 0.9Rotation = 10Scale = 0.9Rotation = 10MeanSIFT1.16251.34851.27001.2603SURF0.66780.92320.78420.7917BRISK0.53910.58900.56110.5631LATCH0.43370.43740.43550.4355LDB0.45920.46090.43460.4516TPLGD0.33190.31430.29590.3140ORB0.17500.20470.19350.1911MRA0.32150.36880.34160.3439QTORB0.22570.25390.23420.2379


## 4.1. Evaluation Metrics

Four measurement criteria are used in experiment. Evenness is the first criterion. The Evenness [46] calculation setups are shown in Figure 3. Its basic principle is to use the mathematical characteristics between the number of feature points in the same area of various shapes of the image to measure the distribution of feature points. First, the image is divided into ten blocks from five directions: vertical, horizontal, 45 degrees, 135 degrees, internal, and external. By the division method in Figure 3, ten different regions can be obtained.Figure 3
Divide the image in multiple directions.Count the number of feature points in each area as a dataset. Calculate the varianceV of this dataset. The calculation of Evenness is shown in equation (8). The smaller the Evenness value, the better the uniform distribution of feature points.(8)Evenness=101×logV.The second criterion is Precision [44]. The calculation formula is equation (9). Numcorrect matches refers to the number of correct matches while Numall matches refers to the number of all matched points.(9)Precision=Numcorrect matchesNumall matches×100%.The third criterion is the Root-Mean-Square Error (RMSE). RMSE represents the positional distance between the feature point obtained by the matching algorithm and the real feature point. Assume that feature pointApxap,yap and Bqxbq,ybq is a matched point pair obtained by the image matching algorithm. Feature point Apxap,yap is located in image A, while Bqxbq,ybq is located in image B. The true position of point Apxap,yap on image B is Bp∧xbp∧,ybp∧. Then, the true position distance between point Bqxbq,ybq and point Bp∧xbp∧,ybp∧ is as follows:(10)RMSE=xbp−xbp∧2+ybq−ybp∧2.Image operation time is the fourth criterion. In this paper, algorithm operation time includes feature point extraction time, description time, and matching time. Input process time and indicators calculation time are not considered.


## 4.2. Oxford Optical Image Dataset

Provided by Mikolajczyk and Schmid [44], the Oxford dataset is a publicly measurable dataset. There are eight image classes in the dataset including image blur (Trees group and Bikes group), light (Leuven group), JPG compression (Ubc group), rotation scale conversion (Bark group, Boat group), and visual improvements (Wall group and Graf group). As shown in Figure 4, we will pick six image classes from the Oxford dataset.Figure 4
Optical images from Oxford dataset. (a) Bikes (blur). (b) Boat (rotation and scale). (c) Graf (viewpoint). (d) Leuven (illumination). (e) Trees (blur). (f) Ubc (JPG compression).
(a)(b)(c)(d)(e)(f)There are six images prepared for experimentation in each image group. As an input image pair, the first image can be paired with the other five images. The homography matrix between two images in each image pair is provided. The ground truth of the two images can be calculated based on this homography matrix.


## 4.3. SAR Dataset

For tests, a SAR image dataset acquired by the UAV-borne SAR platform is used. These SAR images include several typical scenes of objects on the ground, including cities, lakes, fields, roads, hills, and rivers, in Bedfordshire, southeast England. For a SAR image, rotation and zoom operations are used to create an image to be matched. The SAR image and its image to be matched are matched as an input image pair. As for the ground truth for each image pair, the homography matrix between two images in each image pair can be calculated according to the specific angle degree and scale ratio. In Figure5, some SAR images are shown.Figure 5
Eight SAR images. (a) Hills. (b) Lakes. (c) Town. (d) Road. (e) River. (f) Field. (g) Field. (h) Field.
(a)(b)(c)(d)(e)(f)(g)(h)


## 4.4. Experiment Result on Oxford Dataset

Six image groups from the Oxford dataset are selected for experiment including Bikes (blur), Boat (rotation and scale), Graf (viewpoint), Leuven (illumination), Trees (blur), and Ubc (JPG compression). To ensure the generality of the test results, each image pair is tested thirty times. The average value of each data is used as the final experimental data.


### 4.4.1. Feature Points Evenness and Matching Effect on Oxford Dataset

The SIFT, SURF, BRISK, LATCH, LDB, TPLGD, ORB, MRA, and QTORB algorithms performed feature point extraction and image matching on the images selected from the Oxford dataset. From the distribution of feature points before and after matching, the performance difference of uniform distribution of feature points of the nine algorithms tested can be intuitively found.Figure6 shows us the feature point extraction results of the tested algorithms using the Bikes images group from the Oxford dataset. It can be intuitively found that the feature points extracted by the LATCH, LDB, TPLGD, and ORB algorithms are concentrated in image areas including more prominent edges while almost no feature points are extracted in the homogeneous area. Although the SIFT, SURF, and BRISK algorithms can extract feature points in the homogeneous area of the image, the number of feature points in the homogeneous area is not much. Most of the feature points are still concentrated in the edge area of the image, resulting in a large amount of overlap. As for the MRA and QTORB, the feature points are obviously more uniform than the other tested algorithms. The uniform difference between the MRA and QTORB algorithm is not obvious, but the QTORB algorithm can extract some feature points that the MRA cannot extract in some homogeneous area.Figure 6
The distribution of feature points extracted by the (a) SIFT, (b) SURF, (c) BRISK, (d) LATCH, (e) LDB, (f) TPLGD, (g) ORB, (h) MRA, and (i) QTORB algorithms on the Bikes group image from Oxford dataset
(a)(b)(c)(d)(e)(f)(g)(h)(i)Based on the extracted feature points above, we continue to complete the image matching process. Figure7 shows us the image matching effect of the tested algorithms using the Bikes images group from the Oxford dataset. The feature point pair connected by the green line is a correct matching pair, and the feature point pair connected by the red line is a wrong matching pair. Consistent with the distribution of feature points shown in Figure 6, Figure 7 shows that the correct matching feature points implemented by SIFT, SURF, BRISK, LATCH, LDB, TPLGD, and ORB algorithms also have a relatively serious centralized distribution. The matching results achieved by the MRA and QTORB algorithms are very uniform, and there are correct matches in both the homogeneous area and the edge area. The matching results achieved by the MRA and QTORB are relatively similar, making it difficult to distinguish the difference intuitively. To analyze the performance differences of the tested algorithms in more detail, the feature distribution uniformity of the selected six images groups of Oxford dataset is counted, as shown in Table 2.Figure 7
Image matching effect of the (a) SIFT, (b) SURF, (c) BRISK, (d) LATCH, (e) LDB, (f) TPLGD, (g) ORB, (h) MRA, and (i) QTORB algorithms on the Bikes group image from Oxford dataset.
(a)(b)(c)(d)(e)(f)(g)(h)(i)Table 2
Mean evenness of feature points using Oxford dataset images.
MethodBikesBoatGrafLeuvenTreesUbcSIFT192.5424203.3804159.8065234.0446207.2986253.872SURF254.0986201.3406159.55948273.6258231.2498269.886BRISK196.2948234.982170.94194251.4328255.885289.5048LATCH173.5252137.4798116.26842174.9244140.3472181.9418LDB179.6288143.53562120.0294177.5232144.188185.8752TPLGD174.0654141.31146116.88112175.7902151.1302180.313ORB171.8528148.43458120.23718175.1806146.18394180.0906MRA141.277111.0517484.45352158.6698112.15136168.1602QTORB140.647692.4308484.3705168.1638113.32876166.0178From Table2, under the six scene transformations provided by Oxford dataset, both MRA and QTORB can achieve more uniform feature point extraction than the other tested algorithms and provide a more reasonable distribution of feature points for subsequent image processing. Meanwhile, compared with MRA, the QTORB can better adapt to more scenarios.


### 4.4.2. Matching Precision Using Oxford Dataset Images

This paper has carried out image matching tests on the six images groups provided by Oxford dataset. In each group, five images pairs (from 1–2 to 1–6, the more backward the image pairs, the greater the change between images) need to be matched. The precision of each matching process is shown in Figure8. The symbol “Mean” represents the average precision of the five matching processes in each group. The average precision data is counted in Table 3.Figure 8
Matching precision based on the Oxford dataset. (a) Bikes (blur). (b) Boat (rotation and scale). (c) Graf (viewpoint). (d) Leuven (illumination). (e) Trees (blur). (f) Ubc (JPG compression).
(a)(b)(c)(d)(e)(f)Table 3
Mean precision using Oxford dataset images (%).
MethodBikesBoatGrafLeuvenTreesUbcMeanSIFT81.799583.628844.512190.141473.138997.649778.4784SURF76.279558.699332.377881.073757.160681.430064.5035BRISK80.606974.971845.069093.724761.264594.118874.9593LATCH79.083666.133947.575384.302168.510196.426873.6720LDB75.150860.604543.558684.862765.383796.333070.9822TPLGD79.305979.459447.245387.008569.584396.741676.5575ORB78.047777.954042.680983.906967.359396.874974.4706MRA74.554174.318649.452087.847058.113894.970873.2094QTORB76.523376.394547.181288.902863.397695.317374.6195From Figure8 and Table 3, for the image groups Bikes, Boat, Trees, and Ubc, SIFT is the best performer. The best performance on the Graf and Leuven image groups, respectively, is the MRA algorithm and the BRISK algorithm. The QTORB algorithm proposed in this paper has acceptable performance in terms of precision. The SIFT algorithm is the highest. The QTORB outperforms the SURF, LATCH, LDB, ORB, and MRA.


### 4.4.3. Matching RMSE Using Oxford Dataset Images

Figure9 shows the RMSE indicators of the SIFT, SURF, BRISK, LATCH, LDB, TPLGD, ORB, MRA, and QTORB in different images change scenarios. In Table 4, among all the tested algorithms, the RMSE of the SIFT algorithm is the smallest, which means that the positioning is the most accurate. The performances of other algorithms are at the same level. The QTORB is inferior to the SIFT, SURF, and BRISK algorithms, while still better than LATCH, LDB, TPLGD, ORB, and MRA.Figure 9
Mean RMSE of the SIFT, SURF, BRISK, LATCH, LDB, TPLGD, ORB, MRA, and QTORB algorithms based on Oxford dataset.Table 4
Mean RMSE using Oxford dataset images (pixel).
MethodBikesBoatGrafLeuvenTreesUbcMeanSIFT0.89910.80270.84510.51641.28370.58390.8218SURF1.03741.23991.28510.71721.28890.67971.0414BRISK1.19181.12041.25780.73831.32970.73631.0624LATCH1.24781.24611.23501.10411.49920.29991.1054LDB1.28721.21301.35151.06431.37770.33211.1043TPLGD1.26371.18171.27281.08981.49650.31221.1028ORB1.30301.09761.14181.10091.43860.31261.0658MRA1.28521.10611.17661.00501.50090.32951.0672QTORB1.26231.24871.09490.99531.43160.35151.0641


### 4.4.4. Operation Time Using Oxford Dataset Images

Figure10 visually shows the difference in operation time of SIFT, SURF, BRISK, LATCH, LDB, TPLGD, ORB, MRA, and QTORB. Table 5 records the mean operation time and shows that the running time of SIFT algorithm is the longest while ORB is the least time-consuming. The QTORB algorithm takes slightly longer than ORB, but still much shorter than MRA. At the same time, it also can be found that the real-time performance of the image matching algorithm based on the floating-point feature description is far lower than that based on the binary feature description.Figure 10
Mean operation time of the SIFT, SURF, BRISK, LATCH, LDB, TPLGD, ORB, MRA, and QTORB algorithms based on Oxford dataset.Table 5
Mean operation time using Oxford dataset images (s).
MethodBikesBoatGrafLeuvenTreesUbcMeanSIFT1.26262.93511.33661.07575.44732.17282.3717SURF1.44412.01381.68161.28083.18291.85141.9091BRISK0.39211.87340.68030.55252.28661.84381.2715LATCH0.33770.50610.37990.32950.73370.45050.4562LDB0.54740.71770.59170.52750.93400.66660.6642TPLGD0.28380.47180.35180.24720.73560.50930.4333ORB0.17370.27730.19320.16440.40790.23390.2417MRA0.39680.58230.37690.33000.87780.46230.5044QTORB0.28040.34370.21670.26090.48070.34000.3204


## 4.4.1. Feature Points Evenness and Matching Effect on Oxford Dataset

The SIFT, SURF, BRISK, LATCH, LDB, TPLGD, ORB, MRA, and QTORB algorithms performed feature point extraction and image matching on the images selected from the Oxford dataset. From the distribution of feature points before and after matching, the performance difference of uniform distribution of feature points of the nine algorithms tested can be intuitively found.Figure6 shows us the feature point extraction results of the tested algorithms using the Bikes images group from the Oxford dataset. It can be intuitively found that the feature points extracted by the LATCH, LDB, TPLGD, and ORB algorithms are concentrated in image areas including more prominent edges while almost no feature points are extracted in the homogeneous area. Although the SIFT, SURF, and BRISK algorithms can extract feature points in the homogeneous area of the image, the number of feature points in the homogeneous area is not much. Most of the feature points are still concentrated in the edge area of the image, resulting in a large amount of overlap. As for the MRA and QTORB, the feature points are obviously more uniform than the other tested algorithms. The uniform difference between the MRA and QTORB algorithm is not obvious, but the QTORB algorithm can extract some feature points that the MRA cannot extract in some homogeneous area.Figure 6
The distribution of feature points extracted by the (a) SIFT, (b) SURF, (c) BRISK, (d) LATCH, (e) LDB, (f) TPLGD, (g) ORB, (h) MRA, and (i) QTORB algorithms on the Bikes group image from Oxford dataset
(a)(b)(c)(d)(e)(f)(g)(h)(i)Based on the extracted feature points above, we continue to complete the image matching process. Figure7 shows us the image matching effect of the tested algorithms using the Bikes images group from the Oxford dataset. The feature point pair connected by the green line is a correct matching pair, and the feature point pair connected by the red line is a wrong matching pair. Consistent with the distribution of feature points shown in Figure 6, Figure 7 shows that the correct matching feature points implemented by SIFT, SURF, BRISK, LATCH, LDB, TPLGD, and ORB algorithms also have a relatively serious centralized distribution. The matching results achieved by the MRA and QTORB algorithms are very uniform, and there are correct matches in both the homogeneous area and the edge area. The matching results achieved by the MRA and QTORB are relatively similar, making it difficult to distinguish the difference intuitively. To analyze the performance differences of the tested algorithms in more detail, the feature distribution uniformity of the selected six images groups of Oxford dataset is counted, as shown in Table 2.Figure 7
Image matching effect of the (a) SIFT, (b) SURF, (c) BRISK, (d) LATCH, (e) LDB, (f) TPLGD, (g) ORB, (h) MRA, and (i) QTORB algorithms on the Bikes group image from Oxford dataset.
(a)(b)(c)(d)(e)(f)(g)(h)(i)Table 2
Mean evenness of feature points using Oxford dataset images.
MethodBikesBoatGrafLeuvenTreesUbcSIFT192.5424203.3804159.8065234.0446207.2986253.872SURF254.0986201.3406159.55948273.6258231.2498269.886BRISK196.2948234.982170.94194251.4328255.885289.5048LATCH173.5252137.4798116.26842174.9244140.3472181.9418LDB179.6288143.53562120.0294177.5232144.188185.8752TPLGD174.0654141.31146116.88112175.7902151.1302180.313ORB171.8528148.43458120.23718175.1806146.18394180.0906MRA141.277111.0517484.45352158.6698112.15136168.1602QTORB140.647692.4308484.3705168.1638113.32876166.0178From Table2, under the six scene transformations provided by Oxford dataset, both MRA and QTORB can achieve more uniform feature point extraction than the other tested algorithms and provide a more reasonable distribution of feature points for subsequent image processing. Meanwhile, compared with MRA, the QTORB can better adapt to more scenarios.


## 4.4.2. Matching Precision Using Oxford Dataset Images

This paper has carried out image matching tests on the six images groups provided by Oxford dataset. In each group, five images pairs (from 1–2 to 1–6, the more backward the image pairs, the greater the change between images) need to be matched. The precision of each matching process is shown in Figure8. The symbol “Mean” represents the average precision of the five matching processes in each group. The average precision data is counted in Table 3.Figure 8
Matching precision based on the Oxford dataset. (a) Bikes (blur). (b) Boat (rotation and scale). (c) Graf (viewpoint). (d) Leuven (illumination). (e) Trees (blur). (f) Ubc (JPG compression).
(a)(b)(c)(d)(e)(f)Table 3
Mean precision using Oxford dataset images (%).
MethodBikesBoatGrafLeuvenTreesUbcMeanSIFT81.799583.628844.512190.141473.138997.649778.4784SURF76.279558.699332.377881.073757.160681.430064.5035BRISK80.606974.971845.069093.724761.264594.118874.9593LATCH79.083666.133947.575384.302168.510196.426873.6720LDB75.150860.604543.558684.862765.383796.333070.9822TPLGD79.305979.459447.245387.008569.584396.741676.5575ORB78.047777.954042.680983.906967.359396.874974.4706MRA74.554174.318649.452087.847058.113894.970873.2094QTORB76.523376.394547.181288.902863.397695.317374.6195From Figure8 and Table 3, for the image groups Bikes, Boat, Trees, and Ubc, SIFT is the best performer. The best performance on the Graf and Leuven image groups, respectively, is the MRA algorithm and the BRISK algorithm. The QTORB algorithm proposed in this paper has acceptable performance in terms of precision. The SIFT algorithm is the highest. The QTORB outperforms the SURF, LATCH, LDB, ORB, and MRA.


## 4.4.3. Matching RMSE Using Oxford Dataset Images

Figure9 shows the RMSE indicators of the SIFT, SURF, BRISK, LATCH, LDB, TPLGD, ORB, MRA, and QTORB in different images change scenarios. In Table 4, among all the tested algorithms, the RMSE of the SIFT algorithm is the smallest, which means that the positioning is the most accurate. The performances of other algorithms are at the same level. The QTORB is inferior to the SIFT, SURF, and BRISK algorithms, while still better than LATCH, LDB, TPLGD, ORB, and MRA.Figure 9
Mean RMSE of the SIFT, SURF, BRISK, LATCH, LDB, TPLGD, ORB, MRA, and QTORB algorithms based on Oxford dataset.Table 4
Mean RMSE using Oxford dataset images (pixel).
MethodBikesBoatGrafLeuvenTreesUbcMeanSIFT0.89910.80270.84510.51641.28370.58390.8218SURF1.03741.23991.28510.71721.28890.67971.0414BRISK1.19181.12041.25780.73831.32970.73631.0624LATCH1.24781.24611.23501.10411.49920.29991.1054LDB1.28721.21301.35151.06431.37770.33211.1043TPLGD1.26371.18171.27281.08981.49650.31221.1028ORB1.30301.09761.14181.10091.43860.31261.0658MRA1.28521.10611.17661.00501.50090.32951.0672QTORB1.26231.24871.09490.99531.43160.35151.0641


## 4.4.4. Operation Time Using Oxford Dataset Images

Figure10 visually shows the difference in operation time of SIFT, SURF, BRISK, LATCH, LDB, TPLGD, ORB, MRA, and QTORB. Table 5 records the mean operation time and shows that the running time of SIFT algorithm is the longest while ORB is the least time-consuming. The QTORB algorithm takes slightly longer than ORB, but still much shorter than MRA. At the same time, it also can be found that the real-time performance of the image matching algorithm based on the floating-point feature description is far lower than that based on the binary feature description.Figure 10
Mean operation time of the SIFT, SURF, BRISK, LATCH, LDB, TPLGD, ORB, MRA, and QTORB algorithms based on Oxford dataset.Table 5
Mean operation time using Oxford dataset images (s).
MethodBikesBoatGrafLeuvenTreesUbcMeanSIFT1.26262.93511.33661.07575.44732.17282.3717SURF1.44412.01381.68161.28083.18291.85141.9091BRISK0.39211.87340.68030.55252.28661.84381.2715LATCH0.33770.50610.37990.32950.73370.45050.4562LDB0.54740.71770.59170.52750.93400.66660.6642TPLGD0.28380.47180.35180.24720.73560.50930.4333ORB0.17370.27730.19320.16440.40790.23390.2417MRA0.39680.58230.37690.33000.87780.46230.5044QTORB0.28040.34370.21670.26090.48070.34000.3204


## 4.5. Experiment on SAR Dataset

To check the performance on remote sensing SAR images of the tested algorithms, scaling, rotation, and mixing (scaling and rotation occur simultaneously) were evaluated for three types of image transformations. The scale ratio in the experiment is 0.9 and the angle of rotation is 10 degrees. In the experiment, a total of one hundred SAR images were used.


### 4.5.1. Feature Points Evenness and Matching Effect on SAR Dataset

This section tests the distribution uniformity of the feature points extracted by the SIFT, SURF, BRISK, LATCH, LDB, TPLGD, ORB, MRA, and QTORB, applied to SAR images. In Figure11, a SAR image containing a township scene is selected to display the distribution of feature points. It is intuitive to find that the feature points extracted by the SIFT, SURF, BRISK, LATCH, LDB, TPLGD, and ORB algorithms have obvious concentration in the town area at the bottom-left of the image. For other uniform regions, although SIFT, SURF, and BRISK can also extract feature points, most of the feature points are also concentrated on the edge. The LATCH, LDB, TPLGD, and ORB basically cannot extract feature points in other uniform regions. Most of the feature points are concentrated in the towns and roads. Compared with the images provided by the Oxford dataset, the differences between the scenes contained in the SAR images are more significant, which makes the concentrated distribution of feature points more obvious. The MRA and QTORB algorithms greatly alleviate the concentrated distribution of feature points.Figure 11
The distribution of points extracted by the tested algorithms using the selected SAR image containing a township scene. (a) SIFT. (b) SURF. (c) BRISK. (d) LATCH. (e) LDB. (f) TPLGD. (g) ORB. (h) MRA. (i) QTORB.
(a)(b)(c)(d)(e)(f)(g)(h)(i)Figure12 shows the image matching effect of SIFT, SURF, BRISK, LATCH, LDB, TPLGD, ORB, MRA, and QTORB using SAR images. The feature point pair connected by the green line is a correct matching pair, and the point pair connected by the red line is a wrong matching pair.Figure 12
Matching result of the (a) SIFT, (b) SURF, (c) BRISK, (d) LATCH, (e) LDB, (f) TPLGD, (g) ORB, (h) MRA, and (i) QTORB algorithms using the selected SAR image containing a township scene.
(a)(b)(c)(d)(e)(f)(g)(h)(i)From Figure12, the distribution of the correct matched feature points by the SIFT, SURF, BRISK, LATCH, LDB, TPLGD, ORB, MRA, and QTORB is still consistent with that of the feature points extracted in Figure 11. The performance of the MRA and QTORB algorithms is confirmed again. In SAR images experiment, three image transformation scenarios are set according to the scale ratio and rotation angle including scaling, rotation, and mixing (scaling and rotation exist simultaneously). From Table 6, the QTORB algorithm can achieve the maximum uniform distribution of feature points in all these three image transformation scenarios.Table 6
Mean evenness of feature points on SAR dataset.
MethodScale = 0.9Rotation = 10Scale = 0.9Rotation = 10MeanSIFT259.9164257.2572255.6382257.6039SURF242.1884246.6370236.4556241.7603BRISK263.5500267.2996263.2762264.7086LATCH182.0922177.5690185.4978181.7197LDB180.8172179.3682173.2460177.8105TPLGD184.1988170.0662167.3348173.8666ORB179.6610171.1720185.9520178.9283MRA162.9324157.5000146.1532155.5285QTORB159.4978153.2828141.2876151.3561


### 4.5.2. Matching Precision on SAR Dataset Images

In experiment, one hundred SAR images were used. Five SAR images were randomly selected to visually display the test results. Figure13 shows the matching precision of SIFT, SURF, BRISK, LATCH, LDB, TPLGD, ORB, MRA, and QTORB using the selected five SAR images. Figure 13 intuitively shows that the QTORB algorithm can achieve image matching well in the given three image scenarios. In most cases, it can achieve better matching precision than ORB and MRA.Figure 13
Matching precision based on the selected SAR images. Three transformations are used: (a) scaling (scale = 0.9), (b) rotation (rotation = 10), and (c) mixing (scale = 0.9, rotation = 10).
(a)(b)(c)Table7 is calculated from all the one hundred tested SAR images. Under the three image transformation conditions specifically set, the highest matching precision is the SIFT. According to the mean data in Table 7, the QTORB algorithm is lower than the SIFT, BRISK, and TPLGD algorithms and higher than SURF, LATCH, LDB, ORB, and MRA.Table 7
Mean precision based on SAR dataset (%).
MethodScale = 0.9Rotation = 10Scale = 0.9Rotation = 10MeanSIFT99.727999.761699.726899.7388SURF96.193296.397794.397895.6629BRISK97.311697.939798.329897.8604LATCH96.739896.272896.703996.5721LDB97.239595.980395.846596.3554TPLGD97.965596.566696.513397.0151ORB94.757595.736695.587695.3606MRA95.496495.851897.232696.1936QTORB95.832195.927298.113696.6243


### 4.5.3. Matching RMSE on SAR Dataset Images

Figure14 and Table 8 show the RMSE indicators of the SIFT, SURF, BRISK, LATCH, LDB, TPLGD, ORB, MRA, and QTORB tested on SAR images. Under the three image transformation conditions specifically set, the RMSE of the SIFT is the lowest, which means that the positioning accuracy of the SIFT is the most accurate. According to the mean data of the three cases in Table 8, the RMSE of SIFT and SURF is significantly lower than that of BRISK, LATCH, LDB, TPLGD, ORB, MRA, and QTORB. The RMSE of the QTORB algorithm is lower than that of the ORB algorithm.Figure 14
Mean RMSE of SIFT, SURF, BRISK, LATCH, LDB, TPLGD, ORB, MRA, and QTORB using the SAR dataset images.Table 8
Mean RMSE based on SAR dataset (pixel).
MethodScale = 0.9Rotation = 10Scale = 0.9Rotation = 10MeanSIFT0.10900.11230.12420.1151SURF0.50880.47280.62190.5345BRISK0.69280.49940.60460.5989LATCH0.90150.87520.88370.8868LDB0.80250.81480.81540.8109TPLGD0.77400.91510.88070.8566ORB0.87760.89650.92010.8980MRA0.87800.90980.85190.8799QTORB0.87480.92140.88240.8929


### 4.5.4. Operation Time on SAR Dataset Images

Figure15 and Table 9 record the operation time of SIFT, SURF, BRISK, LATCH, LDB, TPLGD, ORB, MRA, and QTORB. It can be found that among all the algorithms tested, the SIFT algorithm takes the longest time and ORB algorithm takes the least time. The LATCH, LDB, TPLGD, MRA, and QTORB algorithms proposed based on the improvement of the ORB algorithm cost more time than ORB algorithm to different degrees. Compared with the MRA algorithm, the time-consuming increase of the QTORB algorithm is smaller. The operation time of QTORB is only slightly longer than that of ORB and still much less than that of MRA. It can also be concluded through experiments that the image matching algorithm based on binary feature description is much faster than the image matching algorithm based on floating-point feature description.Figure 15
Mean operation time of the SIFT, SURF, BRISK, LATCH, LDB, TPLGD, ORB, MRA, and QTORB based on the SAR dataset.Table 9
Mean operation time based on SAR dataset (s).
MethodScale = 0.9Rotation = 10Scale = 0.9Rotation = 10MeanSIFT1.16251.34851.27001.2603SURF0.66780.92320.78420.7917BRISK0.53910.58900.56110.5631LATCH0.43370.43740.43550.4355LDB0.45920.46090.43460.4516TPLGD0.33190.31430.29590.3140ORB0.17500.20470.19350.1911MRA0.32150.36880.34160.3439QTORB0.22570.25390.23420.2379


## 4.5.1. Feature Points Evenness and Matching Effect on SAR Dataset

This section tests the distribution uniformity of the feature points extracted by the SIFT, SURF, BRISK, LATCH, LDB, TPLGD, ORB, MRA, and QTORB, applied to SAR images. In Figure11, a SAR image containing a township scene is selected to display the distribution of feature points. It is intuitive to find that the feature points extracted by the SIFT, SURF, BRISK, LATCH, LDB, TPLGD, and ORB algorithms have obvious concentration in the town area at the bottom-left of the image. For other uniform regions, although SIFT, SURF, and BRISK can also extract feature points, most of the feature points are also concentrated on the edge. The LATCH, LDB, TPLGD, and ORB basically cannot extract feature points in other uniform regions. Most of the feature points are concentrated in the towns and roads. Compared with the images provided by the Oxford dataset, the differences between the scenes contained in the SAR images are more significant, which makes the concentrated distribution of feature points more obvious. The MRA and QTORB algorithms greatly alleviate the concentrated distribution of feature points.Figure 11
The distribution of points extracted by the tested algorithms using the selected SAR image containing a township scene. (a) SIFT. (b) SURF. (c) BRISK. (d) LATCH. (e) LDB. (f) TPLGD. (g) ORB. (h) MRA. (i) QTORB.
(a)(b)(c)(d)(e)(f)(g)(h)(i)Figure12 shows the image matching effect of SIFT, SURF, BRISK, LATCH, LDB, TPLGD, ORB, MRA, and QTORB using SAR images. The feature point pair connected by the green line is a correct matching pair, and the point pair connected by the red line is a wrong matching pair.Figure 12
Matching result of the (a) SIFT, (b) SURF, (c) BRISK, (d) LATCH, (e) LDB, (f) TPLGD, (g) ORB, (h) MRA, and (i) QTORB algorithms using the selected SAR image containing a township scene.
(a)(b)(c)(d)(e)(f)(g)(h)(i)From Figure12, the distribution of the correct matched feature points by the SIFT, SURF, BRISK, LATCH, LDB, TPLGD, ORB, MRA, and QTORB is still consistent with that of the feature points extracted in Figure 11. The performance of the MRA and QTORB algorithms is confirmed again. In SAR images experiment, three image transformation scenarios are set according to the scale ratio and rotation angle including scaling, rotation, and mixing (scaling and rotation exist simultaneously). From Table 6, the QTORB algorithm can achieve the maximum uniform distribution of feature points in all these three image transformation scenarios.Table 6
Mean evenness of feature points on SAR dataset.
MethodScale = 0.9Rotation = 10Scale = 0.9Rotation = 10MeanSIFT259.9164257.2572255.6382257.6039SURF242.1884246.6370236.4556241.7603BRISK263.5500267.2996263.2762264.7086LATCH182.0922177.5690185.4978181.7197LDB180.8172179.3682173.2460177.8105TPLGD184.1988170.0662167.3348173.8666ORB179.6610171.1720185.9520178.9283MRA162.9324157.5000146.1532155.5285QTORB159.4978153.2828141.2876151.3561


## 4.5.2. Matching Precision on SAR Dataset Images

In experiment, one hundred SAR images were used. Five SAR images were randomly selected to visually display the test results. Figure13 shows the matching precision of SIFT, SURF, BRISK, LATCH, LDB, TPLGD, ORB, MRA, and QTORB using the selected five SAR images. Figure 13 intuitively shows that the QTORB algorithm can achieve image matching well in the given three image scenarios. In most cases, it can achieve better matching precision than ORB and MRA.Figure 13
Matching precision based on the selected SAR images. Three transformations are used: (a) scaling (scale = 0.9), (b) rotation (rotation = 10), and (c) mixing (scale = 0.9, rotation = 10).
(a)(b)(c)Table7 is calculated from all the one hundred tested SAR images. Under the three image transformation conditions specifically set, the highest matching precision is the SIFT. According to the mean data in Table 7, the QTORB algorithm is lower than the SIFT, BRISK, and TPLGD algorithms and higher than SURF, LATCH, LDB, ORB, and MRA.Table 7
Mean precision based on SAR dataset (%).
MethodScale = 0.9Rotation = 10Scale = 0.9Rotation = 10MeanSIFT99.727999.761699.726899.7388SURF96.193296.397794.397895.6629BRISK97.311697.939798.329897.8604LATCH96.739896.272896.703996.5721LDB97.239595.980395.846596.3554TPLGD97.965596.566696.513397.0151ORB94.757595.736695.587695.3606MRA95.496495.851897.232696.1936QTORB95.832195.927298.113696.6243


## 4.5.3. Matching RMSE on SAR Dataset Images

Figure14 and Table 8 show the RMSE indicators of the SIFT, SURF, BRISK, LATCH, LDB, TPLGD, ORB, MRA, and QTORB tested on SAR images. Under the three image transformation conditions specifically set, the RMSE of the SIFT is the lowest, which means that the positioning accuracy of the SIFT is the most accurate. According to the mean data of the three cases in Table 8, the RMSE of SIFT and SURF is significantly lower than that of BRISK, LATCH, LDB, TPLGD, ORB, MRA, and QTORB. The RMSE of the QTORB algorithm is lower than that of the ORB algorithm.Figure 14
Mean RMSE of SIFT, SURF, BRISK, LATCH, LDB, TPLGD, ORB, MRA, and QTORB using the SAR dataset images.Table 8
Mean RMSE based on SAR dataset (pixel).
MethodScale = 0.9Rotation = 10Scale = 0.9Rotation = 10MeanSIFT0.10900.11230.12420.1151SURF0.50880.47280.62190.5345BRISK0.69280.49940.60460.5989LATCH0.90150.87520.88370.8868LDB0.80250.81480.81540.8109TPLGD0.77400.91510.88070.8566ORB0.87760.89650.92010.8980MRA0.87800.90980.85190.8799QTORB0.87480.92140.88240.8929


## 4.5.4. Operation Time on SAR Dataset Images

Figure15 and Table 9 record the operation time of SIFT, SURF, BRISK, LATCH, LDB, TPLGD, ORB, MRA, and QTORB. It can be found that among all the algorithms tested, the SIFT algorithm takes the longest time and ORB algorithm takes the least time. The LATCH, LDB, TPLGD, MRA, and QTORB algorithms proposed based on the improvement of the ORB algorithm cost more time than ORB algorithm to different degrees. Compared with the MRA algorithm, the time-consuming increase of the QTORB algorithm is smaller. The operation time of QTORB is only slightly longer than that of ORB and still much less than that of MRA. It can also be concluded through experiments that the image matching algorithm based on binary feature description is much faster than the image matching algorithm based on floating-point feature description.Figure 15
Mean operation time of the SIFT, SURF, BRISK, LATCH, LDB, TPLGD, ORB, MRA, and QTORB based on the SAR dataset.Table 9
Mean operation time based on SAR dataset (s).
MethodScale = 0.9Rotation = 10Scale = 0.9Rotation = 10MeanSIFT1.16251.34851.27001.2603SURF0.66780.92320.78420.7917BRISK0.53910.58900.56110.5631LATCH0.43370.43740.43550.4355LDB0.45920.46090.43460.4516TPLGD0.33190.31430.29590.3140ORB0.17500.20470.19350.1911MRA0.32150.36880.34160.3439QTORB0.22570.25390.23420.2379


## 5. Discussion

The feature points detected by ORB algorithm and the matched feature points on subsequent matching are all concentrated and overlapping. To make the feature points evenly distributed, the MRA algorithm first adopted the quadtree method to manage and optimize the feature points. Its essence is to redistribute and optimize feature points by dividing the two-dimensional image space with the help of a quadtree. The algorithm achieves the goal of uniform distribution of feature points. In the feature point extraction stage, the MRA algorithm uses a lower threshold than the ORB algorithm, so MRA initially extracts more feature points than the ORB algorithm, and it should contain all the feature points extracted by the ORB. In fact, it can be understood that the feature points extracted by the ORB algorithm are the most prominent part of the feature points extracted by MRA. Therefore, in subsequent image matching, the matching precision of MRA is lower than the ORB algorithm. The QTORB algorithm proposed in this paper uses a dynamic threshold based on local pixels in the feature point extraction stage. Since the threshold value is not fixed, more high-quality feature points may be extracted in some areas than the ORB algorithm. Meanwhile, in feature description stage, gray difference value information is added to obtain more image space information, which improves the discriminative power of feature descriptors. Therefore, the QTORB algorithm can improve the matching precision under the premise of ensuring the uniform distribution of feature points. In terms of operating speed, MRA does not limit the depth of the quadtree. The quadtree will be excessively split, which will reduce the computational efficiency. The QTORB algorithm limits the depth of the quadtree, which can end the continued splitting of deep nodes early. And if the node contains a lot of feature points, ending the splitting of the node early has a more significant effect on reducing the calculation time. The experiment uses a variety of comparative algorithms, including several other algorithms based on ORB improvements. The experimental results prove that the improved algorithms based on the ORB algorithm also have the problem of uneven distribution of feature points, which is mainly caused by the feature point extraction algorithm adopted by the ORB algorithm. Therefore, the main method to improve the distribution of feature points is to improve the extraction algorithm and further carry out reasonable screening of the feature points after extraction. While realizing the homogenization of feature points, the matching precision also requires close attention. The matching precision is not only related to the extraction of feature points, but also more closely related to the description method. The stronger the discriminative power of the feature description vector, the better the precision. Therefore, while homogenizing the feature points, it is also necessary to improve the feature description method. On the premise of ensuring good matching precision and real-time performance, the uniform distribution of feature points can be achieved as much as possible.


## 6. Conclusions

This paper proposes a uniform ORB algorithm based on local dynamic threshold and improved quadtree, defined as Quadtree ORB (QTORB). QTORB, based on the ORB algorithm, focuses on research and improvement in the feature point extraction stage and feature point description stage. First, a feature point extraction method based on local dynamic threshold is adopted. This method can extract feature points uniformly on the image. After extracting the feature points, this paper draws on the quadtree-based method proposed by MRA to manage and optimize the feature points, avoiding excessive concentration and overlap of feature points. At the same time, in order to solve the time-consuming problem of the quadtree method, a limit is set on the split depth of the quadtree, which effectively reduces the calculation time. In the feature point description stage, an improved feature description method is proposed, which binarizes the image gray level difference information and implements a new binary feature using the feature fusion method. Through the verification of Oxford and SAR datasets, the QTORB algorithm proposed in this paper can effectively realize the uniform distribution of feature points under the premise that the precision and real-time performance are basically equivalent to the ORB algorithm and its other related improved algorithms.



---

*Source: 6693627-2021-01-05.xml*