# Image Antiblurring and Statistic Filter of Feature Space Displacement: Application to Visual Odometry for Outdoor Ground Vehicle

Authors: Xiangmo Zhao; Haigen Min; Zhigang Xu; Xia Wu; Xiaochi Li; Pengpeng Sun
Journal: Journal of Sensors
 (2018)

---

## Abstract

Precise, reliable, and low-cost vehicular localization across a continuous spatiotemporal domain is an important problem in the field of outdoor ground vehicles. This paper proposes a visual odometry algorithm, where an ultrarobust and fast feature-matching scheme is combined with an effective antiblurring frame selection strategy. Our method follows the procedure of finding feature correspondences from consecutive frames and minimizing their reprojection error. The blurred image is a great challenge for localization with a sharp turn or fast movement. So we attempt to mitigate the impact of blur with an image singular value decomposition antiblurring algorithm. Moreover, a statistic filter of feature space displacement and circle matching are proposed to screen or prune potential matching features, so as to remove the outliers caused by mismatching. An evaluation of benchmark dataset KITTI and real outdoor data, with blur, low texture, and illumination change, demonstrates that the proposed ego-motion scheme significantly achieved performance with respect to the other state-of-the-art visual odometry approaches to a certain extent.


---

## Body

## 1. Introduction

Precise, reliable, and low-cost vehicular localization across a continuous spatiotemporal domain of highway transportation is an important problem in the field of outdoor ground vehicles. Solving this issue is a breakthrough in promoting the future development of intelligent transportation, specifically in the following five aspects:


### 1.1. Conducive to Fine, Real-Time Traffic Information Accurate Perception

Compared with the traditional video-based or magnetic coil-based detection method, we can obtain more refined real-time traffic parameters by the high-precision vehicle position. With this information, we can calculate accurate micro and macrotraffic parameters (e.g., time headway, space headway, lane occupancy rate, average speed, traffic flow, traffic density, queue length, driving trajectory, congestion level, and accurate OD (origin-destination)). These parameters provide powerful data support for advanced intelligent transportation applications (e.g., vehicle collision avoidance, parking induction, navigation, platoon collaborative control, and road network dynamic traffic allocation).


### 1.2. Conducive to Establishment of the Vehicle Credit Records in the Vehicular Network

With the long time record of the precise position of the vehicle, the behavior safety level of the vehicle can be evaluated according to the driving trajectory of the vehicle. The recorded credit is valuable to vehicle collision warning, vehicle road right dynamic allocation, vehicle priority ranking, and other applications in the vehicular network.


### 1.3. Conducive to Early Warning of Traffic Accidents

In the future intelligent transportation system, if each vehicle can obtain its own precise position and that of the surrounding vehicle in real time, it can fuse other information (e.g., weather, road condition, road line shape, and obstacle) from the roadside to evaluate vehicle risk dynamically. The advance warning of traffic accident can avoid or reduce the probability of traffic accidents.


### 1.4. Conducive to Postanalysis of Traffic Accidents

In the future vehicular network, if the real-time location of the vehicle can be obtained and recorded exactly, microscale data in “before the accident-in the accident-after the accident” scenes can be obtained, which is favorable to reconstructing the situation before the collision and affirming the responsibility scientifically and objectively. The long-term observation of accurate data of traffic accidents is helpful to discovering the distribution law of traffic black spots in time and space, which can provide an important basis for road design and traffic accident prevention.


### 1.5. Conducive to the Cooperative Security Decision-Making Analysis with V2V (Vehicle to Vehicle) and V2I (Vehicle to Infrastructure)

A precise and reliable location is the most basic and important data for dynamic traffic information collection and integration and security control with V2V and V2I. The accuracy localization of the vehicle helps to get more refined real-time traffic parameters according to these precise traffic parameters of the vehicle: the optimal speed in the designated area, the probability of joining or leaving a platoon, and the optimal path selection under the premise of comprehensive consideration of trip time and driving distance.In short, a major breakthrough of large-scale, precise, reliable, low-cost, and real-time vehicular localization will greatly reduce the road traffic system uncertainty and randomness, make the road traffic system further close to the rail transit system, and greatly improve the transport efficiency and safety without reducing the mobility at the same time. In this paper, a precise, reliable, and low-cost ego-motion estimation methodology, especially ones involving an image antiblurring and statistic filter of feature space displacement framework, is proposed. There are three main contributions of this paper.(1)
An image singular value decomposition (ISVD) antiblurring algorithm is proposed to mitigate the effect of image blur. To highlight the efficiency and performance of the proposed method ISVD, many comparisons have been conducted with other image-blurred degree assessment algorithms.
(2)
For feature displacement, not only adjacent matched framesIlp,Ilc or Irp,Irc but also the left and right matched frames Ilp,Irp or Ilc,Irc in a stereo model conform to a Laplacian distribution. Based on this discovery, we proposed a statistic filter of a feature space displacement algorithm to remove the outliers in feature correspondences.
(3)
Experiments are conducted on a public benchmark dataset and data captured with our outdoor ground vehicle platform. We analyze the results from quantitative and qualitative aspects to demonstrate the superiority of our proposed method.


## 1.1. Conducive to Fine, Real-Time Traffic Information Accurate Perception

Compared with the traditional video-based or magnetic coil-based detection method, we can obtain more refined real-time traffic parameters by the high-precision vehicle position. With this information, we can calculate accurate micro and macrotraffic parameters (e.g., time headway, space headway, lane occupancy rate, average speed, traffic flow, traffic density, queue length, driving trajectory, congestion level, and accurate OD (origin-destination)). These parameters provide powerful data support for advanced intelligent transportation applications (e.g., vehicle collision avoidance, parking induction, navigation, platoon collaborative control, and road network dynamic traffic allocation).


## 1.2. Conducive to Establishment of the Vehicle Credit Records in the Vehicular Network

With the long time record of the precise position of the vehicle, the behavior safety level of the vehicle can be evaluated according to the driving trajectory of the vehicle. The recorded credit is valuable to vehicle collision warning, vehicle road right dynamic allocation, vehicle priority ranking, and other applications in the vehicular network.


## 1.3. Conducive to Early Warning of Traffic Accidents

In the future intelligent transportation system, if each vehicle can obtain its own precise position and that of the surrounding vehicle in real time, it can fuse other information (e.g., weather, road condition, road line shape, and obstacle) from the roadside to evaluate vehicle risk dynamically. The advance warning of traffic accident can avoid or reduce the probability of traffic accidents.


## 1.4. Conducive to Postanalysis of Traffic Accidents

In the future vehicular network, if the real-time location of the vehicle can be obtained and recorded exactly, microscale data in “before the accident-in the accident-after the accident” scenes can be obtained, which is favorable to reconstructing the situation before the collision and affirming the responsibility scientifically and objectively. The long-term observation of accurate data of traffic accidents is helpful to discovering the distribution law of traffic black spots in time and space, which can provide an important basis for road design and traffic accident prevention.


## 1.5. Conducive to the Cooperative Security Decision-Making Analysis with V2V (Vehicle to Vehicle) and V2I (Vehicle to Infrastructure)

A precise and reliable location is the most basic and important data for dynamic traffic information collection and integration and security control with V2V and V2I. The accuracy localization of the vehicle helps to get more refined real-time traffic parameters according to these precise traffic parameters of the vehicle: the optimal speed in the designated area, the probability of joining or leaving a platoon, and the optimal path selection under the premise of comprehensive consideration of trip time and driving distance.In short, a major breakthrough of large-scale, precise, reliable, low-cost, and real-time vehicular localization will greatly reduce the road traffic system uncertainty and randomness, make the road traffic system further close to the rail transit system, and greatly improve the transport efficiency and safety without reducing the mobility at the same time. In this paper, a precise, reliable, and low-cost ego-motion estimation methodology, especially ones involving an image antiblurring and statistic filter of feature space displacement framework, is proposed. There are three main contributions of this paper.(1)
An image singular value decomposition (ISVD) antiblurring algorithm is proposed to mitigate the effect of image blur. To highlight the efficiency and performance of the proposed method ISVD, many comparisons have been conducted with other image-blurred degree assessment algorithms.
(2)
For feature displacement, not only adjacent matched framesIlp,Ilc or Irp,Irc but also the left and right matched frames Ilp,Irp or Ilc,Irc in a stereo model conform to a Laplacian distribution. Based on this discovery, we proposed a statistic filter of a feature space displacement algorithm to remove the outliers in feature correspondences.
(3)
Experiments are conducted on a public benchmark dataset and data captured with our outdoor ground vehicle platform. We analyze the results from quantitative and qualitative aspects to demonstrate the superiority of our proposed method.


## 2. Related Work

This paper focuses on the innovation in the front end of the visual information-based ego-motion estimation method, so here we just review some works in the front-end procedure. The accuracy and reliability of feature detection and matching are the bases of the whole vision-based ego-motion estimation method. To eliminate the error caused by feature detection and matching or tracking, some effective tricks have been developed in the previous relevant literatures. Badino et al. [1] used the whole history of the tracked feature points to compute the motion of the camera, which reduced drift significantly with a negligible additional computational cost. They also discovered that the feature interframe tracking error follows a Laplacian distribution in all three dimensions, while this property was not employed to remove the outliers. The concept of temporal flow [2] restricts the uncertainty to reduce error propagation from matches. A more direct way is careful feature selection and tracking, for example, ORB based [3], KLT based [4], and AKAZE based [5]. With further analysis, we find that the feature displacement statistic for a sequential frame conforms to a Laplacian-like distribution with a zero mean. Badino et al. [1] had explored this distribution while they ignored the ability of this distribution in the matching procedure.In the feature-based framework, common feature detectors, however, cannot detect enough features from a blurred image, especially for the outdoor ground vehicle in a sharp turn. The blur effect can further degrade the matching accuracy between frames (Figure1(a); the number of good matches is decreasing from 111 to 15 with a more and more serious blur impact) and detect the inaccurate loop closure (Figure 1(b)). Figures 1(b), b1 and 1(b), b2 show an almost similar scene, and Figure 1(b), b2 is blurring. The similarity between Figures 1(b), b1 and 1(b), b2 is 76%, while that between Figures 1(b), b1 and 1(b), b3 is 92%, which has a great effect on reconstruction and localization performance. When an image is captured when the camera is moving during the exposure time, the one-to-one relationship between the scene points and the image points is broken and a certain number of scene points are projected at a single pixel contributing to the final pixel value. This effect is called motion blur. Motion blur is the most common artifact in images captured by a moving robot mounted with a camera. Usually, there are four types of blur, that is, linear blur, rotation blur, zoom blur, and Gaussian blur, considered as to occur in a moving environment. In order to remove or reduce the harm caused by blurry images, generally, there are two ways used, image deblurring and antiblurring. The former [6] recovers the blurred image by deconvolution, which will sacrifice the feature accuracy and is time consuming. The latter [7–9] classifies the images into blurry and clear images and then eliminates the blurry ones. Lee et al. [10] presented robust data association and drift-free simultaneous localization and mapping (SLAM) with blurred images. The blurred images were recovered by deconvolution. Obviously, this deblurring method was not applicable for outdoor ground vehicle localization. Zhao et al. [11] presented an adaptive blurred image classification algorithm, which was effective but could not be applied to outdoor ground vehicles with feature-matching failure. Pretto et al. [12] proposed a visual odometry (VO) method robust to motion blur based on the improved feature detector SIFT [13]. However, only Gaussian blur is considered in this work; the other three common types of blurring effect are ignored.Figure 1
The blur impact on feature detection and matching and loop closure detection: (a) the impact on feature detection and matching; (b) the impact on loop closure detection.
(a)
(b)In general, there are three ways to estimate visual ego-motion, namely, VO, structure from motion (SFM), and SLAM. VO is devoted to the real-time and accurate estimation of camera movement while SFM focuses on 3-dimensional reconstruction as well as camera pose estimation and usually refines the estimation with bundle adjustment. VO can be thought of as a special case of SFM, whereas SLAM’s purposes are for localizing a robot in an unknown environment. More specifically, SLAM recovers the trajectory of the camera and builds a map at the same time. The concept of VO was formally put forward by Nister et al. [14], who realized a real-time VO system. The later research was mostly based on this VO framework. The most successful VO applications were NASA’s Mars Exploration Rovers, Spirit and Opportunity [15]. An image pyramid was introduced to help feature tracking in 2011 on Curiosity. Howard [16] implemented a stereo VO with an adopted Harris and Fast feature to ensure real-time performance and employed a feature-matching method [17] to find the correspondences. The aforementioned research laid a good foundation for the development of VO, but they suffer from a low accuracy. Geiger et al. [18, 19] used a simple Sobel template operator to detect the feature for a real-time system. They tested the VO algorithm on the KITTI benchmark dataset and obtained the positioning result with high efficiency and accuracy. A KITTI vision benchmark suite was established by Andreas (MPI Tübingen), Philip Lenz (KIT), Christoph Stiller (KIT), Raquel Urtasun (University of Toronto), and so on. Bellavia et al. [2] proposed key frame matching and loop closure to build a stereo camera SLAM system. Badino et al. [1] integrated features from multiple frames to improve the accuracy of the motion estimation, and they also combined the historical image information to obtain the refined motion estimation. Mur-Artal et al. [20] proposed a feature-based monocular ORB-SLAM system, which selected the points and keyframes for reconstruction and obtained an excellent performance. ORB-SLAM is the most outstanding work with a feature-based system that runs in real time, in small and large indoor and outdoor environments. Differently from the feature-based method, Engel et al. [21] relied on the photoconsistency of high-contrast pixels, including corners, edges, and high-texture areas. What is more, an illumination invariance method removed the effect of brightness changes. DSO [22] and LSD-SLAM [23] abandoned the feature detection procedure and directly used the actual sensor value light received from a certain direction over a certain time period. Hu and Chen [24] combined a monocular and an inertial measurement unit (IMU) with a visual odometer. They employed trifocal tensor geometry information and a multistate constraint Kalman filter in the algorithm architecture to reduce the time consumption and enhance the accuracy of the algorithm. Besides the fusion applications of visual information and IMU, VO is usually used as a supplementary to the global positioning system (GPS) [25, 26]. Fusion approaches are becoming the mainstream of research about the practical positioning and navigation system. Although there are many multisensor solution schemes for localization, the potential of VO has not been unlocked completely. With the development of deep learning (DL) in computer vision, some researchers have imported DL into localization [27, 28]. Typically, DL is used to detect the loop closure, which is different from the conventional way of bag of words (BoW) [29]. For a more comprehensive and in-depth literature survey of the VO algorithms that have been developed, the reader may refer to [30–34].


## 3. Proposed Method

In this section, the system model and motion parameterization are briefly introduced. Figure2 shows our experimental platform equipped with a high-resolution stereo camera rig (Basler acA1600 GigE, image size 1200 × 800 pixels) and differential GPS (DGPS, RT2000).Figure 2
Our vehicle platform with stereo camera and DGPS.Then, the image antiblurring algorithm based on blurred degree calculation with ISVD is presented. What is more, we propose a statistic filter of feature displacement to obtain the ultrarobust feature correspondences.


### 3.1. System Model and Motion Parameterization

In this paper, we parameterize the motion as the vectorT.
(1)T=rx,ry,rz,tx,ty,tz,where rx,ry,rz and tx,ty,tz represent the rotation and translation vectors between the consecutive frames, respectively.In each procedure of pose estimation,T is estimated with N matched points. For stereo VO, there are four feature points in each matching step, namely, ulc,vlc, urc,vrc, ulp,vlp, and urp,vrp, where l,r,c,andp represent the left, right, current, and previous frames, respectively. Xlc,Ylc,Zlc, Xrc,Yrc,Zrc, Xlp,Ylp,Zlp, and Xrp,Yrp,Zrp are the 3d coordinates corresponding to the four feature points, as shown in
(2)X=u−cu∗bd,Y=v−cv∗bd,Z=f∗bd,where
(3)C=f0cu0fcv001is the camera intrinsic matrix, f is the focal length, cu,cv is the principal point of the camera, b is the base length, and d is the disparity.During theith Gaussian-Newton gradient decent iteration, the motion vector Ti=rxi,ryi,rzi,txi,tyi,tzi is refined by three random features selected by random sample consensus (RANSAC) [35]. The incremental equation represents
(4)JJT∆T′=Jr,where r=w∗u,v−u^−v^ is the residual and ∆T′ is the homogeneous form of ∆T. Here, w is designed for reducing the error caused by inaccurate camera calibration. w=cu/u−cu is small if the feature point is far away from camera. J is the Jacobian matrix represented as
(5)Ju,vT=∂u∂rx∂u∂ry∂u∂rz∂u∂tx∂u∂ty∂u∂tz∂v∂rx∂v∂ry∂v∂rz∂v∂tx∂v∂ty∂v∂tz.If the stopping criterion for iteration meetsTi+1−Ti<ε, gradient decent iteration is terminated. Otherwise, gradient decent iteration will continue with Ti+1=Ti+JJT−1Jr.


### 3.2. ISVD Antiblurring Algorithm

Image blurring is a tough problem in visual applications, especially for outdoor ground vehicles with a sharp turn. Some efforts have been made for deblurring [6, 36] with a deconvolutional model, which is not calculable in a real-time system, for example, VO and SLAM. Zhao et al. [11] designed an adaptive blurred image classification framework, which selected an image in a relatively slightly blurred degree to calculate the ego-motion. This method can eliminate the blur effect to a certain extent while the image-blurred degree calculation algorithm SIGD is less impressive than the other no-reference image quality assessment algorithms, for example, Marziliano [7], JNB [8], and CPBD [9]. This paper proposes an outstanding image quality metric assessment method, image singular value decomposition (ISVD), to enhance the performance of image antiblurring.For an image matrixA∈Rm×n, the orthogonal matrixes U∈Rm×m and V∈Rn×n compose A=UWVT. δ1≥δ2≥⋯≥δn in W=diagδ1,δ2,…,δn are the nonzero singular values. To explore the relationship between the singular value and image quality, we choose six images (Figures 3(a)–3(e)) with an increasing blur degree from the LIVE database [37], as shown in Figure 3.Figure 3
Images with different blur degrees from LIVE2 database: image (a): 1.59098; image (b): 9.15433; image (c): 2.37732; image (d): 3.10557; image (e): 4.24759; image (f): 0.84185.
(a)
(b)
(c)
(d)
(e)
(f)Figure4 shows the singular value curve of those different blurred degree images in Figure 3. After decomposing the image matrix, the singular value histogram is built on the nonzero singular values. In this figure, every singular value corresponds to a different singular value number. So we can verify the blur degree with a singular value number without any ambiguity, exactly as the threshold line describes. ISVD is defined as follows:
(6)ISVD=∑inSn,(7)S=1,ifδi≥Cthres,0,ifδi<Cthres,where n is the size of a singular value of W and δi is the singular value in W.Cthres is a singular value threshold, and we set 100 in this paper.Figure 4
The singular value curve for images in Figure3.The image blurred degree calculation with ISVD is summarized as Algorithm1.Algorithm 1: Blurred degree calculation with ISVD.
Input: image I.
Output: blurred degree b of image I
Initialization: b=0,num=0
1 convertI into grayscale Igray
2 calculation singular valueW=diagδ1,δ2,…,δn with singular value decomposition on Igray
3 forδi in W
4  ifδi≥Cthres
5num++
6  end
7 end
8 return blurred degreeb=num/nThen, an adaptive blurred image classification is adopted to select a less blurred image for the following procedures. ISVD’s time complexity isON3. If the image matrix is too large, the decomposition procedure is too slow. To overcome the disadvantage of time consumption, antiblurring is operated only on keyframe, which performs more efficiently than a frame-by-frame model does.


### 3.3. Statistic Filter of Feature Space Displacement

For the discovery in [9], the feature tracking error follows a Laplacian distribution. We identify that not only the adjacent matched frames Ilp,Ilc or Irp,Irc but also the left and right matched frames Ilp,Irp or Ilc,Irc in the stereo model conform to this distribution. The feature space displacement is the displacement along the x or y coordinate in the feature space. In all the original matching features, including inliers and outliers, displacements are collected, the fact being that the displacement for inliers and outliers between frames is fitted with a Laplacian distribution with a long tail, as shown in Figure 5. The corresponding features are more close to the peak bin, which are more likely inliers. Feature displacements for inliers between frames are shown in Figure 6 with a short tail. The x-axis means the displacement in pixels, and the y-axis is the number of features collected by histogram statistics.Figure 5
Feature displacement between frames, including inliers and outliers. Blue, measured displacement error; red, fitted Laplacian probability distribution function (pdf) with a long tail.
(a)
x coordinate
(b)
y coordinateFigure 6
Feature displacement for inliers between frames. Blue, measured displacement error; red, fitted Laplacian pdf with a short tail.
(a)
x coordinate
(b)
y coordinateOn the basis of this discovery, we propose a statistic filter of feature space displacement (SFFSD) to remove the outliers and retain the inliers as shown in Algorithm2. This algorithm only describes the statistic filter between two candidate matched images I1 and I2.Algorithm 2: Statistic filter of feature displacement.
Input: Candidate matched images I1,I2.
Output: good feature matches VMgood
1 detect featuresI1,I2 to get descriptors Mdes1,Mdes2 and key-points Mkeys1,Mkeys2
2 matchMdes1,Mdes2 to get the original matches VMmats with brute force matcher and hamming distance
3 Calculate the key-points displacements forVMmats in x and y components ∆u=u1−u2, ∆v=v1−v2
4 create a 2D histogram with∆u and ∆v to confirm the highest bin for mode approximation
5 use the sample within the radius to perform parameter estimation of Laplacian distribution in x and y
6 determine the min and max boundary value to include a certain percentageratio=0.9 of inliers, assuming a Laplacian distribution
7 find the matchesVMu_mats according to the boundary in Step 6
8 repeat Step 6 to Step 7 to find the matchesVMv_mats
9 calculate the common elementVcom from the VMu_mats and VMv_mats
10 forid in VMmats
11  ifid in Vcom
12      pushback the corresponding element intoVMgood
13  end
14 endThe bin number is set asn=5. In that case, the corresponding features cover 80% inliers. However, the value of n is not set in stone, which depends on the diversity of the detected features. To further refine the matches after statistic filter, circle matching is employed to verify the matches [19]. Three points are randomly selected to implement RANSAC from the above good feature matches at each iteration. The ultrarobust matches improve the accuracy; meanwhile, they accelerate the convergence process of RANSAC.


## 3.1. System Model and Motion Parameterization

In this paper, we parameterize the motion as the vectorT.
(1)T=rx,ry,rz,tx,ty,tz,where rx,ry,rz and tx,ty,tz represent the rotation and translation vectors between the consecutive frames, respectively.In each procedure of pose estimation,T is estimated with N matched points. For stereo VO, there are four feature points in each matching step, namely, ulc,vlc, urc,vrc, ulp,vlp, and urp,vrp, where l,r,c,andp represent the left, right, current, and previous frames, respectively. Xlc,Ylc,Zlc, Xrc,Yrc,Zrc, Xlp,Ylp,Zlp, and Xrp,Yrp,Zrp are the 3d coordinates corresponding to the four feature points, as shown in
(2)X=u−cu∗bd,Y=v−cv∗bd,Z=f∗bd,where
(3)C=f0cu0fcv001is the camera intrinsic matrix, f is the focal length, cu,cv is the principal point of the camera, b is the base length, and d is the disparity.During theith Gaussian-Newton gradient decent iteration, the motion vector Ti=rxi,ryi,rzi,txi,tyi,tzi is refined by three random features selected by random sample consensus (RANSAC) [35]. The incremental equation represents
(4)JJT∆T′=Jr,where r=w∗u,v−u^−v^ is the residual and ∆T′ is the homogeneous form of ∆T. Here, w is designed for reducing the error caused by inaccurate camera calibration. w=cu/u−cu is small if the feature point is far away from camera. J is the Jacobian matrix represented as
(5)Ju,vT=∂u∂rx∂u∂ry∂u∂rz∂u∂tx∂u∂ty∂u∂tz∂v∂rx∂v∂ry∂v∂rz∂v∂tx∂v∂ty∂v∂tz.If the stopping criterion for iteration meetsTi+1−Ti<ε, gradient decent iteration is terminated. Otherwise, gradient decent iteration will continue with Ti+1=Ti+JJT−1Jr.


## 3.2. ISVD Antiblurring Algorithm

Image blurring is a tough problem in visual applications, especially for outdoor ground vehicles with a sharp turn. Some efforts have been made for deblurring [6, 36] with a deconvolutional model, which is not calculable in a real-time system, for example, VO and SLAM. Zhao et al. [11] designed an adaptive blurred image classification framework, which selected an image in a relatively slightly blurred degree to calculate the ego-motion. This method can eliminate the blur effect to a certain extent while the image-blurred degree calculation algorithm SIGD is less impressive than the other no-reference image quality assessment algorithms, for example, Marziliano [7], JNB [8], and CPBD [9]. This paper proposes an outstanding image quality metric assessment method, image singular value decomposition (ISVD), to enhance the performance of image antiblurring.For an image matrixA∈Rm×n, the orthogonal matrixes U∈Rm×m and V∈Rn×n compose A=UWVT. δ1≥δ2≥⋯≥δn in W=diagδ1,δ2,…,δn are the nonzero singular values. To explore the relationship between the singular value and image quality, we choose six images (Figures 3(a)–3(e)) with an increasing blur degree from the LIVE database [37], as shown in Figure 3.Figure 3
Images with different blur degrees from LIVE2 database: image (a): 1.59098; image (b): 9.15433; image (c): 2.37732; image (d): 3.10557; image (e): 4.24759; image (f): 0.84185.
(a)
(b)
(c)
(d)
(e)
(f)Figure4 shows the singular value curve of those different blurred degree images in Figure 3. After decomposing the image matrix, the singular value histogram is built on the nonzero singular values. In this figure, every singular value corresponds to a different singular value number. So we can verify the blur degree with a singular value number without any ambiguity, exactly as the threshold line describes. ISVD is defined as follows:
(6)ISVD=∑inSn,(7)S=1,ifδi≥Cthres,0,ifδi<Cthres,where n is the size of a singular value of W and δi is the singular value in W.Cthres is a singular value threshold, and we set 100 in this paper.Figure 4
The singular value curve for images in Figure3.The image blurred degree calculation with ISVD is summarized as Algorithm1.Algorithm 1: Blurred degree calculation with ISVD.
Input: image I.
Output: blurred degree b of image I
Initialization: b=0,num=0
1 convertI into grayscale Igray
2 calculation singular valueW=diagδ1,δ2,…,δn with singular value decomposition on Igray
3 forδi in W
4  ifδi≥Cthres
5num++
6  end
7 end
8 return blurred degreeb=num/nThen, an adaptive blurred image classification is adopted to select a less blurred image for the following procedures. ISVD’s time complexity isON3. If the image matrix is too large, the decomposition procedure is too slow. To overcome the disadvantage of time consumption, antiblurring is operated only on keyframe, which performs more efficiently than a frame-by-frame model does.


## 3.3. Statistic Filter of Feature Space Displacement

For the discovery in [9], the feature tracking error follows a Laplacian distribution. We identify that not only the adjacent matched frames Ilp,Ilc or Irp,Irc but also the left and right matched frames Ilp,Irp or Ilc,Irc in the stereo model conform to this distribution. The feature space displacement is the displacement along the x or y coordinate in the feature space. In all the original matching features, including inliers and outliers, displacements are collected, the fact being that the displacement for inliers and outliers between frames is fitted with a Laplacian distribution with a long tail, as shown in Figure 5. The corresponding features are more close to the peak bin, which are more likely inliers. Feature displacements for inliers between frames are shown in Figure 6 with a short tail. The x-axis means the displacement in pixels, and the y-axis is the number of features collected by histogram statistics.Figure 5
Feature displacement between frames, including inliers and outliers. Blue, measured displacement error; red, fitted Laplacian probability distribution function (pdf) with a long tail.
(a)
x coordinate
(b)
y coordinateFigure 6
Feature displacement for inliers between frames. Blue, measured displacement error; red, fitted Laplacian pdf with a short tail.
(a)
x coordinate
(b)
y coordinateOn the basis of this discovery, we propose a statistic filter of feature space displacement (SFFSD) to remove the outliers and retain the inliers as shown in Algorithm2. This algorithm only describes the statistic filter between two candidate matched images I1 and I2.Algorithm 2: Statistic filter of feature displacement.
Input: Candidate matched images I1,I2.
Output: good feature matches VMgood
1 detect featuresI1,I2 to get descriptors Mdes1,Mdes2 and key-points Mkeys1,Mkeys2
2 matchMdes1,Mdes2 to get the original matches VMmats with brute force matcher and hamming distance
3 Calculate the key-points displacements forVMmats in x and y components ∆u=u1−u2, ∆v=v1−v2
4 create a 2D histogram with∆u and ∆v to confirm the highest bin for mode approximation
5 use the sample within the radius to perform parameter estimation of Laplacian distribution in x and y
6 determine the min and max boundary value to include a certain percentageratio=0.9 of inliers, assuming a Laplacian distribution
7 find the matchesVMu_mats according to the boundary in Step 6
8 repeat Step 6 to Step 7 to find the matchesVMv_mats
9 calculate the common elementVcom from the VMu_mats and VMv_mats
10 forid in VMmats
11  ifid in Vcom
12      pushback the corresponding element intoVMgood
13  end
14 endThe bin number is set asn=5. In that case, the corresponding features cover 80% inliers. However, the value of n is not set in stone, which depends on the diversity of the detected features. To further refine the matches after statistic filter, circle matching is employed to verify the matches [19]. Three points are randomly selected to implement RANSAC from the above good feature matches at each iteration. The ultrarobust matches improve the accuracy; meanwhile, they accelerate the convergence process of RANSAC.


## 4. Experimental Results and Analysis

### 4.1. Performance Evaluation of ISVD

This paper applies the video quality assessment index proposed by the Video Quality Experts Group (VQEG) to test the performance of different blurred degree assessment algorithms. A few parameters are mentioned as follows. With these parameters, we can evaluate the performance of different algorithms.(1)
Outlier ratio (OR):(8)OR=n1n,where n1 is the number of outlier images and n is the total number of images. If the difference between scores calculated by the blurred degree assessment algorithm and actual reference score is larger than the threshold value, the image is regarded as an outlier image. The smaller the OR is, the better the consistency of the algorithm.
(2)
Root mean square error (RMSE):(9)RMSE=1n∑i=1nsi−spi2,where si and spi are the reference score and score calculated by the blurred degree assessment algorithm of the ith image, respectively. The smaller the RMSE, the better the predicted result of the assessment algorithm.
(3)
Mean absolute error (MAE):(10)MAE=1n∑i=1nsi−spi,where MAE sums the absolute value of the residual si−spi. In statistics, MAE is a measure of the difference between two continuous variables. The smaller MAE is, the better the value calculated by the assessment algorithm.
(4)
Spearman rank-order correlation coefficient (SROCC):(11)SROCC=1−6∑i=1nMOSri−MOSrpi2nn2−1,where MOSri (mean opinion scores) is the reference score of the ith image and MOSrpi is the value calculated by the assessment algorithm of the ith image. The bigger SROCC is, the better the performance of the assessment algorithm.
(5)
Pearson linear correlation coefficient (PCC):(12)PCC=∑i=1nMOSi−MOS¯MOSpi−MOSp¯∑i=1nMOSi−MOS¯2∑i=1nMOSpi−MOSp¯2,where MOSi is the actual reference score, MOS¯ is the average reference score of n images, MOSpi is the scores calculated by the assessment algorithm of the ith image, and MOSp¯ is the average calculated score of n images. The bigger PCC is, the better the consistency between the score calculated by the assessment algorithm and the actual reference score.With different blurred degree assessment algorithms (Marziliano, JNB, CPBD, SIGD, and ISVD), we calculate the blurred degree for 174 images in dataset LIVE [37]. The actual reference score of those images is known. The image quality parameters OR, RMSE, MAE, SROCC, and PCC are calculated with different assessment algorithms, as Table 1 shows. Obviously, the proposed ISVD performs more outstandingly than do other algorithms. The blurred degree calculated by ISVD remains monotonic over the full range of the data, as suggested in [36].Table 1
Image blur metrics of five different algorithms.
Algorithm
PCC
SROCC
RMSE
MAE
OR
Marziliano
0.8562
0.8672
8.8354
6.4382
0.1264
JNB
0.8498
0.8345
9.0117
6.9317
0.1322
CPBD
0.9257
0.9444
6.4662
4.7098
0.0632
SIGD
0.7596
0.6877
11.1176
9.0331
0.2759
ISVD
0.9405
0.9293
5.8069
4.4090
0.0230The high efficiency is critical to the real-time system. As shown in Table2, ISVD is less significantly efficient than SIGD, but it is sufficient for real-time VO.Table 2
The execution times of five different algorithms.
Algorithm
Marziliano
JNB
CPBD
SIGD
ISVD
Time (s)
0.5860
0.6156
0.7084
0.0226
0.0082


### 4.2. Performance Evaluation of Statistic Filter

We analyze SFFSD’s performance against the alternative ratio test (RT; if the ratio of the nearest correspondence distance to the second nearest correspondence distance is less than a threshold, a good match is declared), introduced in SIFT; on the aspect of precisionP=numberofcorrectmatches/totalnumberofmatches, recall R=numberofcorrectmatches/numberofcorrespondences, and F1-measure F1=2PR/P+R, which are critical to the matching performance. An evaluation is conducted on Oxford dataset VGG [37, 38] and TUM [39], as shown in Figure 7.Figure 7
Precision, recall, andF-measure score. The x-axis of VGG represents the image index, while the x-axis of TUM represents the relative rotation in degrees. SFFSD is black; RT is red.
(a)
(b)VGG has different photometric and geometric transformations, for example, Gaussian blur, lighting variation, scale, and rotation changes. The first frame in each set is designated as the reference. All other 5 frames, indexed with 0–4, from the same set are matched to the reference. TUM has video sequences with low texture and blurriness. Obviously, SFFSD has relatively higherF1 than the RT.Figure8 shows examples of outlier removal with SFFSD, where blue and red lines represent inliers and outliers, respectively. After the SFFSD, circle matching is employed to verify the matches (Figure 9).Figure 8
Outlier removing with SFFSD. An image pair and its putative correspondences. Blue and red lines represent inliers and outliers, respectively.Figure 9
The sample images in real outdoor environment and the circle matching described in Section4.2.
(a)
Regular scene
(b)
Pedestrian
(c)
Car
(d)
High exposure
(e)
Surface gathered water
(f)
Sharp turn, motion blur


### 4.3. Experiments and Comparisons

Experiments have been conducted on the open benchmark dataset KITTI [19] and the real outdoor and indoor datasets. In consideration of feature detection and matching efficiency≥efficiency, ORB [37] is regarded as a better choice. The comparison with other detectors and descriptors, like SIFT, SURF [40], KAZE [41], and AKAZE [42], has been carried out in [41]. To emphasize the excellent performance, we employ two basic VO algorithms, libviso2 (VISO2) [43] and ORB-SLAM2 [3], as comparisons in our experiments. The feature detector and descriptor in VISO2 are replaced with ORB, named ORB-VISO2. Our method is classified into two types, that is, method without loop closure and bundle adjustment based on VISO2 (Our-method1) and method with loop closure and bundle adjustment based on ORB-SLAM2 (Our-method2). No matter what the algorithms Our-method1 and Our-method2 are based on, the major difference is our contribution work of image antiblurring and statistic filter of feature space displacement. Table 3 shows the comparison of processing times among VIOS2, ORB-VISO2, ORB-SLAM2, Our-method1, and Our-method2. The spending time of ISVD and SFFSD is within 13 ms, which has a negligible effect on the total execution time.Table 3
The average processing time of one image frame (ms).
Method
VISO2
ORB-VISO2
ORB-SLAM2
Our-method1
Our-method2
Time (ms)
17.833
16.130
62.002
21.676
67.519


#### 4.3.1. Experiments on Benchmark Datasets

The first experiment is conducted≥conducted on the KITTI benchmark dataset. The KITTI dataset consists of 22 stereo sequences, and we randomly take sequences 02 and 05 as comparison datasets. The trajectories of different algorithms and ground truth are drawn on the same figure.From Figure10, we can see that our method is closer to the ground truth than the art-of-the-state method. Regardless of the rotation error or translation error, our method performs better. Compared with other VO algorithms, the absolute trajectory errors of sequence 00–10 are shown in Table 4. In nonloop closure and nonbundle adjustment methods, Our-method1 is better than VISO2 and ORB-VISO2. In sequences “KITTI 01” and “KITTI 08,” ORB-VISO2 fails with fast motion on the highway or illumination changing. In loop closure and bundle adjustment methods, Our-method2 is better than ORB-SLAM2. The longer the trajectory is, the more superiority Our-method2 has.Figure 10
Randomly taking sequences 02 and 05 on the KITTI dataset as comparison datasets. Motion estimation on sequences 02 and 05 with ground truth, VISO2, ORB-VISO2, ORB-SLAM2, Our-method1, and Our-method2. The left column shows the trajectory, and the right column shows the translation and rotation error. (a) Comparison result on KITTI 02. (b) Comparison result on KITTI 05.
(a)
(b)Table 4
Absolute trajectory error RMSE (m).
Sequence
Environment
Distances
VISO2
ORB-VISO2
ORB-SLAM2
Our-method1
Our-method2
KITTI 00
Urban
3714 m
30.133
29.430
1.328
21.016
0.673
KITTI 01
Highway
4268 m
51.430
X
10.936
45.595
4.391
KITTI 02
Urban + country
5075 m
36.441
26.831
5.534
23.185
5.412
KITTI 03
Country
563 m
2.570
2.339
0.619
2.146
0.309
KITTI 04
Country
397 m
1.483
1.3230
0.223
1.190
0.114
KITTI 05
Urban
2223 m
13.653
12.287
0.846
11.740
0.337
KITTI 06
Urban
1239 m
4.318
3.917
0.758
3.757
0.302
KITTI 07
Urban
695 m
5.561
5.017
0.554
4.800
0.221
KITTI 08
Urban + country
3225 m
20.289
X
3.660
17.464
1.470
KITTI 09
Urban + country
1717 m
19.581
16.298
3.427
15.986
1.373
KITTI 10
Urban + country
919 m
4.367
3.812
1.165
3.593
0.468


#### 4.3.2. Experiments in Outdoor and Indoor Environment

In this section, to verify the stability and scalability, we also carry out the experiments in real outdoor and indoor environments. The data are captured in a considerably cluttered environment on campus, which contains challenging scenes like dynamic objects, illumination changing, surface gathered water, and motion blur. The image samples are shown in Figure9.The trajectories recovered using the proposed method and other methods are shown in Figure11(a). The trajectory was approximately 1.45 km and lasted 236.4 s. We regard the trajectory of DGPS as the ground truth. VISO2 and ORB-VISO2 failed to locate with the instability factor in an outdoor environment. It can be found that Our-method1 is not reliable with the error accumulation. ORB-SLAM2 and Our-method2 perform well with loop closure and bundle adjustment. Obviously, Our-method2 has less drift than ORB-SLAM2 has. As shown in Table 5, the proposed method outperforms other methods in terms of overall RMSE and end-point error.Figure 11
Trajectory estimated using the proposed method and DGPS trajectory. The camera frequency was 30 Hz. (a) The trajectory was approximately 2.76 km and lasted 476.4 s. (b) The trajectory was approximately 1.17 km and lasted 109.3 s.
(a)
(b)Table 5
Overall RMSE and end-point error, corresponding to Figure11(a).
Error
VISO2
ORB-VISO2
ORB-SLAM2
Our-method1
Our-method2
Overall position RMSE (m)
X
X
9.006
34.961
5.750
Overall orientation RMSE (deg)
X
X
0.108
0.489
0.060
End-point position error (m)
X
X
2.927
87.788
1.175
End-point orientation error (deg)
X
X
0.434
2.506
0.358To further support the opinions proposed in this paper, we conduct another experiment to compare the performance of different VO and SLAM kind algorithms. The trajectory in Figure11(b) is 1.17 km and lasted 109.3 s. The average speed is about 38.5 km/h. Unlike the dataset in Figure 11(a), images in Figure 11(b) contain serious blurring effects or illumination changes, which are common in an outdoor challenging environment. VISO2 and ORB-VISO2, without any exception, fail to estimate the motion of the ground vehicle with this challenging dataset. Because of image blur, moving objects, or illumination change, even ORB-SLAM2 fails to locate the feature. However, Our-method1 and Our-method2 still perform well in the whole process with acceptable rotation and translation accumulative errors. Table 6 shows the overall RMSEs and end-point errors of the listed algorithms corresponding to outdoor data in Figure 11(b).Table 6
Overall RMSE and end-point error, corresponding to Figure11(b).
Error
VISO2
ORB-VISO2
ORB-SLAM2
Our-method1
Our-method2
Overall position RMSE (m)
X
X
X
54.961
4.438
Overall orientation RMSE (deg)
X
X
X
0.616
0.079
End-point position error (m)
X
X
X
48.875
1.964
End-point orientation error (deg)
X
X
X
2.464
0.297Figure12 shows that our proposed methods perform well at the underground parking lot (image samples are shown in Figures 12(a) and 12(b)). Compared with the other vision-based methods, only our proposed method can still locate the vehicle with poor light and dynamic objects while VISO2 and ORB-VISO2 all fail to locate, as shown in Figure 12(c). The repeated indoor parking experiments demonstrate that ORB-SLAM2 is not a stable work because of failure with exquisite lightening change.Figure 12
Vehicle positioning method results at the parking lot. Different with outdoor experiments, we regard the result of the laser method as ground truth. The map in the left bottom of figure is generated by laser point.
(a)
(b)
(c)


## 4.1. Performance Evaluation of ISVD

This paper applies the video quality assessment index proposed by the Video Quality Experts Group (VQEG) to test the performance of different blurred degree assessment algorithms. A few parameters are mentioned as follows. With these parameters, we can evaluate the performance of different algorithms.(1)
Outlier ratio (OR):(8)OR=n1n,where n1 is the number of outlier images and n is the total number of images. If the difference between scores calculated by the blurred degree assessment algorithm and actual reference score is larger than the threshold value, the image is regarded as an outlier image. The smaller the OR is, the better the consistency of the algorithm.
(2)
Root mean square error (RMSE):(9)RMSE=1n∑i=1nsi−spi2,where si and spi are the reference score and score calculated by the blurred degree assessment algorithm of the ith image, respectively. The smaller the RMSE, the better the predicted result of the assessment algorithm.
(3)
Mean absolute error (MAE):(10)MAE=1n∑i=1nsi−spi,where MAE sums the absolute value of the residual si−spi. In statistics, MAE is a measure of the difference between two continuous variables. The smaller MAE is, the better the value calculated by the assessment algorithm.
(4)
Spearman rank-order correlation coefficient (SROCC):(11)SROCC=1−6∑i=1nMOSri−MOSrpi2nn2−1,where MOSri (mean opinion scores) is the reference score of the ith image and MOSrpi is the value calculated by the assessment algorithm of the ith image. The bigger SROCC is, the better the performance of the assessment algorithm.
(5)
Pearson linear correlation coefficient (PCC):(12)PCC=∑i=1nMOSi−MOS¯MOSpi−MOSp¯∑i=1nMOSi−MOS¯2∑i=1nMOSpi−MOSp¯2,where MOSi is the actual reference score, MOS¯ is the average reference score of n images, MOSpi is the scores calculated by the assessment algorithm of the ith image, and MOSp¯ is the average calculated score of n images. The bigger PCC is, the better the consistency between the score calculated by the assessment algorithm and the actual reference score.With different blurred degree assessment algorithms (Marziliano, JNB, CPBD, SIGD, and ISVD), we calculate the blurred degree for 174 images in dataset LIVE [37]. The actual reference score of those images is known. The image quality parameters OR, RMSE, MAE, SROCC, and PCC are calculated with different assessment algorithms, as Table 1 shows. Obviously, the proposed ISVD performs more outstandingly than do other algorithms. The blurred degree calculated by ISVD remains monotonic over the full range of the data, as suggested in [36].Table 1
Image blur metrics of five different algorithms.
Algorithm
PCC
SROCC
RMSE
MAE
OR
Marziliano
0.8562
0.8672
8.8354
6.4382
0.1264
JNB
0.8498
0.8345
9.0117
6.9317
0.1322
CPBD
0.9257
0.9444
6.4662
4.7098
0.0632
SIGD
0.7596
0.6877
11.1176
9.0331
0.2759
ISVD
0.9405
0.9293
5.8069
4.4090
0.0230The high efficiency is critical to the real-time system. As shown in Table2, ISVD is less significantly efficient than SIGD, but it is sufficient for real-time VO.Table 2
The execution times of five different algorithms.
Algorithm
Marziliano
JNB
CPBD
SIGD
ISVD
Time (s)
0.5860
0.6156
0.7084
0.0226
0.0082


## 4.2. Performance Evaluation of Statistic Filter

We analyze SFFSD’s performance against the alternative ratio test (RT; if the ratio of the nearest correspondence distance to the second nearest correspondence distance is less than a threshold, a good match is declared), introduced in SIFT; on the aspect of precisionP=numberofcorrectmatches/totalnumberofmatches, recall R=numberofcorrectmatches/numberofcorrespondences, and F1-measure F1=2PR/P+R, which are critical to the matching performance. An evaluation is conducted on Oxford dataset VGG [37, 38] and TUM [39], as shown in Figure 7.Figure 7
Precision, recall, andF-measure score. The x-axis of VGG represents the image index, while the x-axis of TUM represents the relative rotation in degrees. SFFSD is black; RT is red.
(a)
(b)VGG has different photometric and geometric transformations, for example, Gaussian blur, lighting variation, scale, and rotation changes. The first frame in each set is designated as the reference. All other 5 frames, indexed with 0–4, from the same set are matched to the reference. TUM has video sequences with low texture and blurriness. Obviously, SFFSD has relatively higherF1 than the RT.Figure8 shows examples of outlier removal with SFFSD, where blue and red lines represent inliers and outliers, respectively. After the SFFSD, circle matching is employed to verify the matches (Figure 9).Figure 8
Outlier removing with SFFSD. An image pair and its putative correspondences. Blue and red lines represent inliers and outliers, respectively.Figure 9
The sample images in real outdoor environment and the circle matching described in Section4.2.
(a)
Regular scene
(b)
Pedestrian
(c)
Car
(d)
High exposure
(e)
Surface gathered water
(f)
Sharp turn, motion blur


## 4.3. Experiments and Comparisons

Experiments have been conducted on the open benchmark dataset KITTI [19] and the real outdoor and indoor datasets. In consideration of feature detection and matching efficiency≥efficiency, ORB [37] is regarded as a better choice. The comparison with other detectors and descriptors, like SIFT, SURF [40], KAZE [41], and AKAZE [42], has been carried out in [41]. To emphasize the excellent performance, we employ two basic VO algorithms, libviso2 (VISO2) [43] and ORB-SLAM2 [3], as comparisons in our experiments. The feature detector and descriptor in VISO2 are replaced with ORB, named ORB-VISO2. Our method is classified into two types, that is, method without loop closure and bundle adjustment based on VISO2 (Our-method1) and method with loop closure and bundle adjustment based on ORB-SLAM2 (Our-method2). No matter what the algorithms Our-method1 and Our-method2 are based on, the major difference is our contribution work of image antiblurring and statistic filter of feature space displacement. Table 3 shows the comparison of processing times among VIOS2, ORB-VISO2, ORB-SLAM2, Our-method1, and Our-method2. The spending time of ISVD and SFFSD is within 13 ms, which has a negligible effect on the total execution time.Table 3
The average processing time of one image frame (ms).
Method
VISO2
ORB-VISO2
ORB-SLAM2
Our-method1
Our-method2
Time (ms)
17.833
16.130
62.002
21.676
67.519


### 4.3.1. Experiments on Benchmark Datasets

The first experiment is conducted≥conducted on the KITTI benchmark dataset. The KITTI dataset consists of 22 stereo sequences, and we randomly take sequences 02 and 05 as comparison datasets. The trajectories of different algorithms and ground truth are drawn on the same figure.From Figure10, we can see that our method is closer to the ground truth than the art-of-the-state method. Regardless of the rotation error or translation error, our method performs better. Compared with other VO algorithms, the absolute trajectory errors of sequence 00–10 are shown in Table 4. In nonloop closure and nonbundle adjustment methods, Our-method1 is better than VISO2 and ORB-VISO2. In sequences “KITTI 01” and “KITTI 08,” ORB-VISO2 fails with fast motion on the highway or illumination changing. In loop closure and bundle adjustment methods, Our-method2 is better than ORB-SLAM2. The longer the trajectory is, the more superiority Our-method2 has.Figure 10
Randomly taking sequences 02 and 05 on the KITTI dataset as comparison datasets. Motion estimation on sequences 02 and 05 with ground truth, VISO2, ORB-VISO2, ORB-SLAM2, Our-method1, and Our-method2. The left column shows the trajectory, and the right column shows the translation and rotation error. (a) Comparison result on KITTI 02. (b) Comparison result on KITTI 05.
(a)
(b)Table 4
Absolute trajectory error RMSE (m).
Sequence
Environment
Distances
VISO2
ORB-VISO2
ORB-SLAM2
Our-method1
Our-method2
KITTI 00
Urban
3714 m
30.133
29.430
1.328
21.016
0.673
KITTI 01
Highway
4268 m
51.430
X
10.936
45.595
4.391
KITTI 02
Urban + country
5075 m
36.441
26.831
5.534
23.185
5.412
KITTI 03
Country
563 m
2.570
2.339
0.619
2.146
0.309
KITTI 04
Country
397 m
1.483
1.3230
0.223
1.190
0.114
KITTI 05
Urban
2223 m
13.653
12.287
0.846
11.740
0.337
KITTI 06
Urban
1239 m
4.318
3.917
0.758
3.757
0.302
KITTI 07
Urban
695 m
5.561
5.017
0.554
4.800
0.221
KITTI 08
Urban + country
3225 m
20.289
X
3.660
17.464
1.470
KITTI 09
Urban + country
1717 m
19.581
16.298
3.427
15.986
1.373
KITTI 10
Urban + country
919 m
4.367
3.812
1.165
3.593
0.468


### 4.3.2. Experiments in Outdoor and Indoor Environment

In this section, to verify the stability and scalability, we also carry out the experiments in real outdoor and indoor environments. The data are captured in a considerably cluttered environment on campus, which contains challenging scenes like dynamic objects, illumination changing, surface gathered water, and motion blur. The image samples are shown in Figure9.The trajectories recovered using the proposed method and other methods are shown in Figure11(a). The trajectory was approximately 1.45 km and lasted 236.4 s. We regard the trajectory of DGPS as the ground truth. VISO2 and ORB-VISO2 failed to locate with the instability factor in an outdoor environment. It can be found that Our-method1 is not reliable with the error accumulation. ORB-SLAM2 and Our-method2 perform well with loop closure and bundle adjustment. Obviously, Our-method2 has less drift than ORB-SLAM2 has. As shown in Table 5, the proposed method outperforms other methods in terms of overall RMSE and end-point error.Figure 11
Trajectory estimated using the proposed method and DGPS trajectory. The camera frequency was 30 Hz. (a) The trajectory was approximately 2.76 km and lasted 476.4 s. (b) The trajectory was approximately 1.17 km and lasted 109.3 s.
(a)
(b)Table 5
Overall RMSE and end-point error, corresponding to Figure11(a).
Error
VISO2
ORB-VISO2
ORB-SLAM2
Our-method1
Our-method2
Overall position RMSE (m)
X
X
9.006
34.961
5.750
Overall orientation RMSE (deg)
X
X
0.108
0.489
0.060
End-point position error (m)
X
X
2.927
87.788
1.175
End-point orientation error (deg)
X
X
0.434
2.506
0.358To further support the opinions proposed in this paper, we conduct another experiment to compare the performance of different VO and SLAM kind algorithms. The trajectory in Figure11(b) is 1.17 km and lasted 109.3 s. The average speed is about 38.5 km/h. Unlike the dataset in Figure 11(a), images in Figure 11(b) contain serious blurring effects or illumination changes, which are common in an outdoor challenging environment. VISO2 and ORB-VISO2, without any exception, fail to estimate the motion of the ground vehicle with this challenging dataset. Because of image blur, moving objects, or illumination change, even ORB-SLAM2 fails to locate the feature. However, Our-method1 and Our-method2 still perform well in the whole process with acceptable rotation and translation accumulative errors. Table 6 shows the overall RMSEs and end-point errors of the listed algorithms corresponding to outdoor data in Figure 11(b).Table 6
Overall RMSE and end-point error, corresponding to Figure11(b).
Error
VISO2
ORB-VISO2
ORB-SLAM2
Our-method1
Our-method2
Overall position RMSE (m)
X
X
X
54.961
4.438
Overall orientation RMSE (deg)
X
X
X
0.616
0.079
End-point position error (m)
X
X
X
48.875
1.964
End-point orientation error (deg)
X
X
X
2.464
0.297Figure12 shows that our proposed methods perform well at the underground parking lot (image samples are shown in Figures 12(a) and 12(b)). Compared with the other vision-based methods, only our proposed method can still locate the vehicle with poor light and dynamic objects while VISO2 and ORB-VISO2 all fail to locate, as shown in Figure 12(c). The repeated indoor parking experiments demonstrate that ORB-SLAM2 is not a stable work because of failure with exquisite lightening change.Figure 12
Vehicle positioning method results at the parking lot. Different with outdoor experiments, we regard the result of the laser method as ground truth. The map in the left bottom of figure is generated by laser point.
(a)
(b)
(c)


## 4.3.1. Experiments on Benchmark Datasets

The first experiment is conducted≥conducted on the KITTI benchmark dataset. The KITTI dataset consists of 22 stereo sequences, and we randomly take sequences 02 and 05 as comparison datasets. The trajectories of different algorithms and ground truth are drawn on the same figure.From Figure10, we can see that our method is closer to the ground truth than the art-of-the-state method. Regardless of the rotation error or translation error, our method performs better. Compared with other VO algorithms, the absolute trajectory errors of sequence 00–10 are shown in Table 4. In nonloop closure and nonbundle adjustment methods, Our-method1 is better than VISO2 and ORB-VISO2. In sequences “KITTI 01” and “KITTI 08,” ORB-VISO2 fails with fast motion on the highway or illumination changing. In loop closure and bundle adjustment methods, Our-method2 is better than ORB-SLAM2. The longer the trajectory is, the more superiority Our-method2 has.Figure 10
Randomly taking sequences 02 and 05 on the KITTI dataset as comparison datasets. Motion estimation on sequences 02 and 05 with ground truth, VISO2, ORB-VISO2, ORB-SLAM2, Our-method1, and Our-method2. The left column shows the trajectory, and the right column shows the translation and rotation error. (a) Comparison result on KITTI 02. (b) Comparison result on KITTI 05.
(a)
(b)Table 4
Absolute trajectory error RMSE (m).
Sequence
Environment
Distances
VISO2
ORB-VISO2
ORB-SLAM2
Our-method1
Our-method2
KITTI 00
Urban
3714 m
30.133
29.430
1.328
21.016
0.673
KITTI 01
Highway
4268 m
51.430
X
10.936
45.595
4.391
KITTI 02
Urban + country
5075 m
36.441
26.831
5.534
23.185
5.412
KITTI 03
Country
563 m
2.570
2.339
0.619
2.146
0.309
KITTI 04
Country
397 m
1.483
1.3230
0.223
1.190
0.114
KITTI 05
Urban
2223 m
13.653
12.287
0.846
11.740
0.337
KITTI 06
Urban
1239 m
4.318
3.917
0.758
3.757
0.302
KITTI 07
Urban
695 m
5.561
5.017
0.554
4.800
0.221
KITTI 08
Urban + country
3225 m
20.289
X
3.660
17.464
1.470
KITTI 09
Urban + country
1717 m
19.581
16.298
3.427
15.986
1.373
KITTI 10
Urban + country
919 m
4.367
3.812
1.165
3.593
0.468


## 4.3.2. Experiments in Outdoor and Indoor Environment

In this section, to verify the stability and scalability, we also carry out the experiments in real outdoor and indoor environments. The data are captured in a considerably cluttered environment on campus, which contains challenging scenes like dynamic objects, illumination changing, surface gathered water, and motion blur. The image samples are shown in Figure9.The trajectories recovered using the proposed method and other methods are shown in Figure11(a). The trajectory was approximately 1.45 km and lasted 236.4 s. We regard the trajectory of DGPS as the ground truth. VISO2 and ORB-VISO2 failed to locate with the instability factor in an outdoor environment. It can be found that Our-method1 is not reliable with the error accumulation. ORB-SLAM2 and Our-method2 perform well with loop closure and bundle adjustment. Obviously, Our-method2 has less drift than ORB-SLAM2 has. As shown in Table 5, the proposed method outperforms other methods in terms of overall RMSE and end-point error.Figure 11
Trajectory estimated using the proposed method and DGPS trajectory. The camera frequency was 30 Hz. (a) The trajectory was approximately 2.76 km and lasted 476.4 s. (b) The trajectory was approximately 1.17 km and lasted 109.3 s.
(a)
(b)Table 5
Overall RMSE and end-point error, corresponding to Figure11(a).
Error
VISO2
ORB-VISO2
ORB-SLAM2
Our-method1
Our-method2
Overall position RMSE (m)
X
X
9.006
34.961
5.750
Overall orientation RMSE (deg)
X
X
0.108
0.489
0.060
End-point position error (m)
X
X
2.927
87.788
1.175
End-point orientation error (deg)
X
X
0.434
2.506
0.358To further support the opinions proposed in this paper, we conduct another experiment to compare the performance of different VO and SLAM kind algorithms. The trajectory in Figure11(b) is 1.17 km and lasted 109.3 s. The average speed is about 38.5 km/h. Unlike the dataset in Figure 11(a), images in Figure 11(b) contain serious blurring effects or illumination changes, which are common in an outdoor challenging environment. VISO2 and ORB-VISO2, without any exception, fail to estimate the motion of the ground vehicle with this challenging dataset. Because of image blur, moving objects, or illumination change, even ORB-SLAM2 fails to locate the feature. However, Our-method1 and Our-method2 still perform well in the whole process with acceptable rotation and translation accumulative errors. Table 6 shows the overall RMSEs and end-point errors of the listed algorithms corresponding to outdoor data in Figure 11(b).Table 6
Overall RMSE and end-point error, corresponding to Figure11(b).
Error
VISO2
ORB-VISO2
ORB-SLAM2
Our-method1
Our-method2
Overall position RMSE (m)
X
X
X
54.961
4.438
Overall orientation RMSE (deg)
X
X
X
0.616
0.079
End-point position error (m)
X
X
X
48.875
1.964
End-point orientation error (deg)
X
X
X
2.464
0.297Figure12 shows that our proposed methods perform well at the underground parking lot (image samples are shown in Figures 12(a) and 12(b)). Compared with the other vision-based methods, only our proposed method can still locate the vehicle with poor light and dynamic objects while VISO2 and ORB-VISO2 all fail to locate, as shown in Figure 12(c). The repeated indoor parking experiments demonstrate that ORB-SLAM2 is not a stable work because of failure with exquisite lightening change.Figure 12
Vehicle positioning method results at the parking lot. Different with outdoor experiments, we regard the result of the laser method as ground truth. The map in the left bottom of figure is generated by laser point.
(a)
(b)
(c)


## 5. Conclusions

In this paper, an image antiblurring and statistic filter of feature space displacement-based visual odometry algorithm is proposed. The approach achieves a low drift error even for long paths. The impact of image blurring is mitigated by an antiblurring algorithm. A statistic filter of feature space displacement and circle matching are proposed and adopted to screen or prune potential matching features, so as to remove the outliers caused by the mismatching. An evaluation of benchmark dataset KITTI and the real outdoor dataset demonstrates that the proposed method is suitable to be used in robot ego-motion estimation, even for outdoor ground vehicles. Our further research will be devoted to establishing a dense SLAM system based on the proposed method because a dense map is vital to vehicle navigation application.



---

*Source: 2987819-2018-04-29.xml*