# A Front Water Recognition Method Based on Image Data for Off-Road Intelligent Vehicle

Authors: Haiwei Wang; Yibing Zhao
Journal: Journal of Advanced Transportation
 (2020)

---

## Abstract

Off-road intelligent vehicle is an important application about Internet of Vehicles technology used in the transportation field, and the front obstacle recognition method is the key technology for off-road intelligent vehicle. In this paper, based on smart data aggregation inspired paradigm of IoT applications, we mainly study perception technology in vehicle networking by using image data and one symmetrical speeded-up robust features detector (SURF). By considering symmetry and image data aggregation, we found that data aggregation had the ability of providing global information for Internet of Vehicles systems. After we have built the experiment platform, the experiment results showed that this method is faster than Scale-Invariant Feature Transform algorithm in this case, which can satisfy the water detection accuracy and the real-time requirement. So, this method is effective for the water images detection with great symmetry to off-road intelligent vehicle, and it also gives a useful reference about environment perception technology and smart data aggregation inspired paradigm used in future Internet of Vehicles, intelligent vehicle, and traffic safety applications.


---

## Body

## 1. Introduction

The off-road intelligent vehicle has important value in the fields of national defense security, wild rescue, geological survey, etc. During the autonomous navigation of intelligent vehicle and safety driving assist system, the negative obstacles’ detection of middle distance affects the vehicle passing ability, and the water area is one of the negative obstacles. One of the most important duties in vision image processing is to recognize the correspondence between two images or two objects in one image. And its application includes object or gesture recognition, feature retrieve, environment perception and modeling, image restoration, object tracking, and the industrial inspection. In order to reliably fulfill these missions, it is very important to detect the feature points with high repeatability in two images, even in case of various lighting conditions and geometric transformations. Scale-Invariant Feature Transform (SIFT) is commonly adapted to represent these feature points [1]. The SIFT provides these advantages: one is being partially invariant to illumination changes and affine distortion, and the other is being invariant to scale and orientation. However, the shortcoming is its slow speed for the online processing as a result of different Gaussian (DOG) filters application. SURF with the capability of scale invariant was put forward by Bay et al. [2], and each pair of feature points can be represented by constructing the SURF detector. This method applies integral images for image convolutions, which could boost the computing speed and comparing rate of feature points. However, it is not able to detect symmetrical pairs of feature points. Actually, various symmetrical objects exist in natural scenes and many man-made bodies, such as mirror reflection, animals and fruits, furniture, vehicles, and buildings. This paper mainly studies off-road intelligent vehicle’s front water recognition and reflection analysis [3, 4].Water detection is the key important part of environment perception in automatic navigation vehicles or safety driving assist system. The accurate water detection and location in video image are necessary for enabling environment perception. Small water bodies with no reflection are relatively easier to detect. However, it is still great challenge to recognize some large water bodies with the terrain reflection. Actually, there exist some significant appearance variations in reflection color, size, and shape. In this research field, it mainly involves the following aspects, including traditional features, perceptual information, mathematical model, and Internet of Things (IoT). Many different approaches have been presented against these variation problems, which employ various features and learning methods to recognize the whole water bodies [5–10]. Most of related techniques employ the color information of sky reflection to detect waters. Lakshmi Ratan uses brightness and texture features to detect the water with no reflection area by employing stereo vision information [11–16]. Hidden Markov Model was used to distinguish vehicles on the road. However, the motion feature is not helpful for some still images. In related articles, various types of detectors or descriptors have been proposed [17–21]. Most of them adopt corners as the local feature since they are characterized by high gray or color gradient. Because they are variant to affined and scale transformations, both Harris and Kanade Lucas Tomasi corner detectors are not widely used. In order to settle this problem, interest points are detected by means of automatic scale selection. Hessian matrix is scale adapted and used to create one scale-invariant feature detector, which depends on Laplacian matrix to decide the scale [22, 23]. A rotation-invariant descriptor SIFT is proposed by approximating the Laplacian matrix of Gaussians. To improve matching efficiency, a principal component analysis (PCA) is applied, which was not as distinctive as SIFT and slowed feature computation speed.In recent years, the Internet of Things has led to the explosive development of data in almost each engineering field. At the same time, big data becoming has strong advances in modern information technology, and it has gained relevance in many areas of society. Therefore, the concept of smart data aggregation has been proposed, which is considered as a new strategy for implementing IoT big data ecosystems. Data aggregation refers to the selection, analysis, and classification of relevant data, and finally it obtained the results that people want. As a result, the Internet of Vehicles (IoV) is a typical representative. IoV is a huge network of vehicle position, speed and route, and image information. So, data aggregation has the ability of providing global information for IoV systems [24–27]. In this paper, based on smart data aggregation inspired paradigm of IoT applications, we mainly study perception technology in vehicle networking by using image data. And we detect some possible symmetrical points by adopting the symmetrical speeded-up robust features detector; then we determine the matching pairs and extract the feature points. After we have built the experiment platform, the experiment results showed that this method is faster than Scale-Invariant Feature Transform algorithm in this case, which can satisfy the water detection accuracy and the real-time requirement. Therefore, this research work is important to the off-road intelligent vehicle, future Internet of Things, and safety driving assist system applications.Here are the research topics of this article. Section2 briefly surveys several steps of feature point detection. Section 3 presents how to construct the new feature point descriptor. Section 4 provides experimental results of water detection. Section 5 has conclusions and related future work.


## 2. Feature Point Detector Based on Symmetrical Surf

### 2.1. Internet of Things with Big Data Aggregation for IoV Systems

Internet of Vehicles is a typical application in the Internet of Things used field. The IoT big data potentials to the development of IoV are especially relevant for physical vehicle applications, such as road safety, traffic efficiency, and automated driving. Even before the advent of the IoT concept, vehicles are at the center of Intelligent Transportation Systems (ITS). With the development of automobiles in recent years, intelligent vehicles have gradually become a research hotspot. Automotive intelligent technology has become the main direction of automotive technology progress. It has greatly improved the cars safety, and it is also changing people’s travel and driving habits. Automotive intelligent technology includes active safety technology, driving assistance technology, and driverless car technology. According to the classification of the environmental use, intelligent vehicles can be divided into urban intelligent vehicles and off-road intelligent vehicles. Actually, off-road intelligent vehicle is a part of IoV and ITS systems. Through the delivery of different data, such as image data in IoV, environmental data in ITS and driver behavior data, and IoT smart data aggregation, the useful information that people want can be directly given back to IoV system. That is to say, the data transmission of off-road intelligent vehicles is an inspired paradigm of IoT applications [2, 28–36].At present, most scholars mainly focus on new analytical method about IoV big data, information transmission and reception of vehicle technology for vehicle interconnection, and optimization of data aggregation methods. For the application in the IoV and ITS systems, it is mainly used for image data and environmental data collection. For the development of vehicle intelligent technology, it mainly focuses on the research of vehicle real-time status information. Figure1 is the smart data aggregation inspired paradigm of IoT applications. In this paper, we mainly consider the symmetry and image data aggregation for off-road intelligent vehicle.Figure 1
Smart data aggregation inspired paradigm of IoT applications.Based on urban sustainable development demand and new generation information technology application, Internet of Things became people’s future mode. With the urbanization deepening, the way to promote urban development should be changed by Internet of Things principle design. For example, with the rapid development of Internet of Things technology, especially the research and application of wireless sensor networks, Cooperative Vehicle Infrastructure System came into being. Intelligent Transportation and Traffic Safety, Intelligent Vehicle, and Assist Driving are the frontier directions in the fields of Internet of Things in the transportation and vehicle engineering in the world. They have become one of the key research disciplines vigorously promoted by all countries in the world and are considered to be effective in improving road traffic reliability and safety and reducing environmental pollution. At the same time, the Cooperative Vehicle Infrastructure System is based on wireless communication, sensor detection, and other technologies to obtain vehicle-road information, through vehicle-road information exchange and sharing, and to achieve intelligent collaboration and cooperation between vehicles and infrastructure, vehicles and vehicles, so as to optimize the use of system resources, improving road traffic safety and slowness. According to the different functions, the key technologies of driverless vehicles can be divided into three modules: environment perception module, planning decision module, and motion control module. The three parts cooperate with each other. The environmental perception module is responsible for understanding the external world and providing necessary information for planning decision-making path; the planning decision module sends motion instructions to the motion control module based on the perception information calculation; and the motion control module manipulates various control mechanisms of the vehicle according to the instructions issued by the planning decision-making module such as steering wheel, brake, and accelerator. At present, in order to realize the automatic driving of intelligent vehicle far away from structured road and accomplish related tasks, named as the off-road intelligent vehicle, the most challenging technical problem must be solved to realize the reliable perception of the environment. Obstacle detection and recognition is an important part of the field of environmental perception technology. Unmanned vehicles must understand the geometric characteristics of rocks, slopes, trees, reefs, pits, and other materials, as well as the nongeometric characteristics of vegetation, air turbidity, water, and mud, in order to cope freely in outdoor complex terrain environment.As a typical application scenario of Internet of Things technology, vehicle-road collaboration is the core technology of intelligent transportation and intelligent vehicles. Vehicle-road collaboration is to adopt advanced wireless communication and new generation information technology, implement all-round dynamic real-time information exchange between vehicle and road, and carry out vehicle active safety control and road collaborative management on the basis of full-time dynamic traffic information collection and integration, so as to fully realize the effective collaboration between people and vehicles and ensure traffic safety, improve traffic efficiency, and form a safe, efficient, and environmentally friendly road traffic system.


### 2.2. Creating Integral Image

Integral image as an important conception of SURF algorithm is proposed by Viola and Jones [24, 25, 28], and it greatly improves the speed of feature point detection. The value SAT (x, y) of the point (x, y) in integral image represents the sum of pixels gray value, and these pixels lie in one rectangle wire from the upper-left corner of the original image to the point (x, y). It can be calculated by the following formula:(1)SATx,y=∑x′≤x,y′≤yIx′,y′.The integral image is obtained by traversing the whole image. Consider a rectangle with four corner pointsx+w−1,y+h−1, x+w−1,y−1, x−1,y+h−1, and x−1,y−1, as is shown in Figure 2. The sum of pixel gray value is calculated by the following formula:(2)RecSumr=SATx+w−1,y+h−1+SATx−1,y−1−SATx−1,y+h−1−SATx+w−1,y−1.Figure 2
Area computation using integral images.


### 2.3. Creating Hessian Matrix

In order to get an integral image, SURF employs a Hessian matrix approximation. The Hessian matrix can be used to describe the local curvature by calculating the second order partial derivatives. Since an integral image is employed to approximate the second order Gaussian derivatives, SURF is more efficient to extract interest points. Suppose that two variables contribute to a continuous function and the function value of pointx,yis denoted by Fx,y. H represents the Hessian matrix calculated by the following formula:(3)HFx,y=∂2F∂x2∂2F∂x∂y∂2F∂x∂y∂2F∂y2.The determinant of matrix H is calculated by the following formula:(4)detH=∂2F∂x2∂2F∂y2−∂2F∂x∂y2.It is not difficult to apply this theory on an image rather than a continuous function. Firstly, the function valueFx,y is replaced by the image pixel value Ix,y. Next step is to take the second order standard Gaussian function as the filter and then to compute partial derivatives by using convolution kernel. As a result, each element in Hessian matrix can be computed by constructing convolution kernels in x, y, and x-y direction. Hessian matrix H is expressed by using a function, including both space X=x,y and scale σ based on the following formula [2, 28]:(5)HX,σ=LxxX,σLxyX,σLyxX,σLyyX,σ.Here, LxxX,σ refers to the convolution of the second order Gaussian derivative ∂2/∂x2gσ, similar expression for Lyy and Lxy. Convolution mask is used in the actual calculation process as shown in Figure 3.Figure 3
The second order Gaussian derivative in different direction.Bay proposed a box filter to simplify Laplacian matrix of Gaussians, and representations of the respective kernels are used.LyyX,σ and LxyX,σ are approximated by using box filters [2]. We refer to these as Dyy and Dxyas shown in Figure 4.Figure 4
Approximations ofLyyX,σ and LxyX,σ by using box filters.After that, the discrimination of Hessian matrix can be defined by the following formula:(6)detHapprox=DxxDyy−ωDxy2,where Dxx,Dyy,Dxy refer to the results of convolution operation between the box filter and the image Ix,y and XF is Fresenius norm. Coefficient ω is adopted to balance the Hessian determinant and reduce the error between the approximation and the real values. Interest points can be extracted after finding local maxima of this function in all the scale spaces in an image.


### 2.4. Constructing the Scale Space

When constructing a scale space, subsequent layers are repeatedly smoothed by using Gaussian filter and image size is varied. However, SURF approach just varies the filter size and the original image is not changed [37]. In [28], the author proposed the SURF and used the SURF algorithm to detect vehicle symmetry. For our paper, we used the SURF algorithm to detect the front water symmetry. The difference between the SURF scale space and traditional structure is illustrated in Figure 5.Figure 5
The traditional and SURF approach of constructing scale space.The scale space is separated into many octaves composed of a series of different scale layers. When constructing larger filters, it is most important to define scale increasing rules, and the factor must be taken into consideration. The lobe size is one-third of the filter length for approximated filters. Since only one central pixel is needed, the dimensions must be increased equally around this location. Between consecutive filters, the minimum of lobe size just reaches 2 and the smallest step size is 6, as is shown in Figure6.Figure 6
The filter structure increased in size by a minimum of 6.Since the smallest step size is 6 in the first octave, the size of filters corresponds to9×9,15×15,21×21,27×27. For other octaves, Bay proposes the fact that filter size is arithmetic series with the common difference d=6×2n−1 and the common difference d grows with the increase of octave sequence number. For example, if d=12 in the second octave, then d=24 in the third octave; to use an analogy, the SURF Filter Pyramid is finally constructed. Figure 7 illustrates the filter side length for different octaves.Figure 7
The filter side length of different octaves.


### 2.5. Feature Point Localization

After Hessian matrix processing, feature points can be localized according to the determinant of Hessian matrix. Firstly, a threshold is set, then to remove all the values below the predetermined threshold. Only the stronger values are taken as interest points. The nonmaximal suppression is used to get a series of interest point candidates. Each pixel is compared to other 26 points in its neighborhood, which consists of 8 points in native scale space, 9 in upper octave, and 9 in lower octave. The nonmaximal suppression step is illustrated in Figure8.Figure 8
The pixel markedX chosen as a maxima and 26 points in its neighborhood.


## 2.1. Internet of Things with Big Data Aggregation for IoV Systems

Internet of Vehicles is a typical application in the Internet of Things used field. The IoT big data potentials to the development of IoV are especially relevant for physical vehicle applications, such as road safety, traffic efficiency, and automated driving. Even before the advent of the IoT concept, vehicles are at the center of Intelligent Transportation Systems (ITS). With the development of automobiles in recent years, intelligent vehicles have gradually become a research hotspot. Automotive intelligent technology has become the main direction of automotive technology progress. It has greatly improved the cars safety, and it is also changing people’s travel and driving habits. Automotive intelligent technology includes active safety technology, driving assistance technology, and driverless car technology. According to the classification of the environmental use, intelligent vehicles can be divided into urban intelligent vehicles and off-road intelligent vehicles. Actually, off-road intelligent vehicle is a part of IoV and ITS systems. Through the delivery of different data, such as image data in IoV, environmental data in ITS and driver behavior data, and IoT smart data aggregation, the useful information that people want can be directly given back to IoV system. That is to say, the data transmission of off-road intelligent vehicles is an inspired paradigm of IoT applications [2, 28–36].At present, most scholars mainly focus on new analytical method about IoV big data, information transmission and reception of vehicle technology for vehicle interconnection, and optimization of data aggregation methods. For the application in the IoV and ITS systems, it is mainly used for image data and environmental data collection. For the development of vehicle intelligent technology, it mainly focuses on the research of vehicle real-time status information. Figure1 is the smart data aggregation inspired paradigm of IoT applications. In this paper, we mainly consider the symmetry and image data aggregation for off-road intelligent vehicle.Figure 1
Smart data aggregation inspired paradigm of IoT applications.Based on urban sustainable development demand and new generation information technology application, Internet of Things became people’s future mode. With the urbanization deepening, the way to promote urban development should be changed by Internet of Things principle design. For example, with the rapid development of Internet of Things technology, especially the research and application of wireless sensor networks, Cooperative Vehicle Infrastructure System came into being. Intelligent Transportation and Traffic Safety, Intelligent Vehicle, and Assist Driving are the frontier directions in the fields of Internet of Things in the transportation and vehicle engineering in the world. They have become one of the key research disciplines vigorously promoted by all countries in the world and are considered to be effective in improving road traffic reliability and safety and reducing environmental pollution. At the same time, the Cooperative Vehicle Infrastructure System is based on wireless communication, sensor detection, and other technologies to obtain vehicle-road information, through vehicle-road information exchange and sharing, and to achieve intelligent collaboration and cooperation between vehicles and infrastructure, vehicles and vehicles, so as to optimize the use of system resources, improving road traffic safety and slowness. According to the different functions, the key technologies of driverless vehicles can be divided into three modules: environment perception module, planning decision module, and motion control module. The three parts cooperate with each other. The environmental perception module is responsible for understanding the external world and providing necessary information for planning decision-making path; the planning decision module sends motion instructions to the motion control module based on the perception information calculation; and the motion control module manipulates various control mechanisms of the vehicle according to the instructions issued by the planning decision-making module such as steering wheel, brake, and accelerator. At present, in order to realize the automatic driving of intelligent vehicle far away from structured road and accomplish related tasks, named as the off-road intelligent vehicle, the most challenging technical problem must be solved to realize the reliable perception of the environment. Obstacle detection and recognition is an important part of the field of environmental perception technology. Unmanned vehicles must understand the geometric characteristics of rocks, slopes, trees, reefs, pits, and other materials, as well as the nongeometric characteristics of vegetation, air turbidity, water, and mud, in order to cope freely in outdoor complex terrain environment.As a typical application scenario of Internet of Things technology, vehicle-road collaboration is the core technology of intelligent transportation and intelligent vehicles. Vehicle-road collaboration is to adopt advanced wireless communication and new generation information technology, implement all-round dynamic real-time information exchange between vehicle and road, and carry out vehicle active safety control and road collaborative management on the basis of full-time dynamic traffic information collection and integration, so as to fully realize the effective collaboration between people and vehicles and ensure traffic safety, improve traffic efficiency, and form a safe, efficient, and environmentally friendly road traffic system.


## 2.2. Creating Integral Image

Integral image as an important conception of SURF algorithm is proposed by Viola and Jones [24, 25, 28], and it greatly improves the speed of feature point detection. The value SAT (x, y) of the point (x, y) in integral image represents the sum of pixels gray value, and these pixels lie in one rectangle wire from the upper-left corner of the original image to the point (x, y). It can be calculated by the following formula:(1)SATx,y=∑x′≤x,y′≤yIx′,y′.The integral image is obtained by traversing the whole image. Consider a rectangle with four corner pointsx+w−1,y+h−1, x+w−1,y−1, x−1,y+h−1, and x−1,y−1, as is shown in Figure 2. The sum of pixel gray value is calculated by the following formula:(2)RecSumr=SATx+w−1,y+h−1+SATx−1,y−1−SATx−1,y+h−1−SATx+w−1,y−1.Figure 2
Area computation using integral images.


## 2.3. Creating Hessian Matrix

In order to get an integral image, SURF employs a Hessian matrix approximation. The Hessian matrix can be used to describe the local curvature by calculating the second order partial derivatives. Since an integral image is employed to approximate the second order Gaussian derivatives, SURF is more efficient to extract interest points. Suppose that two variables contribute to a continuous function and the function value of pointx,yis denoted by Fx,y. H represents the Hessian matrix calculated by the following formula:(3)HFx,y=∂2F∂x2∂2F∂x∂y∂2F∂x∂y∂2F∂y2.The determinant of matrix H is calculated by the following formula:(4)detH=∂2F∂x2∂2F∂y2−∂2F∂x∂y2.It is not difficult to apply this theory on an image rather than a continuous function. Firstly, the function valueFx,y is replaced by the image pixel value Ix,y. Next step is to take the second order standard Gaussian function as the filter and then to compute partial derivatives by using convolution kernel. As a result, each element in Hessian matrix can be computed by constructing convolution kernels in x, y, and x-y direction. Hessian matrix H is expressed by using a function, including both space X=x,y and scale σ based on the following formula [2, 28]:(5)HX,σ=LxxX,σLxyX,σLyxX,σLyyX,σ.Here, LxxX,σ refers to the convolution of the second order Gaussian derivative ∂2/∂x2gσ, similar expression for Lyy and Lxy. Convolution mask is used in the actual calculation process as shown in Figure 3.Figure 3
The second order Gaussian derivative in different direction.Bay proposed a box filter to simplify Laplacian matrix of Gaussians, and representations of the respective kernels are used.LyyX,σ and LxyX,σ are approximated by using box filters [2]. We refer to these as Dyy and Dxyas shown in Figure 4.Figure 4
Approximations ofLyyX,σ and LxyX,σ by using box filters.After that, the discrimination of Hessian matrix can be defined by the following formula:(6)detHapprox=DxxDyy−ωDxy2,where Dxx,Dyy,Dxy refer to the results of convolution operation between the box filter and the image Ix,y and XF is Fresenius norm. Coefficient ω is adopted to balance the Hessian determinant and reduce the error between the approximation and the real values. Interest points can be extracted after finding local maxima of this function in all the scale spaces in an image.


## 2.4. Constructing the Scale Space

When constructing a scale space, subsequent layers are repeatedly smoothed by using Gaussian filter and image size is varied. However, SURF approach just varies the filter size and the original image is not changed [37]. In [28], the author proposed the SURF and used the SURF algorithm to detect vehicle symmetry. For our paper, we used the SURF algorithm to detect the front water symmetry. The difference between the SURF scale space and traditional structure is illustrated in Figure 5.Figure 5
The traditional and SURF approach of constructing scale space.The scale space is separated into many octaves composed of a series of different scale layers. When constructing larger filters, it is most important to define scale increasing rules, and the factor must be taken into consideration. The lobe size is one-third of the filter length for approximated filters. Since only one central pixel is needed, the dimensions must be increased equally around this location. Between consecutive filters, the minimum of lobe size just reaches 2 and the smallest step size is 6, as is shown in Figure6.Figure 6
The filter structure increased in size by a minimum of 6.Since the smallest step size is 6 in the first octave, the size of filters corresponds to9×9,15×15,21×21,27×27. For other octaves, Bay proposes the fact that filter size is arithmetic series with the common difference d=6×2n−1 and the common difference d grows with the increase of octave sequence number. For example, if d=12 in the second octave, then d=24 in the third octave; to use an analogy, the SURF Filter Pyramid is finally constructed. Figure 7 illustrates the filter side length for different octaves.Figure 7
The filter side length of different octaves.


## 2.5. Feature Point Localization

After Hessian matrix processing, feature points can be localized according to the determinant of Hessian matrix. Firstly, a threshold is set, then to remove all the values below the predetermined threshold. Only the stronger values are taken as interest points. The nonmaximal suppression is used to get a series of interest point candidates. Each pixel is compared to other 26 points in its neighborhood, which consists of 8 points in native scale space, 9 in upper octave, and 9 in lower octave. The nonmaximal suppression step is illustrated in Figure8.Figure 8
The pixel markedX chosen as a maxima and 26 points in its neighborhood.


## 3. Construction of Feature Point Descriptor

### 3.1. Preliminary Cluster Analysis and Computing the Dominant Orientation

At the same time, based on smart data aggregation inspired paradigm of IoT applications and the previous research fruits, the image data clustering analysis shows thatk points need to be selected as the initial clustering center in advance. On this basis, the distance from each sample to the cluster center is calculated, and the samples are classified into the nearest cluster center. For this reason, in order to improve the classification efficiency in the later stage, this text initially carries out clustering analysis of data. In particular, it is used to distinguish areas that are not intersected with each other. The working principle of clustering algorithms is shown in Figure 9.Figure 9
Working principle of clustering algorithms.


### 3.2. Computing the Dominant Orientation

Then, the dominant orientation of each feature point can achieve image rotation invariance. To determine the orientation,x-responses and y-responses can be computed by using 4σ Haar template, within a circle centered at the feature points, using 6σ for radius. Figure 10 illustrates the Haar wavelets. Then, the x-responses and y-responses are weighted by using Gaussian. The weighted responses of Haar wavelet can be represented in vector space. In order to select the dominant orientation, a sector is with an angle of π/3 and rotated around the origin. Each component response is added to generate new vector at each position. The dominant orientation is determined by the largest vector [38–41]. This procedure is illustrated in Figures 10 and 11.Figure 10
The blue sector representsx- and y-responses added to form one new vector.
(a)(b)(c)Figure 11
(a) Computing Haar wavelet responses. (b) and (c) The responses inx and y direction computed by using Haar wavelet filters with a 4 s width.


### 3.3. Creating Symmetrical SURF Descriptor

We need to construct a square window with the size20σ in order to extract the SURF descriptor. This window is regularly separated into 4×4 subregion, and in each subregion 2σ Haar wavelets are calculated by 25 sample points, which are regularly distributed. Thus, ∑dx,∑dx,∑dy,∑dy is computed. Here, dx and dy refer to x-wavelet responses and y-wavelet responses, respectively. Each vector length is 16×4=64.The whole process is shown in Figure 12.Figure 12
Process of SURF descriptor construction.The shortcoming of the original SURF dentition is not good at matching symmetrical points. To compensate for the deficiency, between symmetrical points, some relation of SURF descriptors needs to be derived. Hsieh puts forward the symmetrical SURF detector [37]; the method details can be found in [28]. And we used this method proposed by in [28] to match symmetrical points. Different from [28], we mainly research on the water detection by the symmetrical SURF detector. Figure 13 shows feature points detection results. Each pair of symmetrical points can be successfully matched.Figure 13
Symmetrical SURF points detection results.
(a)(b)(c)(d)(e)(f)


## 3.1. Preliminary Cluster Analysis and Computing the Dominant Orientation

At the same time, based on smart data aggregation inspired paradigm of IoT applications and the previous research fruits, the image data clustering analysis shows thatk points need to be selected as the initial clustering center in advance. On this basis, the distance from each sample to the cluster center is calculated, and the samples are classified into the nearest cluster center. For this reason, in order to improve the classification efficiency in the later stage, this text initially carries out clustering analysis of data. In particular, it is used to distinguish areas that are not intersected with each other. The working principle of clustering algorithms is shown in Figure 9.Figure 9
Working principle of clustering algorithms.


## 3.2. Computing the Dominant Orientation

Then, the dominant orientation of each feature point can achieve image rotation invariance. To determine the orientation,x-responses and y-responses can be computed by using 4σ Haar template, within a circle centered at the feature points, using 6σ for radius. Figure 10 illustrates the Haar wavelets. Then, the x-responses and y-responses are weighted by using Gaussian. The weighted responses of Haar wavelet can be represented in vector space. In order to select the dominant orientation, a sector is with an angle of π/3 and rotated around the origin. Each component response is added to generate new vector at each position. The dominant orientation is determined by the largest vector [38–41]. This procedure is illustrated in Figures 10 and 11.Figure 10
The blue sector representsx- and y-responses added to form one new vector.
(a)(b)(c)Figure 11
(a) Computing Haar wavelet responses. (b) and (c) The responses inx and y direction computed by using Haar wavelet filters with a 4 s width.


## 3.3. Creating Symmetrical SURF Descriptor

We need to construct a square window with the size20σ in order to extract the SURF descriptor. This window is regularly separated into 4×4 subregion, and in each subregion 2σ Haar wavelets are calculated by 25 sample points, which are regularly distributed. Thus, ∑dx,∑dx,∑dy,∑dy is computed. Here, dx and dy refer to x-wavelet responses and y-wavelet responses, respectively. Each vector length is 16×4=64.The whole process is shown in Figure 12.Figure 12
Process of SURF descriptor construction.The shortcoming of the original SURF dentition is not good at matching symmetrical points. To compensate for the deficiency, between symmetrical points, some relation of SURF descriptors needs to be derived. Hsieh puts forward the symmetrical SURF detector [37]; the method details can be found in [28]. And we used this method proposed by in [28] to match symmetrical points. Different from [28], we mainly research on the water detection by the symmetrical SURF detector. Figure 13 shows feature points detection results. Each pair of symmetrical points can be successfully matched.Figure 13
Symmetrical SURF points detection results.
(a)(b)(c)(d)(e)(f)


## 4. Experimental Results and Analysis

According to the research requirements, we set up the intelligent vehicle research platform, named as DLUT-IV. DLUT-IV is a fully intelligent ground vehicle with sensors for localization and mapping, real-time industrial personal computers, and drive-by-wire capability [29]. All of the computer and sensor systems and their communication protocols are illustrated in Figure 14, and the sensors have different communication protocols; in-vehicle communication is performed via CAN, Ethernet, RS232, and 1394 firewire. The vehicle is 1.5 m wide and 4.8 m long. It weighs approximately 1,450 kg and can accommodate 4 people. The experiment speed is 20 km/h. Details about this platform can be found in[35]. Figure 15 shows our experiment platform for intelligent vehicle with laser and vision system.Figure 14
Sensors, computer system, and communication system in the DLU-IV for Cooperative Vehicle Infrastructure System: a case for Internet of Things principle.Figure 15
Intelligent vehicle with laser and vision system.In order to verify the algorithm effectiveness, we preferred to use offline testing. According to the previous aggregation analysis, the search efficiency of Internet of Things application is greatly accelerated. To test the accuracy of symmetrical SURF descriptor, we choose some completely symmetrical butterfly specimens images and some water images to test. Figures16 and 17 illustrate the results of different images. The test results show that, in dealing with nearly completely symmetrical images, SURF can maintain a higher success matching rate. The results are shown as follows:(a)
The image data which have completely symmetrical features rarely exist in nature.(b)
Even in calm water conditions, the details of original object cannot be completely rendered in reflection, so the feature extraction, description, and matching of feature points become difficult.(c)
Although SURF descriptor has rotation invariance for image feature detection, rotation does not mean flipping. The local features of the flipping image have strong correlation with original image, and the arrangement of SURF descriptor does not completely express the correlation shown in Figure18. And the performance of these algorithms was shown in Table 1.Figure 16
Symmetrical SURF points matching results of water images.
(a)(b)(c)(d)(e)(f)Figure 17
Symmetrical SURF points matching results of butterfly specimens’ images.
(a)(b)With the deepening of our research work, we find that it is very difficult for off-road intelligent vehicles to rely solely on image information to detect front obstacle. At the same time, the water area symmetry in the whole image is not at the center of coordinates. So far, we have tried to identify the water area in the front vehicle after incorporating real-time laser data. As shown in Figure19, the flow chart of the algorithm has been preliminarily carried out. However, we find that it is very difficult to ensure the reliability and robustness of obstacle identification when the object is an obstacle in off-road environment only relying on a single feature. The method in this paper has some reference value, but it cannot identify vehicle waters completely and reliably. Based on data fusion, machine learning, and smart data aggregation, how to improve the recognition rate is still the focus of the next step research work.Figure 18
Flipping image (left) and rotation image (right).Table 1
Performance of three algorithms.
MethodTimeScaleRotationBlurIlluminationAffineSiftCommonBestBestCommonCommonGoodPCA-siftGoodGoodGoodBestGoodBestSurfBestCommonCommonGoodBestGoodFigure 19
Preliminary front water recognition for off-road intelligent vehicle. (a) CCD image. (b) Water detection based on CCD image. (c) Laser image. (d) Water detection based on laser image.
(a)(b)(c)(d)


## 5. Conclusions and Future Research

Off-road intelligent vehicle is an important topic of robotics in the field of Internet of Vehicles, safety driving assist system, and vehicle active safety. Water hazards detection is one of most difficult and key problems for vehicle autonomous navigation in off-road environment. It is a great challenge to recognize some large water bodies with the terrain features reflection. The water reflection generally is divided into two cases: (1) terrain feature reflection and (2) sky reflection, nonreflection area in this paper. This paper just analyzes the features of water reflection area based on smart data aggregation inspired paradigm of IoT applications, and the real terrain and its reflection area in water are symmetrical. For the reflection area, it is suggested to extract the interest points and construct SURF descriptor. Then, feature points can be matched between the real image and its reflection image in water. After this, the waterline could be detected based on the symmetrical property.It is efficient for real-time water detection and exactly accurate to get water edges even with the reflection in waters. The property of symmetrical reflection image ultimately complies with that of the mirror reflection. It is greatly important to apply SURF for matching pairs and then to extract the feature points. For traditional methods, the symmetrical SURF detector could improve its efficiency. It can satisfy the high accuracy of water detection and the real-time requirement and it is faster than Scale-Invariant Feature Transform (SIFT) algorithm. Experiment results demonstrate that this method is effective for the water images with great symmetry. Under normal illumination conditions, this method has well adaptability for intelligent vehicles in the off-road environment.In this paper, if we just adopt the symmetrical SURF and only image data, this also has some defects as described in Section4. This paper is still a preliminary study work, and it is a tentative study by considering symmetry and image data aggregation. In the future work, the first target is to improve the feasibility of our new method to overcome the above problems. Besides, vehicle to vehicle communication and vehicle platoon control will be our research focus in the future. In our future work, there are three models of vehicle platoon control, overtaking model, lane-changing model, and car-following model. Furthermore, under off-road environment, vehicle and vehicle platoon stability control is also a very interesting research content. Therefore, with the continuous development of low-cost multisensor technology, the data information of laser, vision, and radar needs to be fused continuously. Before the fusion, data need to be aggregated and analyzed to facilitate the customer application of vehicle networking. At the same time, in the redundancy design of intelligent fleet, clustering analysis of data is also an important requirement to meet the real-time adaptability. Finally, the water detection research in various light conditions is still hard and we will continue to do more research work in this field. So, this work gives a reference to intelligent vehicle autonomous navigation’s perception under cross-country environment, and this is the first and tentative step work by considering symmetry and image data aggregation. It also gives a reference to safety driving assist system and vehicle active safety [42, 43].



---

*Source: 2949170-2020-09-03.xml*