# Roads and Intersections Extraction from High-Resolution Remote Sensing Imagery Based on Tensor Voting under Big Data Environment

Authors: Ke Sun; Junping Zhang; Yingying Zhang
Journal: Wireless Communications and Mobile Computing
 (2019)

---

## Abstract

Currently, big data is a new and hot object of research. In particular, the development of the Internet of things (IoT) results in a sharp increase in data. Enormous amounts of networking sensors are constantly collecting and transmitting data for storage and processing in the cloud including remote sensing data, environmental data, geographical data, etc. Road information extraction from remote sensing data is mainly researched in this paper. Roads are typical man-made objects. Extracting roads from remote sensing imagery has great significance in various applications such as GIS data updating, urban planning, navigation, and military. In this paper a multistage and multifeature method to extract roads and detect road intersections from high-resolution remotely sensed imagery based on tensor voting is presented. Firstly, the input remote sensing image is segmented into two groups including road candidate regions and nonroad regions using template matching; then we can obtain preliminary road map. Secondly, nonroad regions are removed by geometric characteristics of road (large area and long strip). Thirdly, tensor voting is used to overcome the broken roads and discontinuities caused by the different disturbing factors and then delete the nonroad areas that are mixed into the road areas due to mis-segmentation, improving the completeness of extracted roads. And then, all the road intersections are extracted by using tensor voting. The experiments are conducted on different remote sensing images to test the effectiveness of our method. The experimental results show that our method can get more complete and accurate extracted results than the state-of-the-art methods.


---

## Body

## 1. Introduction

Recently, with the continuous development of Internet of things (IoT) big data, mobile Internet, grid computing, cloud computing, and other new technologies, system integration becomes more complex. When processing information and data, it encounters many challenges such as data storage and management, efficient processing of massive data, structured and unstructured data fusion and analysis, and multitype data visualization. In particular, remote sensing technologies are promoted quickly; the spectral, spatial, radiative, and temporal resolution of remote sensing data are becoming higher and higher, which contain abundant data. Remote sensing data has the distinctive big data characteristics such as large capacity, high efficiency, multitype, and high value. All the above trends indicate that remote sensing has entered into a big data era. Based on the aerospace science and technologies, an integrated space-air information network has been formed, which provides ultrahigh dimensional and frequency earth observation data. Remote sensing big data is a revolution of traditional data processing and information extraction methods [1]. The traditional processing methods cannot meet the precision and efficiency requirements of remote sensing big data [2, 3].How to extract information of interest from remote sensing images quickly and efficiently has always been a research hotspot in the field of remote sensing data processing. The acquisition of road attribute information is an important part of it [4]. In high-resolution remote sensing images, many narrow roads that are difficult to discern on the low-resolution image can be distinguished. However, nonobject noise also increases. Currently, there are two main problems for extracting road information from high-resolution remote sensing images: (1) straight line inside road has the same direction with road (such as road boundary). (2) Straight line has the different directions with road (such as zebra crossing) [5]. If the resolution is higher, the buildings are clearer. The top or the shadow of the buildings often forms road parallel lines. What is more, there are many cars and trees. These factors would result in difficult problems of road extraction.Due to the complexity and diversity of roads in the real world, existing road extraction methods partially solves the problems at some stages (such as filtering [8], segmentation [9] in preprocessing stage, the extraction stage [10], split [11], and merge [12] in postprocessing stage). The methods of road extraction are reviewed in detail in [13, 14], including seed point-based [15], knowledge based method [16], and dynamic programming [17]. In addition, some researchers divided road extraction methods into automatic and semiautomatic methods. Wanget al. [18] proposed an object-oriented method for extracting roads. They selected some spectral features and textures parameters and used object-oriented methods to extract roads from the input images. Saatiet al. [19] proposed an automatic method to extract road centerlines from SAR images. They extracted three features of the road, defining the road features by the backscattering coefficient of each pixel and the adjacent pixels of the SAR images. The feature extracted by the fusion was then used to detect the road regions by using a fuzzy inference system. Weiet al. [20] proposed an end-to-end road centerline extraction method by learning a confidence graph. They extracted road centerline directly from images, rather than obtaining the road centerline by thinning the road segments. Guptaet al. [21] developed an automatic method to extract roads by using fuzzy, genetic algorithm, and mathematical morphology.Recently, tensor voting algorithm is widely used for feature extraction, especially in remote sensing images. Miaoet al. [22] extracted road intersection areas by tensor voting, and the roads were decomposed to isolated parts at the detected junction areas; then they were able to extract the centerlines for each individual section of the road. Zhanget al. [23] used tensor voting to extract roads and road intersections from remote sensing images. Ishidaet al. [24] proposed two voting schemes to estimate and classify the position accurately. The first was based on geometric feature extraction of multiframe sparse tensor voting, and the second was contour localization using the resulting tensor field. Zhuet al. [25] presented a tensor voting method for image denoising; they considered that the calculation of voting field was the key step of image denoising based on tensor voting, and it was a robust feature extraction method.For the last few years, the rapid development of the earth observation capability and the intelligent computing technology [26, 27] has provided opportunities for the advancement and even revolution of remote sensing information technology. Remote sensing information technology is gradually entering the era of remotely sensed big data [28] era. This will inevitably put forward higher requirements for automatic analysis and mining of big data. Deep learning [29–31] has been widely used to extract information from remote sensing images, and its accuracy has exceeded the accuracy of manual recognition. The great success of deep learning in the field of computer vision [32–34] provides an important opportunity for big data to extract information intelligence from remote sensing imagery.At present, many researchers have proposed many methods for road extraction from different perspectives for different remotely sensed imagery and have made great progress. Due to the variety of road forms and the complexity of surrounding environment in reality, most of the existing methods extract roads from specific remote sensing images and road information of specific areas. There is no extraction method that can be applied to all remote sensing images or all types of roads. This paper attempts to divide the road extraction method into different steps and clarify the specific tasks of each step. We propose a multistage and multifeature method based on tensor voting, which includes template matching, geometric feature of road and tensor voting to extract roads, and road intersections in high-resolution remotely sensed imagery. The structure of this paper is as follows. Section2 introduces the proposed road and intersection extraction method. The experiments and analysis are shown in Section 3. Conclusions are given in Section 4.


## 2. Methodology

The proposed method for extracting roads and road intersections is based on multistage and multifeature from high-resolution remotely sensed imagery. The method can extract pure road regions and accurately detect all road intersections from the input image. It consists of three stages. We first segment the input images and get the candidate road regions and then eliminate non-road areas by using road geometric features and get initial road maps and finally purify and smooth the initial road areas and extract road intersections. The overall strategy of the proposed method is shown in Figure1.Figure 1
Overall strategy of the proposed method.


### 2.1. Image Segmentation

The purpose of this step is to segment the input image and obtain preliminary road regions. The SUSAN (Smallest Univalue Segment Assimilating Nucleus) [35] algorithm is adopted to segment images. SUSAN algorithm is the representative of template matching, which was proposed by Smith and Brady. SUSAN algorithm moves the template on image, whether the template center is an edge point determined by whether the matching degree of it reaches the threshold. Because the SUSAN algorithm is based on the grayscale comparison of the pixels in the neighborhood and does not need to calculate the gradient, the interference range of the noise is obviously smaller than the edge detection based on the gradient.The SUSAN algorithm uses a circular template to move over the image [36]. If the difference between the gray level of the pixel in the template and that of the central pixel is lower than the given threshold [37], then it is considered that the point has a similar gray level with the central pixel of the template [38]. A region consisting of pixels satisfying such a condition is called an USAN (Univalue Segment Assimilating Nucleus). In Figure 2,a andb are completely located in the foreground and background of the image, respectively. The area of USAN is the largest. In addition,c,d,e are moving closer to the edge, while the area of USAN is approaching to the minimum. The results show that the USAN values of edge pixels are less than or equal to half of the maximum value of the period. The edge points can be detected.Figure 2
Principle of SUSAN feature detection.The steps of detecting image edge information by using SUSAN algorithm are as follows:(1) Find out the maximum and minimum gray valueImax and Imin from the image, and calculate the best detection threshold: (1)t=Imax-Imin10(2) Traverse the whole image and check location characteristics of each pixel according to the following formula:(2)cr,r0=1Ir-Ir0≤t0Ir-Ir0>twhere c(r,r0) is a discriminant function. If its value is 1, then the pixel is located in the USAN region; I(r0) is the gray value of the central pixel of the template; I(r) is the gray value of any other pixel in the template.The USAN area statistics for each pixel are as follows:(3)nr0=∑r∈Dr0cr,r0where D(r0) is a circular template region and r0 is the center of the circle.(3) The threshold valueg is set. When n(r0)<g, the detected pixel position r0 is assumed to be at the edge position of the image.(4) Traverse the whole image to detect the complete edge information.According to the characteristics of the road in the remote sensing image, in the obtained edge results map, the roads are located in the nonedge homogeneous regions. The results of edge extraction map are segmented using a threshold, and road candidate areas are obtained.


### 2.2. Road Information Extraction

After the previous algorithm is implemented, the image can be segmented into two categories, homogeneous regions and edges. Road segments are in homogeneous regions. But buildings or bare soil segments are also mixed into road areas. The geometric characteristics of road will be used to find out the potential road regions and eliminate other nonroad segments.According to the geometric features, roads do not have small areas, and the length is much larger than the width. So roads are obviously displayed as narrow, long, linear feature [39]. Through the identification of the big-area regions and linear features, it is easy to remove the small areas and nonroad regions.In this paper, the result of image segmentation is processed from the two aspects aiming to eliminate the nonroad regions.(1) Compute the area of each independent region in the result of segmentation, and remove the regions whose value is less than the threshold. It can be expressed as(4)Region=roadregionRarea>Rthrenon-roadreginotherwisewhere Rarea is the area of region and Rthre is the threshold.(2) The ratio of length to width of the minimum external rectangle in each region is calculated in the result of segmentation. If the ratio of length to width is greater than the set threshold in this region, then the region can be marked as the road area. The ratio of length to width of the region is defined as(5)R=LminWminwhere Lmin is the length of the minimum external rectangle of the region and Wmin is the width of the rectangle.When road is bent and the linear features cannot be well described, there will be errors in calculating the aspect ratio by using the above formula. The formula needs to be modified to overcome this limitation. We take the total number of pixels in the detection region as the area of the external rectangle and create a new rectangle with the diagonal line of the smallest external rectangle as the long edge. Therefore,(6)W=nLwhere W is the width of the new rectangle; n is the number of pixels of the region; and L is the length of the new rectangle; the value of L can be calculated by(7)L=Lmin2+Wmin2Therefore, in practical application, the aspect ratio of the region can be calculated by(8)Rmin=LW=Lmin2+Wmin2nThe results of removing nonroad regions using road’s shape features are shown in Figure3. The test image is a QuickBird image; the resolution is 0.61m/pixel.Figure 3
Results of removing nonroad regions using shape features on QuickBird image. (a) Input image. (b) Results of preliminary road extraction. (c) Results of removing small area regions. (d) Results of eliminating nonlinear features.
(a) (b) (c) (d)


### 2.3. Tensor Voting

After previous extraction, the main road regions have been extracted. However, there are still some large holes, gaps, and many other nonroad regions in the results of detected roads. Generally, mathematical morphology is used to delete the isolated nonroad regions and connect the small gaps of the road. However, this method cannot fill the large holes in the results. It cannot connect the large gaps and completely delete the nonroad regions [40]. Therefore, this paper uses tensor voting [41–43] algorithm to solve the above problems.Tensor voting is a robust method for feature extraction. The main idea of tensor voting algorithm is that every point in the space collects the tensor information from other points in the neighborhood and encodes it as a new tensor to be used in the next voting. After the voting, it decomposes the new tensor. Thus, the salience map of various characteristics is obtained. It can be used to detect the geometric structures, which consists of two parts, the tensor representation of the data (tensor coding) and the nonlinear voting between tensors (tensor voting).Tensor voting can obtain the salient features of the image, so it is possible to detect the geometric features of the typical objects. Road has obvious elongated shape, and intersection has obvious ball shape. We can use tensor voting to extract roads and detect road intersections from remotely sensed images.Firstly, the second-order positive semidefinite symmetric tensor is used to represent the direction and significance of the pixels in the image. In a two-dimensional space, tensor can be decomposed into a linear combination of eigenvalues and eigenvectors:(9)T=λ1-λ2e1⃑e1⃑T+λ2e1⃑e1⃑T+e2⃑e2⃑Twhereλ1 and λ2 are nonnegative eigenvalues and λ1>λ2.  e1⃑  and  e2⃑ are the corresponding eigenvectors. e1⃑e1⃑T is stick tensor (representing curve characteristics), e1⃑ represents the direction of a curve, and λ1-λ2 is a significant index of the curve. e1⃑e1⃑T+e2⃑e2⃑T is ball tensor (representing node characteristics), and λ2 is a significant index of the node.Then tensor voting is executed. Firstly, it initializes the pixels to the ball tensor and vote; the voting field is shown in Figure4(a). Next, the initial direction is obtained by analyzing the number of votes received at each point, and the initial direction is assigned to the stick voting field as shown in Figure 4(b). Then voting in the stick field, the voting rules are as follows (see Figure 4(c)); in coordinate systemOxy, there are two tensors atO andP, respectively, whereO is the voting point,P is the receiving point, andNO andNP are their normal vectors, respectively.Figure 4
Tensor voting. (a) Ball voting field. (b) Stick voting field. (c) Schematic of tensor voting rules.
(a) (b) (c)LetL denote the distance between the voting point and the receiving point; then θ is the angle between the tangent line of the voting point on a close circle and the straight line between the voting point and the receiving point, ands andk denote the arc length and curvature, respectively.Then voting ofP received by tensor atO can be defined as(10)VP=DFs,k,σNPNPTNP=NO-sin⁡2θ,cos⁡2θT,DFs,k,σ=e-s2+ck2/σ2where DFs,k,σ is saliency decay function and σ is the scale factor that determines the size of the voting fields, and the only parameter that can be changed.c is the parameter to control the degree of attenuation.After tensor voting, every pixel collects all votes projected by the tensor in its neighborhood and integrates them into a new tensor. The accumulation of votes is obtained by adding up the tensor. Finally, the new tensor is decomposed into the form of (9) and the eigenvalue is calculated. The saliency of the probability of each point in the image is obtained. The structural features of the image can be judged by calculating the saliency.When the saliency of a pixel isλ1-λ2>λ2, the pixel is a point on the curve, which can be judged as a road point. If λ1≈λ2>0, the point is a region or junction point, but the area belongs to road intersection and has more than two branches, so it can be detected. In other cases, the pixel is singular and does not require processing.The directionless point does not have the stick tensor part, for the segmented road with no direction; it cannot vote out the stick, so it is impossible to extract the saliency of the curve. Therefore, all road points are coded by ball tensor, and each road point is coded asT=1001; then all coding points are voted sparsely in a sphere of voting, which makes the road points have a certain directivity. Then the ball tensor encodes the nonroad points and reconducts a dense voting in the stick voting field for all points. After two votes, the stick voting tensor obtained from each point is decomposed, and the curve saliency λ1-λ2 of each road point can be extracted. To extract all points, it should satisfy λ1=λ2, which is a cross region; it is as a candidate area of road intersections.


## 2.1. Image Segmentation

The purpose of this step is to segment the input image and obtain preliminary road regions. The SUSAN (Smallest Univalue Segment Assimilating Nucleus) [35] algorithm is adopted to segment images. SUSAN algorithm is the representative of template matching, which was proposed by Smith and Brady. SUSAN algorithm moves the template on image, whether the template center is an edge point determined by whether the matching degree of it reaches the threshold. Because the SUSAN algorithm is based on the grayscale comparison of the pixels in the neighborhood and does not need to calculate the gradient, the interference range of the noise is obviously smaller than the edge detection based on the gradient.The SUSAN algorithm uses a circular template to move over the image [36]. If the difference between the gray level of the pixel in the template and that of the central pixel is lower than the given threshold [37], then it is considered that the point has a similar gray level with the central pixel of the template [38]. A region consisting of pixels satisfying such a condition is called an USAN (Univalue Segment Assimilating Nucleus). In Figure 2,a andb are completely located in the foreground and background of the image, respectively. The area of USAN is the largest. In addition,c,d,e are moving closer to the edge, while the area of USAN is approaching to the minimum. The results show that the USAN values of edge pixels are less than or equal to half of the maximum value of the period. The edge points can be detected.Figure 2
Principle of SUSAN feature detection.The steps of detecting image edge information by using SUSAN algorithm are as follows:(1) Find out the maximum and minimum gray valueImax and Imin from the image, and calculate the best detection threshold: (1)t=Imax-Imin10(2) Traverse the whole image and check location characteristics of each pixel according to the following formula:(2)cr,r0=1Ir-Ir0≤t0Ir-Ir0>twhere c(r,r0) is a discriminant function. If its value is 1, then the pixel is located in the USAN region; I(r0) is the gray value of the central pixel of the template; I(r) is the gray value of any other pixel in the template.The USAN area statistics for each pixel are as follows:(3)nr0=∑r∈Dr0cr,r0where D(r0) is a circular template region and r0 is the center of the circle.(3) The threshold valueg is set. When n(r0)<g, the detected pixel position r0 is assumed to be at the edge position of the image.(4) Traverse the whole image to detect the complete edge information.According to the characteristics of the road in the remote sensing image, in the obtained edge results map, the roads are located in the nonedge homogeneous regions. The results of edge extraction map are segmented using a threshold, and road candidate areas are obtained.


## 2.2. Road Information Extraction

After the previous algorithm is implemented, the image can be segmented into two categories, homogeneous regions and edges. Road segments are in homogeneous regions. But buildings or bare soil segments are also mixed into road areas. The geometric characteristics of road will be used to find out the potential road regions and eliminate other nonroad segments.According to the geometric features, roads do not have small areas, and the length is much larger than the width. So roads are obviously displayed as narrow, long, linear feature [39]. Through the identification of the big-area regions and linear features, it is easy to remove the small areas and nonroad regions.In this paper, the result of image segmentation is processed from the two aspects aiming to eliminate the nonroad regions.(1) Compute the area of each independent region in the result of segmentation, and remove the regions whose value is less than the threshold. It can be expressed as(4)Region=roadregionRarea>Rthrenon-roadreginotherwisewhere Rarea is the area of region and Rthre is the threshold.(2) The ratio of length to width of the minimum external rectangle in each region is calculated in the result of segmentation. If the ratio of length to width is greater than the set threshold in this region, then the region can be marked as the road area. The ratio of length to width of the region is defined as(5)R=LminWminwhere Lmin is the length of the minimum external rectangle of the region and Wmin is the width of the rectangle.When road is bent and the linear features cannot be well described, there will be errors in calculating the aspect ratio by using the above formula. The formula needs to be modified to overcome this limitation. We take the total number of pixels in the detection region as the area of the external rectangle and create a new rectangle with the diagonal line of the smallest external rectangle as the long edge. Therefore,(6)W=nLwhere W is the width of the new rectangle; n is the number of pixels of the region; and L is the length of the new rectangle; the value of L can be calculated by(7)L=Lmin2+Wmin2Therefore, in practical application, the aspect ratio of the region can be calculated by(8)Rmin=LW=Lmin2+Wmin2nThe results of removing nonroad regions using road’s shape features are shown in Figure3. The test image is a QuickBird image; the resolution is 0.61m/pixel.Figure 3
Results of removing nonroad regions using shape features on QuickBird image. (a) Input image. (b) Results of preliminary road extraction. (c) Results of removing small area regions. (d) Results of eliminating nonlinear features.
(a) (b) (c) (d)


## 2.3. Tensor Voting

After previous extraction, the main road regions have been extracted. However, there are still some large holes, gaps, and many other nonroad regions in the results of detected roads. Generally, mathematical morphology is used to delete the isolated nonroad regions and connect the small gaps of the road. However, this method cannot fill the large holes in the results. It cannot connect the large gaps and completely delete the nonroad regions [40]. Therefore, this paper uses tensor voting [41–43] algorithm to solve the above problems.Tensor voting is a robust method for feature extraction. The main idea of tensor voting algorithm is that every point in the space collects the tensor information from other points in the neighborhood and encodes it as a new tensor to be used in the next voting. After the voting, it decomposes the new tensor. Thus, the salience map of various characteristics is obtained. It can be used to detect the geometric structures, which consists of two parts, the tensor representation of the data (tensor coding) and the nonlinear voting between tensors (tensor voting).Tensor voting can obtain the salient features of the image, so it is possible to detect the geometric features of the typical objects. Road has obvious elongated shape, and intersection has obvious ball shape. We can use tensor voting to extract roads and detect road intersections from remotely sensed images.Firstly, the second-order positive semidefinite symmetric tensor is used to represent the direction and significance of the pixels in the image. In a two-dimensional space, tensor can be decomposed into a linear combination of eigenvalues and eigenvectors:(9)T=λ1-λ2e1⃑e1⃑T+λ2e1⃑e1⃑T+e2⃑e2⃑Twhereλ1 and λ2 are nonnegative eigenvalues and λ1>λ2.  e1⃑  and  e2⃑ are the corresponding eigenvectors. e1⃑e1⃑T is stick tensor (representing curve characteristics), e1⃑ represents the direction of a curve, and λ1-λ2 is a significant index of the curve. e1⃑e1⃑T+e2⃑e2⃑T is ball tensor (representing node characteristics), and λ2 is a significant index of the node.Then tensor voting is executed. Firstly, it initializes the pixels to the ball tensor and vote; the voting field is shown in Figure4(a). Next, the initial direction is obtained by analyzing the number of votes received at each point, and the initial direction is assigned to the stick voting field as shown in Figure 4(b). Then voting in the stick field, the voting rules are as follows (see Figure 4(c)); in coordinate systemOxy, there are two tensors atO andP, respectively, whereO is the voting point,P is the receiving point, andNO andNP are their normal vectors, respectively.Figure 4
Tensor voting. (a) Ball voting field. (b) Stick voting field. (c) Schematic of tensor voting rules.
(a) (b) (c)LetL denote the distance between the voting point and the receiving point; then θ is the angle between the tangent line of the voting point on a close circle and the straight line between the voting point and the receiving point, ands andk denote the arc length and curvature, respectively.Then voting ofP received by tensor atO can be defined as(10)VP=DFs,k,σNPNPTNP=NO-sin⁡2θ,cos⁡2θT,DFs,k,σ=e-s2+ck2/σ2where DFs,k,σ is saliency decay function and σ is the scale factor that determines the size of the voting fields, and the only parameter that can be changed.c is the parameter to control the degree of attenuation.After tensor voting, every pixel collects all votes projected by the tensor in its neighborhood and integrates them into a new tensor. The accumulation of votes is obtained by adding up the tensor. Finally, the new tensor is decomposed into the form of (9) and the eigenvalue is calculated. The saliency of the probability of each point in the image is obtained. The structural features of the image can be judged by calculating the saliency.When the saliency of a pixel isλ1-λ2>λ2, the pixel is a point on the curve, which can be judged as a road point. If λ1≈λ2>0, the point is a region or junction point, but the area belongs to road intersection and has more than two branches, so it can be detected. In other cases, the pixel is singular and does not require processing.The directionless point does not have the stick tensor part, for the segmented road with no direction; it cannot vote out the stick, so it is impossible to extract the saliency of the curve. Therefore, all road points are coded by ball tensor, and each road point is coded asT=1001; then all coding points are voted sparsely in a sphere of voting, which makes the road points have a certain directivity. Then the ball tensor encodes the nonroad points and reconducts a dense voting in the stick voting field for all points. After two votes, the stick voting tensor obtained from each point is decomposed, and the curve saliency λ1-λ2 of each road point can be extracted. To extract all points, it should satisfy λ1=λ2, which is a cross region; it is as a candidate area of road intersections.


## 3. Experiments

### 3.1. Road and Intersection Extraction

The first experiment is conducted on a IKONOS remote sending image with a spatial resolution 1.0 m/pixel and the size1024×1024. There are circular, straight, and curved roads in the image, which belong to the hybrid road network (see Figure 5(a)) [44]. The SUSAN algorithm is applied to segment the input image, the image is segmented into two categories, road regions and nonroad regions, and then shape features of road are used to delete nonroad regions from preliminary road extraction results which are shown in Figure 5(b). Then we use tensor voting to delete the nonroad areas and connect the broken road areas. The results of stick salience and ball salience after vote analysis are illustrated in Figures 5(c) and 5(d), respectively. Figures 5(e) and 5(f) show the results of ball regions and stick regions, respectively, which are potential roads and intersections. Figure 5(g) shows the results by our method. Figure 5(h) shows the extracted roads and road intersections overlapped on the test image.Figure 5
Results of the first experiment. (a) Testing image. (b) Coarse road regions extracted by template matching and road shape features. (c) Results of stick salience by vote analysis. (d) Results of ball salience by vote analysis. (e) Ball regions. (f) Stick regions. (g) Results of extracted road by the proposed method. (h) The extracted roads and road intersections overlapped on the test image.
(a) (b) (c) (d) (e) (f) (g) (h)According to the experimental results, we purify and optimize the preliminary results of road extraction. After tensor voting, the results are very satisfactory. The ball tensor voting to the road intersection detection is also very accurate. From the results of the experiment, the proposed method can extract complete road segments accurately; meanwhile it can find all the road intersections from the input image. Therefore, it can be concluded that the proposed method can effectively detect all the road regions and road intersections in IKONOS panchromatic image.The results of road and road intersection extraction using proposed method from QuickBird image, the size of the test image being512×512, are shown in Figure 6. The result of road extraction using SUSAN algorithm and geometric filtering is illustrated in Figure 6(a). Figure 6(b) shows the ball regions detected by tensor voting; we can see that all the potential intersections have been detected. The result of roads extraction by our method is shown in Figure 6(c). Figure 6(d) illustrates the result of the overlay of the detected roads and intersections on the original remote sensing image.Figure 6
Results for QuickBird image with spatial size of512×512. (a) Coarse road regions. (b) Results of ball salience. (c) Results of stick salience. (d) The extracted roads and road intersections overlaped on the input image.
(a) (b) (c) (d)The next two experiments use Geoeye image and WordView-I image. The size of the input image is512×512 and 1500×1500, respectively. The results of each step in the experiment using Geoeye image are shown in Figures 7(a)–7(c), and those using WordView-I image are shown in Figures 7(d)–7(f).Figure 7
Results for Geoeye image and WorldView-I image. (a)(d) Testing image. (b)(e) Coarse road regions. (c)(f) The extracted roads and road intersections.
(a) (b) (c) (d) (e) (f)From all the experimental results in Figures5–7, we can see that our method can extract pure roads obtained by different sensors and can detect almost all road intersections. Then the accuracy of the method will be quantitatively analyzed in the next section.


### 3.2. Accuracy Assessment of Purification Result

For evaluating the proposed method, there are five indexes of accuracy used in this paper [45], which are defined as [46](11)Completeness=TPTP+FN×100%(12)Correctness=TPTP+FP×100%(13)Quality=TPTP+FP+FN×100%(14)Omit=FNTP+FN×100%(15)Redundancy=FPTP+FN×100%where TP is the area where the extracted road regions and reference roads coincide with each other, FP is the area where the roads are extracted but does not exist in the reference roads, and FN is the area that exists in the reference roads not extracted.We use the number of pixels in the regions as the area of the regions. The reference road maps are hand-drawn according to the input image. Figure8 illustrates the evaluation principle [47].Figure 8
The evaluation principle.Then we use completeness, correctness, quality, omit and redundancy indicators to analyze the previous experimental results. The reference road, preliminary results, and results extracted by using our method are shown in Figure9. Reference road maps of the test image being hand-drawn are shown in Figures 9(a), 9(d), 9(g), and 9(j). Figures 9(c), 9(f), 9(i), and 9(l) show the results of road extracted by using our method. Figures 9(b), 9(e), 9(h), and 9(k) show the results without further processing by tensor voting. By comparing the results obtained with proposed method and those without tensor voting, we can see that the results extracted by our method are smoother than those without tensor voting, and they are much more similar to the reference roads.Figure 9
Results of road extraction without using and using the proposed method. (a)(d)(g)(j) Reference road maps. (b)(e)(h)(k) Results of coarse roads extracted without using the proposed method. (c)(f)(i)(l) Results of roads extracted by the proposed method.
(a) (b) (c) (d) (e) (f) (g) (h) (i) (j) (k) (l)All indicators with and without using tensor voting method to extract roads are shown in Table1. It can be seen that the precision and extraction rate of the road extracted by this method are very high, while the false alarm rate and the missing rate are very low, which shows that the method can extract road information effectively.Table 1
Performance evaluation of the proposed method.
Image IKONOS QuickBird Geoeye WorldView-I Figure9(c) Figure9(b) Figure9(f) Figure9(e) Figure9(i) Figure9(l) TP(pixels) 148922 125545 62612 64020 28767 429342 FP(pixels) 1502 27816 2805 20843 2553 9701 FN(pixels) 2583 26030 8541 7133 1969 19618  Completeness 98.30% 82.83% 88.00% 89.98% 93.59% 95.63% Correctness 99.00% 81.86% 95.71% 75.44% 91.85% 97.79% Quality 97.33% 69.98% 84.66% 69.59% 86.42% 93.61% Omit 1.70% 17.17% 12.00% 10.02% 6.41% 4.37% Redundancy 0.99% 18.35% 3.94% 29.29% 8.31% 2.16%


### 3.3. Comparison with the State-of-the-Art Methods

The method in this paper is compared with SSC [6] and knowledge-based [7]. Figure 10 gives the comparision results of the methods mentioned in SSC, knowledge-based, and proposed methods. From the results of extraction, we can see that the results extracted by our method are better than those obtained by the other two methods. The proposed method by using tensor voting is superior in eliminating nonroad information and connecting the broken roads.Figure 10
The results of the proposed method and the state-of-the-art methods. (a) Reference maps. (b) SSC [6]. (c) Knowledge-based method [7]. (d) The proposed method.
(a) (b) (c) (d)In order to evaluate these methods quantitatively and compare the extraction effect results of the three methods, the above five measurement indexes are calculated, and the results are shown in Table2. It is clear from the quantitative results that the method proposed in this paper is the best.Table 2
Comparison of the proposed method with the state-of-the-art methods.
Method Completeness Correctness Quality Omit Redundancy SSC [6] 85.05% 85.94% 74.66% 14.95% 13.92% Knowledge-based [7] 64.43% 76.03% 53.55% 35.57% 20.31% Proposed method 97.45 % 93.70 % 93.52 % 2.55 % 1.77 %


## 3.1. Road and Intersection Extraction

The first experiment is conducted on a IKONOS remote sending image with a spatial resolution 1.0 m/pixel and the size1024×1024. There are circular, straight, and curved roads in the image, which belong to the hybrid road network (see Figure 5(a)) [44]. The SUSAN algorithm is applied to segment the input image, the image is segmented into two categories, road regions and nonroad regions, and then shape features of road are used to delete nonroad regions from preliminary road extraction results which are shown in Figure 5(b). Then we use tensor voting to delete the nonroad areas and connect the broken road areas. The results of stick salience and ball salience after vote analysis are illustrated in Figures 5(c) and 5(d), respectively. Figures 5(e) and 5(f) show the results of ball regions and stick regions, respectively, which are potential roads and intersections. Figure 5(g) shows the results by our method. Figure 5(h) shows the extracted roads and road intersections overlapped on the test image.Figure 5
Results of the first experiment. (a) Testing image. (b) Coarse road regions extracted by template matching and road shape features. (c) Results of stick salience by vote analysis. (d) Results of ball salience by vote analysis. (e) Ball regions. (f) Stick regions. (g) Results of extracted road by the proposed method. (h) The extracted roads and road intersections overlapped on the test image.
(a) (b) (c) (d) (e) (f) (g) (h)According to the experimental results, we purify and optimize the preliminary results of road extraction. After tensor voting, the results are very satisfactory. The ball tensor voting to the road intersection detection is also very accurate. From the results of the experiment, the proposed method can extract complete road segments accurately; meanwhile it can find all the road intersections from the input image. Therefore, it can be concluded that the proposed method can effectively detect all the road regions and road intersections in IKONOS panchromatic image.The results of road and road intersection extraction using proposed method from QuickBird image, the size of the test image being512×512, are shown in Figure 6. The result of road extraction using SUSAN algorithm and geometric filtering is illustrated in Figure 6(a). Figure 6(b) shows the ball regions detected by tensor voting; we can see that all the potential intersections have been detected. The result of roads extraction by our method is shown in Figure 6(c). Figure 6(d) illustrates the result of the overlay of the detected roads and intersections on the original remote sensing image.Figure 6
Results for QuickBird image with spatial size of512×512. (a) Coarse road regions. (b) Results of ball salience. (c) Results of stick salience. (d) The extracted roads and road intersections overlaped on the input image.
(a) (b) (c) (d)The next two experiments use Geoeye image and WordView-I image. The size of the input image is512×512 and 1500×1500, respectively. The results of each step in the experiment using Geoeye image are shown in Figures 7(a)–7(c), and those using WordView-I image are shown in Figures 7(d)–7(f).Figure 7
Results for Geoeye image and WorldView-I image. (a)(d) Testing image. (b)(e) Coarse road regions. (c)(f) The extracted roads and road intersections.
(a) (b) (c) (d) (e) (f)From all the experimental results in Figures5–7, we can see that our method can extract pure roads obtained by different sensors and can detect almost all road intersections. Then the accuracy of the method will be quantitatively analyzed in the next section.


## 3.2. Accuracy Assessment of Purification Result

For evaluating the proposed method, there are five indexes of accuracy used in this paper [45], which are defined as [46](11)Completeness=TPTP+FN×100%(12)Correctness=TPTP+FP×100%(13)Quality=TPTP+FP+FN×100%(14)Omit=FNTP+FN×100%(15)Redundancy=FPTP+FN×100%where TP is the area where the extracted road regions and reference roads coincide with each other, FP is the area where the roads are extracted but does not exist in the reference roads, and FN is the area that exists in the reference roads not extracted.We use the number of pixels in the regions as the area of the regions. The reference road maps are hand-drawn according to the input image. Figure8 illustrates the evaluation principle [47].Figure 8
The evaluation principle.Then we use completeness, correctness, quality, omit and redundancy indicators to analyze the previous experimental results. The reference road, preliminary results, and results extracted by using our method are shown in Figure9. Reference road maps of the test image being hand-drawn are shown in Figures 9(a), 9(d), 9(g), and 9(j). Figures 9(c), 9(f), 9(i), and 9(l) show the results of road extracted by using our method. Figures 9(b), 9(e), 9(h), and 9(k) show the results without further processing by tensor voting. By comparing the results obtained with proposed method and those without tensor voting, we can see that the results extracted by our method are smoother than those without tensor voting, and they are much more similar to the reference roads.Figure 9
Results of road extraction without using and using the proposed method. (a)(d)(g)(j) Reference road maps. (b)(e)(h)(k) Results of coarse roads extracted without using the proposed method. (c)(f)(i)(l) Results of roads extracted by the proposed method.
(a) (b) (c) (d) (e) (f) (g) (h) (i) (j) (k) (l)All indicators with and without using tensor voting method to extract roads are shown in Table1. It can be seen that the precision and extraction rate of the road extracted by this method are very high, while the false alarm rate and the missing rate are very low, which shows that the method can extract road information effectively.Table 1
Performance evaluation of the proposed method.
Image IKONOS QuickBird Geoeye WorldView-I Figure9(c) Figure9(b) Figure9(f) Figure9(e) Figure9(i) Figure9(l) TP(pixels) 148922 125545 62612 64020 28767 429342 FP(pixels) 1502 27816 2805 20843 2553 9701 FN(pixels) 2583 26030 8541 7133 1969 19618  Completeness 98.30% 82.83% 88.00% 89.98% 93.59% 95.63% Correctness 99.00% 81.86% 95.71% 75.44% 91.85% 97.79% Quality 97.33% 69.98% 84.66% 69.59% 86.42% 93.61% Omit 1.70% 17.17% 12.00% 10.02% 6.41% 4.37% Redundancy 0.99% 18.35% 3.94% 29.29% 8.31% 2.16%


## 3.3. Comparison with the State-of-the-Art Methods

The method in this paper is compared with SSC [6] and knowledge-based [7]. Figure 10 gives the comparision results of the methods mentioned in SSC, knowledge-based, and proposed methods. From the results of extraction, we can see that the results extracted by our method are better than those obtained by the other two methods. The proposed method by using tensor voting is superior in eliminating nonroad information and connecting the broken roads.Figure 10
The results of the proposed method and the state-of-the-art methods. (a) Reference maps. (b) SSC [6]. (c) Knowledge-based method [7]. (d) The proposed method.
(a) (b) (c) (d)In order to evaluate these methods quantitatively and compare the extraction effect results of the three methods, the above five measurement indexes are calculated, and the results are shown in Table2. It is clear from the quantitative results that the method proposed in this paper is the best.Table 2
Comparison of the proposed method with the state-of-the-art methods.
Method Completeness Correctness Quality Omit Redundancy SSC [6] 85.05% 85.94% 74.66% 14.95% 13.92% Knowledge-based [7] 64.43% 76.03% 53.55% 35.57% 20.31% Proposed method 97.45 % 93.70 % 93.52 % 2.55 % 1.77 %


## 4. Conclusions

A multistage and multifeature method is proposed to extract roads and road intersections from high-resolution remote sensing images based on template matching and tensor voting. Firstly, SUSAN algorithm is used to segment the input image, to find out the potential road candidate points, and obtain initial road regions. Secondly, shape features of road are used to identify the initial road areas and remove the nonroad areas for the purpose of obtaining purer segments of road. Due to the influence of cars, pedestrians, and roadside trees on the road, there will be some holes and gaps in the road regions, and some nonroad areas are stuck in the road areas. Thirdly, tensor voting is used to delete the nonroad areas that adhere to the road regions, fill the holes in the roads and connect the gaps, and extract the pure and complete segments of the road. Finally, we extract all the ball regions from the road results and then detect all the information of the road intersections from the ball regions according to the characteristics of the road intersections. The experimental results show that our method can extract pure road information and accurately detect road intersections. Compared with other methods, our method significantly improves the accuracy of road extraction. Future work will focus on reducing the effects of interference factors in the preprocessing stage, automatically selecting the optimal parameters for image segmentation during the image segmentation phase, so as to make the road extraction more accurate.



---

*Source: 6513418-2019-03-04.xml*