# A Simple Outdoor Environment Obstacle Detection Method Based on Information Fusion of Depth and Infrared

Authors: Yaguang Zhu; Baomin Yi; Tong Guo
Journal: Journal of Robotics
 (2016)

---

## Abstract

In allusion to the existing low recognition rate and robustness problem in obstacle detection; a simple but effective obstacle detection algorithm of information fusion in the depth and infrared is put forward. The scenario is segmented by the mean-shift algorithm and the pixel gradient of foreground is calculated. After pretreatment of edge detection and morphological operation, the depth information and infrared information are fused. The characteristics of depth map and infrared image in edge detection are used for the raised method, the false rate of detection is reduced, and detection precision is improved. Since the depth map and infrared image are not affected by natural sunlight, the influence on obstacle recognition due to the factors such as light intensity and shadow is effectively reduced and the robustness of the algorithm is also improved. Experiments indicate that the detection algorithm of information fusion can accurately identify the small obstacle in the view and the accuracy of obstacle recognition will not be affected by light. Hence, this method has great significance for mobile robot or intelligent vehicles on obstacle detection in outdoor environment.


---

## Body

## 1. Introduction

Detection of the obstacle has a wide application in smart robots, intelligent vehicles, and autonomous agricultural vehicles. Among them, obstacle detection based on visual image information [1, 2] is mainly used in vision system of automatic driving vehicle or robot, The system completes some special functions similar to the human eye and provides reliable and accurate environmental information for path planning. Hence, the visual navigation system must be able to detect obstacles quickly and accurately and has better robustness.The sensors used for traditional obstacle detection mainly include laser radar sensors, ultrasonic sensors, infrared sensors, and visual equipment [3]. The basic principle of laser radar sensors is that the distance information can be obtained by calculating the time interval between emission and receiving of laser [4]. Intelligent vehicle named Stanley [5] from Stanford University installs a lot of laser radar sensors at the top of the car, which is used for judging whether the area is an obstacle or an area that can be passed through by 2D grid based on Markov model and height difference between adjacent points. However, the size of the laser radar sensor is relatively large, the cost is very high, and the installation and use are also more complex. In the study of detecting obstacles by using ultrasonic sensors, Figueroa and Lamancusa [6] proposed a method to detect obstacles by using the relationship between the phase delay and the peak value of the echo and then combined with echo time, which had higher detection accuracy. Furthermore, Hua et al. [7] proposed a method to detect phase position by modulating double-sideband amplitude, in which excitation signal of ultrasonic pulse was modulated by using a low frequency signal and adopting appropriate modulation data. This method not only guarantees the resolution of the phase detection obstacle, but also expands the scope of the obstacle detection to a certain extent. But the orientation of the ultrasonic sensors is poorer during the detection. When used in moving or complex conditions such as wild environment, it will not perform well. The principle of infrared sensor is infrared thermal imaging technology [8], which is a wavelength conversion technology. It is the transformation of infrared radiation to the visible graphics. Infrared thermal imaging system has a certain ability to penetrate and it can distinguish right from wrong and has strong anti-interference ability. According to the characteristics of infrared imaging, the infrared sensor has better adaptability and stronger direction. But it is very easy to be affected by the color of the object, and it cannot provide distance information. In recent years, with the development of computer technology, it has become more and more popular to use binocular camera to obtain the depth information of object and scene. However, the disadvantage of this method is that it is easy to be affected by light [9, 10]. Zhang and Zhu [11] proposed a method to segment image based on binocular vision, but it was very easy to be affected by light and has low robustness. Although the use of multiple sensors can avoid the shortage of a single sensor, it is difficult to achieve the integration of information and to ensure the synchronization between the various sensors.Aiming at the problems of poor robustness and low detection accuracy, a fast and accurate detection method based on depth information and infrared information is proposed. Kinect V2.0 sensor is used in this paper, which can obtain depth information and infrared information at the same time. The main processes of obstacle detection in this paper are as follows. Firstly, the areas to be detected are segmented by the mean-shift algorithm. Secondly, gradient calculation is carried out to highlight the edge information of the object. Finally, the obstacle can be detected by using edge detection or other methods and fusing infrared and depth information. The method proposed in the paper can solve the problems of the accuracy and robustness of obstacle detection of mobile robot and intelligent vehicle in complex environment.


## 2. Problem Descriptions

Usually, obstacles need to be detected in both indoor and outdoor environment. The only difference is that the outdoor obstacles may be more complex and diverse. Two different scenes collected by Kinect are shown in Figures1 and 2. It can be seen from the two images that the edge information of contact part between the obstacle and ground in depth map is not very clear. The edge information of the upper part of the obstacle is more obvious. But it is opposite in infrared image. Using the characters of infrared image information, the depth map can be improved. Therefore, if depth information and infrared information are fused together effectively, the detected obstacle boundaries can be segmented from the visual scene accurately. Besides, it will enhance the robustness and can guarantee high recognition rate because the depth information and infrared information are not affected by natural light.Figure 1
The road scene.
(a)
Color image (b)
Depth image (c)
Infrared imageFigure 2
The grassland scene.
(a)
Color image (b)
Depth image (c)
Infrared imageRecently, most of the obstacle detection algorithms based on single vision adopt motion information of the different images [12]. However, due to the lack of additional information, it makes the obstacle produce abnormal values. But it can be solved by fusion of infrared image and depth map. Most obstacles information in the depth map is uniform, because the depth information of the same obstacle has similar characteristics and the depth differences of connection parts between obstacle and upholder are small. Hence, the depth information can be used to remove abnormal values in detection areas as background or some areas beyond the scope of the detection area. If only the depth image is used for obstacle detection and the obstacle is on the edge of the identified range, the depth value of obstacle is similar to the value of the ground when obstacle is smaller. So the ground and obstacle are easily identified as one object, and then errors may occur. Moreover, even though the objects have beyond the setting detection distance, with the good reflection, Kinect camera is still able to get its depth information and be able to detect it. That makes the identified obstacles exceede the scope of the requirements. On the other hand, if only infrared image is used for detection, many wrong areas will be detected because of the characters of infrared information. The colors of areas are diversified and most of these errors are relatively close to the camera and at the same time further obstacles will be ignored. The detection results of infrared images are shown in Figure 3(a).Figure 3
An example of single obstacle.
(a)
The result of infrared detection (b)
The result of fusionAccording to the analysis above, we find that the wrong detection areas of depth map and the wrong detection areas of infrared image are not coincident. So the infrared image can be used for removing the wrong detection areas of depth map. The detection results after the fusion of depth map and infrared image are shown in Figure3(b), in which the obstacle is a can. Therefore, in order to improve precision and performance, the depth map and the infrared image should be fused to detect the obstacles.


## 3. Fusion Method

Kinect V2.0 sensor is adopted in this paper, which can collect depth map and infrared image with 512 × 424 resolutions and 30 fps. Meanwhile, it can also collect color image with 1920 × 1080 resolutions and 30 fps. The effective range is 0.5 m–8 m [13]. Compared with other sensors, Kinect V2.0 can collect depth information and infrared and color information at the same time. It effectively avoids the problem of uncoordinated caused by using different sensors. In addition, it has lots of advantages; for example, this sensor has small volume and can obtain accuracy information of image. Besides, the most important is that it has strong adaptability to complex environment. Here, the method proposed by Zhang et al. [14, 15] is adopted to calibrate Kinect camera before the detection of obstacles. The thick red line in Figure 4 represents the vertical wall. Compared with pictures before calibration, the distortion caused by the camera has been corrected obviously.Figure 4
Depth image with calibration.
(a)
Color image (b)
Depth image before calibration (c)
Depth map after correction distortionAccording to the advantages of Kinect V2.0, obstacle detection algorithm based on information fusion of depth and infrared is proposed. The foreground of depth map is intercepted by mean-shift algorithm firstly, and then the edge information is extracted and processed by the fusion of depth map and infrared image, which is shown in Figure5. The detection range is 0.5 m–4 m. Edge detection based on depth image or infrared image alone has its own characteristics. Gradient features of depth information can be obtained in obstacle detection with depth image alone. However, since the edge information of the areas segmented by mean-shift algorithm is more prominent, they also can be detected. Similarly, in obstacle detection with infrared image alone, the edge information of the front object in the image taken by the Kinect camera can be extracted according to the characteristics of infrared image, while the edge information of the rear object cannot be detected. Finally, a perfect detection result can be obtained as long as combining these two features.Figure 5
Block diagram of detection process.In the depth image obtained by Kinect, it is an effective method to segment foreground of detection area via mean-shift algorithm. It has not received much attention when the mean-shift theory was proposed by Fukunaga and Hostetler [16], until the kernel function was proposed by Cheng [17]. The use scope of the mean-shift algorithm was expanded, and it is only beginning to attract more attention. Comaniciu et al. [18, 19] successfully applied the mean-shift algorithm to image processing and got good results in smoothness and segmentation of image. The kernel function defined by Cheng is that: if a function G:X→R has a profile function g:[0,+∞]→R, namely,(1)Gx=gx2and should meet the following: (1)g is not negative; (2)g is not increasing, namely if a<b, then g(a)≥g(b); (3)g is piecewise continuous, and ∫0∞g(r)dr<∞, then G(x) is a kernel function, in which x is a point in space, represented by column vector and R represents real number field.According to kernel function, the mean-shift formula can be described as follows [16]:(2)Mxj=∑i=1nGHxi-xjwxixi∑i=1nGHxi-xjwxi-xj,in which xj is the depth of the pixel to be processed, xi is the depth of the sampled pixel point, G(x) is an unit kernel function, wxi is a weight of sampling point xi, GH(xi-xj)=H-1/2G(H-1/2(xi-xj)), and H is bandwidth matrix that is usually expressed by a positive definite symmetric matrix with d×d. When this function is used, the error value η is set firstly. The mean-shift algorithm is a process of loop calculation; make(3)mhxj=∑i=1nGHxi-xjwxixi∑i=1nGHxi-xjwxi.Step 1.
Calculatemh(xj).Step 2.
Substitute the value ofmh(xj) into x.Step 3.
Ifmh(xj)-xj<η, as for the location of pixel point xj: location identifier pxj(x,y)=1, the cycle will terminate. At that time, for any pixel point xj, we have(4)Pxj=x,y∈Akin which Ak is region for segmentation at this time, pxj(x,y) is a pixel coordinate of xj in image. Otherwise, the pixel point xj+1 will be taken as a new starting point of space, and then the first step will restart.We can get the expected foreground as long as theA1,A2,A3,…,Ak which satisfied the requirement are combined together. That is,(5)A1,A2,A3,…,Am=A.in which m<k, A is foreground area. The result of foreground obtained by mean-shift algorithm is shown in Figure 6(c). The foreground is a region near the camera, and the obstacles on the front will be segmented into the foreground overall. The purpose of calculating gradient of each pixel point in the foreground is to highlight the edge information of the obstacle according to the characteristic of gradient. In the regions of continuous depth value, the gradient value will be small, and the change will be continuous. But in the edge of the object, the gradient value will change dramatically, and the edge information is more obvious. In the depth image I, when a pixel point is (x,y)∈A and the depth value of it is represented by I(x,y), the convolution gradient operators of the point in x direction and y direction are as follows [20]:(6)Txx,y=-101∗Ix-1,yIx,yIx+1,yTTyx,y=-101T∗Ix,y-1Ix,yIx,y+1.So, the gradient value of pixel point (x,y) is(7)Tx,y=Txx,y2+Tyx,y2.Figure 6
Foreground information image.
(a)
Color image (b)
Depth image (c)
Foreground image (d)
Foreground gradientThe new gradient image is obtained after gradient calculation of foreground, which is shown in Figure6(d). After that, we use edge detection algorithm to detect the gradient image in Figure 6(d). The Canny edge detectors are easier to improve the performance of edge detection algorithms. Similarly, the edge detection of infrared image can be carried out by this way. The value of most areas in the image is uniform. The results of detection indicate that it can completely avoid the effects of light, shadow, and other factors. After obstacles detection in the two images, morphology operations are performed. In detail, dilation operations are carried out, and then erosion operations, finally, merging adjacent points and removing isolated points. A′ represents a foreground that has achieved gradient calculation and edge detection. S1 represents a structural element for dilation. S2 represents a structural element for erosion. (x,y) represents a location of pixel point. Expanding A′ by S1, we have(8)A′=AA′⊕S1x,y=max⁡A′x-x′,y-y′+S1x′,y′∣x′,y′∈DS1in which DS1is a domain of S1. For corrosion A′ by S2, we have(9)A′ΘS2x,y=min⁡A′x-x′,y-y′-S2x′,y′∣x′,y′∈DS2in which DS2is a domain of S2.Then, we label the results of morphological operations and combine the detection result of two kinds of images by intersection calculation. In this way, the wrong results of detection can be removed. So far, the whole process of obstacle detection has been achieved. The value of the center point of the obstacle region is regarded as the depth value of the obstacle.Pd and Pi, respectively, represent pixel value of depth image and infrared image after labeling at the location of (x,y). p represents pixel value of these points after mixture:(10)pdx,y,pix,y=1pdx,y,pix,y=0p=pd⊗pi.When the obstacle is detected, the value ofPd and Pi is 1; if not, it is 0. If the values of Pd and Pi are 1 at the same time, the value of p is 1. Finally, the identified obstacles are marked by the blue box.


## 4. Experiments and Result

### 4.1. Single Image Identification

The results of obstacle detection via a single image on the grass by Kinect sensor are shown in Figure7. In the picture, the size of carton is 25 cm × 45 cm, and the distance from the sensor is 1 m. The size of the paper baskets is 16 cm × 25 cm, and the distance from the sensor is 1.5 m. The angle of inclination of Kinect sensor is 30°. Since the yellow flowers are far away from the sensor and have over 8 m in color image, their information cannot be collected in depth image and infrared image. We can clearly conclude from the pictures that any results from single image cannot accurately identify the obstacles. In color image, due to the abundant color information and color change, the edge information is really distinct. As shown in Figure 7(b), the edge information of foreground and background both can be detected, but only obstacles in foreground are demanded. So the color image is not suitable for our research. Besides, the obtainment of color map is also easily affected by the illumination condition. When the depth map is used alone, the area edge and obstacle will be easily identified as one object if the obstacle is close to the edge of segmented foreground, which is shown in Figure 7(f). Meanwhile, the edge information is not detected in front regions without obstacles and the error information is less in rear regions, which is shown in Figure 7(e). However, when the infrared image is used alone, the edge information of front obstacles can be detected effectively. But the edge information of farther grass can also be detected and the information of the rear object is ignored, which is shown in Figure 7(h). In a word, if only single image is obtained for detection, it will produce many errors. Accurate obstacles identification cannot be achieved and neither does the robustness.Figure 7
The results of using single image identification.
(a)
Color image (b)
Morphological processing (c)
Identification of color image (d)
Depth image (e)
Morphological processing (f)
Identification of depth image (g)
Infrared image (h)
Morphological processing (i)
Identification of infrared image


### 4.2. Experiments and Results Analysis

However, the method of this paper is to fuse image (e) and image (h) in Figure7 reasonably. It can not only identify the position of obstacles accurately but also avoid the effect of illumination condition effectively. The experimental result of the obstacle detection using fusion of depth and infrared information is shown in Figure 8. The experimental parameters of edge detection and morphology operation are shown in Table 1, in which σR and σD, respectively, represent the value of Canny operator in infrared image and depth image. More edge information is detected when the value is small, but when the value is too small, the rough road is also considered as an obstacle. When the value is too large, the edge information of obstacles cannot be detected, so the value should be selected properly. According to our tests, the best value is 3. RR, RD, and RF, respectively, represent expansion radius in three kinds of images. When the value is large, the expansion of the edge information is more obvious. But when the value is too large, it will make the area of the obstacle detection larger than the actual area. On the contrary, if it is too small, one obstacle will be recognized as multiple obstacles. The values in Table 1 will be selected after several tests. NR, ND, and NF, respectively, represent expansion neighborhood in three kinds of images. Their values have little effect on the experimental results so they are 4 in this experiment. The experimental results show that the parameters used above can identify the obstacles accurately.Table 1
Operator parameters.
Operator Value Canny operatorσ σR 3 σD 3 Inflation factors of infrared image RR 3 NR 4 Inflation factors of depth image RD 2 ND 4 Inflation factors of fusion image RF 3 NF 4Figure 8
Results of obstacle detection.
(a)
Color image (b)
Depth image (c)
The results of depth (d)
Infrared image (e)
The results of infrared (f)
Fusion image (g)
Detection resultAfter foreground segmentation, gradient calculation, edge detection, and morphology operation in depth image, the general outline of obstacle will be conducted, but sometimes the contact surface between the ground and the obstacle cannot be segmented. The foreground edge segmented by mean-shift algorithm will coincide with the edge of obstacles. Besides, there is some other messy identification information. In infrared image, the edge information detected from front objects is abundant. After morphology operation, most areas are identified as obstacles. However, after combining Figures8(c) and 8(e) by intersecting, we can get Figure 8(f). From Figure 8(f), the location of obstacles can be easily identified and marked. The result is shown in Figure 8(g). After experimental verification, the accuracy rate of the method reached 98%, over detection rate is 1.1%, missing rate is 0.6% according to 50 groups images each with 1–3 objects as obstacles. The accuracy of the obstacles identification will not be affected by the illumination. The method of obstacle detection with fusion of depth and infrared information is more superior to the method with single image information. In addition, it has a strong practicality.The results of obstacle detection are shown in Table2. It can be concluded that the distance between the obstacle and the camera is the main factor to the results. Take the can as an example: when the actual distance is 1 m, the error between the measured distance and actual distance is 0.011 m. When the distance reaches 3 m, the error reaches 0.029 m. Because the surface of obstacle is not smooth and sometimes the obstacle is a cluster of objects, the depth value is not constant. The limitation of the camera also contributes to that result. However, such accuracy is relatively high for outdoor robots, smart cars, and so forth. We also find that the smaller obstacle will produce higher measurement accuracy in the scene of road. The reason is that the edge information of small obstacle performs better during the fusion of depth image and infrared image. When the obstacle is large, it will affect the edge information of image and the accuracy of measurement, due to the variant colors in different parts. According to the data from the table, the proposed method can detect the obstacle accurately, and it can be applied to practical problems and holds a significant position for its research value.Table 2
Experimental parameters and results.
The type of scene Obstacle Actual distance/m Actual sizeWide × high/cm Actual geometric center(x,y) Measured distance/m Measured sizeWide × high/cm Measured geometric center(x,y) Road Can 3 13 × 6 ( 253,240 ) 2.971 12.6 × 6.2 ( 247,238 ) 1.5 ( 285,240 ) 1.521 12.9 × 5.8 ( 293,243 ) 1 ( 323,248 ) 1.011 13.4 × 6.2 ( 325,253 ) Car 3 160 × 130 ( 247,275 ) 2.914 142.3 × 123.4 ( 257,279 ) Carton 1 45 × 25 ( 62,246 ) 1.013 45.2 × 22.6 ( 67,255 )  Grassland Carton 1.5 45 × 25 ( 213,192 ) 1.485 45.8 × 26.1 ( 209,199 ) Tree 2.2 14 × 200 ( 72,306 ) 2.221 15.1  × 189.2 ( 84,309 ) Tree and flowers 3 14 × 210 ( 132,341 ) 2.965 13.5 × 293.1 ( 146,344 ) — — — — — — 2.5 68 × 170 ( 142,58 ) 2.481 64.3 × 143.2 ( 132,60 ) Carton and paper basket 1 45 × 25 ( 263,348 ) 1.016 44.1 × 26.1 ( 268,350 ) — — — — — — 1.4 16 × 25 ( 231,126 ) 1.379 18.1 × 22.3 ( 223,121 )


## 4.1. Single Image Identification

The results of obstacle detection via a single image on the grass by Kinect sensor are shown in Figure7. In the picture, the size of carton is 25 cm × 45 cm, and the distance from the sensor is 1 m. The size of the paper baskets is 16 cm × 25 cm, and the distance from the sensor is 1.5 m. The angle of inclination of Kinect sensor is 30°. Since the yellow flowers are far away from the sensor and have over 8 m in color image, their information cannot be collected in depth image and infrared image. We can clearly conclude from the pictures that any results from single image cannot accurately identify the obstacles. In color image, due to the abundant color information and color change, the edge information is really distinct. As shown in Figure 7(b), the edge information of foreground and background both can be detected, but only obstacles in foreground are demanded. So the color image is not suitable for our research. Besides, the obtainment of color map is also easily affected by the illumination condition. When the depth map is used alone, the area edge and obstacle will be easily identified as one object if the obstacle is close to the edge of segmented foreground, which is shown in Figure 7(f). Meanwhile, the edge information is not detected in front regions without obstacles and the error information is less in rear regions, which is shown in Figure 7(e). However, when the infrared image is used alone, the edge information of front obstacles can be detected effectively. But the edge information of farther grass can also be detected and the information of the rear object is ignored, which is shown in Figure 7(h). In a word, if only single image is obtained for detection, it will produce many errors. Accurate obstacles identification cannot be achieved and neither does the robustness.Figure 7
The results of using single image identification.
(a)
Color image (b)
Morphological processing (c)
Identification of color image (d)
Depth image (e)
Morphological processing (f)
Identification of depth image (g)
Infrared image (h)
Morphological processing (i)
Identification of infrared image


## 4.2. Experiments and Results Analysis

However, the method of this paper is to fuse image (e) and image (h) in Figure7 reasonably. It can not only identify the position of obstacles accurately but also avoid the effect of illumination condition effectively. The experimental result of the obstacle detection using fusion of depth and infrared information is shown in Figure 8. The experimental parameters of edge detection and morphology operation are shown in Table 1, in which σR and σD, respectively, represent the value of Canny operator in infrared image and depth image. More edge information is detected when the value is small, but when the value is too small, the rough road is also considered as an obstacle. When the value is too large, the edge information of obstacles cannot be detected, so the value should be selected properly. According to our tests, the best value is 3. RR, RD, and RF, respectively, represent expansion radius in three kinds of images. When the value is large, the expansion of the edge information is more obvious. But when the value is too large, it will make the area of the obstacle detection larger than the actual area. On the contrary, if it is too small, one obstacle will be recognized as multiple obstacles. The values in Table 1 will be selected after several tests. NR, ND, and NF, respectively, represent expansion neighborhood in three kinds of images. Their values have little effect on the experimental results so they are 4 in this experiment. The experimental results show that the parameters used above can identify the obstacles accurately.Table 1
Operator parameters.
Operator Value Canny operatorσ σR 3 σD 3 Inflation factors of infrared image RR 3 NR 4 Inflation factors of depth image RD 2 ND 4 Inflation factors of fusion image RF 3 NF 4Figure 8
Results of obstacle detection.
(a)
Color image (b)
Depth image (c)
The results of depth (d)
Infrared image (e)
The results of infrared (f)
Fusion image (g)
Detection resultAfter foreground segmentation, gradient calculation, edge detection, and morphology operation in depth image, the general outline of obstacle will be conducted, but sometimes the contact surface between the ground and the obstacle cannot be segmented. The foreground edge segmented by mean-shift algorithm will coincide with the edge of obstacles. Besides, there is some other messy identification information. In infrared image, the edge information detected from front objects is abundant. After morphology operation, most areas are identified as obstacles. However, after combining Figures8(c) and 8(e) by intersecting, we can get Figure 8(f). From Figure 8(f), the location of obstacles can be easily identified and marked. The result is shown in Figure 8(g). After experimental verification, the accuracy rate of the method reached 98%, over detection rate is 1.1%, missing rate is 0.6% according to 50 groups images each with 1–3 objects as obstacles. The accuracy of the obstacles identification will not be affected by the illumination. The method of obstacle detection with fusion of depth and infrared information is more superior to the method with single image information. In addition, it has a strong practicality.The results of obstacle detection are shown in Table2. It can be concluded that the distance between the obstacle and the camera is the main factor to the results. Take the can as an example: when the actual distance is 1 m, the error between the measured distance and actual distance is 0.011 m. When the distance reaches 3 m, the error reaches 0.029 m. Because the surface of obstacle is not smooth and sometimes the obstacle is a cluster of objects, the depth value is not constant. The limitation of the camera also contributes to that result. However, such accuracy is relatively high for outdoor robots, smart cars, and so forth. We also find that the smaller obstacle will produce higher measurement accuracy in the scene of road. The reason is that the edge information of small obstacle performs better during the fusion of depth image and infrared image. When the obstacle is large, it will affect the edge information of image and the accuracy of measurement, due to the variant colors in different parts. According to the data from the table, the proposed method can detect the obstacle accurately, and it can be applied to practical problems and holds a significant position for its research value.Table 2
Experimental parameters and results.
The type of scene Obstacle Actual distance/m Actual sizeWide × high/cm Actual geometric center(x,y) Measured distance/m Measured sizeWide × high/cm Measured geometric center(x,y) Road Can 3 13 × 6 ( 253,240 ) 2.971 12.6 × 6.2 ( 247,238 ) 1.5 ( 285,240 ) 1.521 12.9 × 5.8 ( 293,243 ) 1 ( 323,248 ) 1.011 13.4 × 6.2 ( 325,253 ) Car 3 160 × 130 ( 247,275 ) 2.914 142.3 × 123.4 ( 257,279 ) Carton 1 45 × 25 ( 62,246 ) 1.013 45.2 × 22.6 ( 67,255 )  Grassland Carton 1.5 45 × 25 ( 213,192 ) 1.485 45.8 × 26.1 ( 209,199 ) Tree 2.2 14 × 200 ( 72,306 ) 2.221 15.1  × 189.2 ( 84,309 ) Tree and flowers 3 14 × 210 ( 132,341 ) 2.965 13.5 × 293.1 ( 146,344 ) — — — — — — 2.5 68 × 170 ( 142,58 ) 2.481 64.3 × 143.2 ( 132,60 ) Carton and paper basket 1 45 × 25 ( 263,348 ) 1.016 44.1 × 26.1 ( 268,350 ) — — — — — — 1.4 16 × 25 ( 231,126 ) 1.379 18.1 × 22.3 ( 223,121 )


## 5. Conclusions

A simple obstacle detection method based on fusion of depth and infrared image is proposed in this paper. The feature of scene and the information of image are analyzed. According to the character of depth and infrared image in traditional edge detection, a field obstacle detection algorithm based on visual information fusion is proposed, which is used for detecting the obstacle in the near area of the sensor (the foreground mentioned before). It overcomes the defects that the color image is easily influenced by the factors such as light and shadow, and it also overcomes the limitation of the single vision algorithm. Finally, the algorithm effectively removes the error detection area caused by using single vision algorithm. The experiments show that the method can identify the obstacle area accurately and it is not easy to be affected by the factors of light condition and intensity. Furthermore, better robustness is achieved. Hence, it can provide reliable data for intelligent robot.



---

*Source: 2379685-2016-12-04.xml*