<figure><img src="image_1.png" /></figure>

HAL
open science

# Path Recognition for Agricultural Robot Vision Navigation under Weed Environment

Peidong Wang, Zhijun Meng, Changhai Luo, Hebo Mei

► To cite this version:

Peidong Wang, Zhijun Meng, Changhai Luo, Hebo Mei. Path Recognition for Agricultural Robot Vision Navigation under Weed Environment. 7th International Conference on Computer and Computing Technologies in Agriculture (CCTA), Sep 2013, Beijing, China. pp.242-248, 10.1007/978-3-642-54344-9_30. hal-01220921

HAL Id: hal-01220921

https://inria.hal.science/hal-01220921

Submitted on 27 Oct 2015

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Path Recognition for Agricultural Robot Vision Navigation under Weed Environment

Peidong Wang<sup>a</sup>, Zhijun Meng<sup>b</sup>, ChangHai Luo<sup>c</sup>, Hebo Mei<sup>d</sup>

Beijing Research Center for Intelligent Agricultural Equipment, Beijing 100097, China

<sup>a</sup>wangpd@nercita.org.cn, <sup>b</sup>mengzj@nercita.org.cn, <sup>c</sup>luoch@nercita.org.cn,

<sup>d</sup>meihb@nercita.org.cn

**Abstract.** In this paper, a path recognition method for agricultural robot vision navigation under weed environment is proposed. The vision navigation is based on color images sampled by a high speed camera. First, the crop and weed information is extracted using an appropriate color feature model to separate the green crop from background; then the image is thresholded, and the noise caused by weed is filtered by deleting small-area objects in the image; the navigation path is extracted through Hough transformation. Experiments are carried out in corn seedling field, and results show that the method can recognize navigation path correctly under weed environment.

**Keywords:** vision navigation, agricultural robot, weed noise filtration, path recognition

## 1 Introduction

Autonomous agricultural robot has many applications such as environment information collection, seeding, fertilizing, spraying, etc. The goal of autonomous robot navigation is to control the trajectory of the robot and keep it along the driving path [1]. Computer vision based navigation has been widely researched recent years due to its advantages such as wide detection range, rich target information, good cost performance and flexibility [2-4].

The key to vision navigation is identifying crop automatically from the images and recognizing the navigation path [5]. Usually, the central line of the crop is extracted as the baseline for agricultural robot vision navigation. But the weed in the field will interfere with the crop information because it often has the same green color as the crop, and causes false navigation path recognition results [6]. Astrand has researched on weeding robot, and used two separate vision system, one is used for navigation and the other is used for weed recognition [7]. Zhao Bo uses neural network algorithm to classify the field environment [8]; the algorithm can classify the weed environment, but it's too time-consuming, and how to recognize the navigation path after environment classification is not specified. Xue Jinlin adopts a variable field-of-view method to guide an agricultural robot [9], which can improve the guiding performance at the end of a crop row. Dong Fuhong has built a row guidance system for an autonomous robot for white asparagus harvesting [10], but the system is specific to asparagus, thus lack of generosity.
In this paper, we propose a path recognition method for agricultural robot vision navigation under weed environment. The areas of the connected components in the image are calculated, and the noise caused by weed is filtered by deleting small-area objects in the image. Experimental results verify the correctness and performance of the proposed method.

## 2 Path recognition method

To recognize the path for agricultural robot vision navigation under weed environment, first we select an appropriate color feature model to extract the green crop and weed information from the image, and separate them from the background; then the image is thresholded, and the noise caused by weed is filtered by deleting small-area objects in the image; the navigation path is extracted through Hough transformation. The method is described in detail below.

### 2.1 Image segmentation based on color feature model

The images used for vision navigation is captured by a color camera. We need to preprocess this image to get useful information [11]. In farmland environment, the crops which are used as guide targets are green, while the soil background is usually not green. In order to extract the crop information and eliminate background information, we need to select an appropriate color feature to raise the weight of the green channel of the color image. In this paper, the color feature model is chosen as below:

$$I = 2G - R - B \quad (1)$$

where R, G and B denotes the red, green and blue channel of the color image respectively. This color feature can raise the weight of the green channel, and enhance the contrast ratio with the non-green background.

After calculating the color feature I, a grayscale image is obtained. We segment the grayscale image using maximum between-class variance method to get a binary image BW for future processing.

### 2.2 Weed interference filtration

Due to the fact that weed has the same green color as the crop, there is still weed information remained on the binary image after image segmentation. The weed information may interfere with the useful crop information, and cause false navigation path recognition result. Therefore, we need to filter the weed interference.

In farmland field, the crops are always cluttered in line, while the weeds are scattered separately between crop lines in the field, and the weeds are usually smaller that crops. In the actual image, the crops are represented as connected regions with big areas, while the weeds are represented as small-area objects. Therefore, we use small-area object deletion method to filter the weed interference. The steps of the method are as follows:

1) Determine the connected components in the binary image BW. The result is
represented as Cᵢ, where i = 1, 2...N;

2) Compute the area of each component, which is represented as Sᵢ.

3) Remove small-area objects. The result is represented as: L = ∪ { Cᵢ | Sᵢ >= P }.
where P is the area threshold.

After eliminating the weed interference, the navigation path based on crop center line can be further extracted.

## 2.3 Navigation path extraction

The crop center line is regarded as the navigation path. In field environment, there may be crop line discontinuities due to miss-seeding. The Hough transformation is a global and robust line detection method, and can be used to find and link line segments in an image. Therefore, we adopt Hough transformation to extract the crop center line. The steps of the algorithm are as follows:

1) Calculate the image size.

2) Determine the size of the parameter space for Hough transformation based on image size, discretize the ρ - θ parameter space as many grids, where one grid represents an accumulator. Allocate memory for the grids.

3) For each point (xᵢ, yᵢ) in the image, substitute the discretized θ into
ρ = xᵢ cosθ + yᵢ sinθ to calculate the corresponding ρ, and the result
falls on a specific grid; Add the accumulator corresponding to that grid by 1.

4) Detect the peak value of the accumulators. Estimate the correctness of the peak value detection through navigation path direction angle statistics.

5) Plot the navigation path in the image according to the detected peak value.

## 3 Experiment verification and results

To test and verify the method proposed in this paper, experiments are carried out in corn field with weed. The type of the camera used in the experiments is Monta G-125C from Allied Vision Technology Inc. The resolution of the camera is 1296*964, and the hardware interface of the camera is Gig E. The camera is fixed on Seekur robot made by MobileRobots Inc.

Figure 1(a) is an original color image obtained in corn field with weed. Figure 1(b) is the grayscale image obtained based on color feature model described in Section 2.1, from which we can see that the soil background is successful eliminated from the image after transformation, but the weed information is still remained between crop lines on the image, and this interferes with the useful crop information.
<figure><img src="image_3.png" /><figcaption>(a) Original color Image of corn field</figcaption></figure>

<figure><img src="image_4.png" /><figcaption>(b) Image after grey level transformation</figcaption></figure>

**Fig. 1.** Corn field images before and after grey level transformation

The binary image obtained by thresholding the grey scale image is shown in Figure 2(a), from which we can see the weed noise more clearly. Figure 2(b) shows the image after filtering the weed noise through small-area objects deletion, we can see that most of the weeds are eliminated from the image, only a few weed clutters are remained, but this does not affect the result of Hough transformation.
<figure><img src="image_5.png" /><figcaption>Fig. 2. Binary images after thresholding and weed noise filtration</figcaption></figure>

Figure 3 shows the crop center lines extracted through Hough transformation, and the navigation path calculated based on the crop center lines. The magenta lines represent crop center lines, and the red line represents the navigation path. The two crop lines around the image center are selected to calculate the navigation path. We can see that the crop center lines and the navigation path are correctly extracted. Similar experiments are carried out, and all of the results verify the correctness and performance of the proposed method.
<figure><img src="image_6.png" /><figcaption>(a) Navigation path shown in original image</figcaption></figure>

<figure><img src="image_7.png" /><figcaption>(b) Navigation path shown in binary image.</figcaption></figure>

<caption>**Fig. 3.** Navigation paths show in different images</caption>

## 4 Conclusions

Computer vision based robot navigation has been a major topic of discussion in precision agriculture, and path recognition under noise environment is a key factor to precise navigation. In this paper, we propose a path recognition method for agricultural robot vision navigation under weed environment. The soil background is eliminated by image segmentation based on color component. The weed noise is filtered by deleting small-area objects in the binary image. The crop centerlines and navigation path are extracted through Hough transformation. Experimental results showed that most of the weed noise is eliminated from the field image. The path
recognition method in this paper is practical and accurate for vision-based robot
navigation.

The method proposed in this paper is aimed at straight-line navigation path, which
is of little curvature. Further research will be focused on navigation paths with big
curvature.

Acknowledgment

Funds for this research was provided by the National High Technology Research and Development Programs of China (863 Program), under Contract 2013AA040401 and Contract 2013AA102308.

References

1. J. Backman, T. Oksanen, A. Visala. Navigation system for agricultural machines: Nonlinear Model Predictive path tracking [J]. Computers and Electronics in Agriculture, 2012, 82: 32–43.

2. Reid J F, Zhang Q, Noguchi N, et al. Agricultural automatic guidance research in North America [J]. Computers and Electronics in Agriculture, 2000, 25: 155–167.

3. Keicher R, Seufert H. Automatic guidance for agricultural vehicles in Europe [J]. Computers and Electronics in Agriculture, 2000, 25:169–194.

4. Torii T. Research in autonomous agriculture vehicles in Japan [J]. Computers and Electronics in Agriculture, 2000, 25: 133–153.

5. Toru TORII, et al. Crop row tracking by an autonomous vehicle using machine vision [J]. JSAE, 2000, 62(5):37–42.

6. Xavier P.Burgos-Artizzu, Angela Ribeiro, Maria Guijarro, et al. Real-time image processing for crop/weed discrimination in maize fields [J]. Computers and Electronics in Agriculture, 2011, 75: 337–346.

7. Bjorn Astrand, Albert-Jan Baerveldt. An agricultural mobile robot with vision-based perception for mechanical weed control [J]. Autonomous Robots, 2002, 13: 21–35.

8. Zhao B, Wang M, Mao E, et al. Recognition and classification for vision navigation application environment of agricultural vehicle [J]. Transactions of CSAM, 2009, 40(7): 166–170.

9. Xue JL, Zhang L, Tony E. Grift. Variable field-of-view machine vision based row guidance of an agricultural robot [J]. Computers and Electronics in Agriculture, 2012, 84: 85–91.

10. Fuhong Dong, Wolfgang Heinemann, Roland Kasper. Development of a row guidance system for an autonomous robot for white asparagus harvesting [J]. Computers and Electronics in Agriculture, 2011, 79: 216–225.

11. Pinto F A C, Reid J F, Zhang Q. Vehicle guidance parameter determination from crop row images using principal component analysis [J]. Journal of Agricultural Engineering Research, 2000, 75(3): 257–264.