# A New Navigation Line Extraction Method for Agriculture Implements Guidance System

Mingxuan Li, Man Zhang, Haiyan Huan, Gang Liu

<figure><img src="image_1.png" /></figure>

## To cite this version:

Mingxuan Li, Man Zhang, Haiyan Huan, Gang Liu. A New Navigation Line Extraction Method for Agriculture Implements Guidance System. 6th Computer and Computing Technologies in Agriculture (CCTA), Oct 2012, Zhangjiajie, China. pp.299-308, 10.1007/978-3-642-36137-1_36. hal-01348245

HAL Id: hal-01348245

https://inria.hal.science/hal-01348245

Submitted on 22 Jul 2016

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>
A new navigation line extraction method for agriculture
implies guidance system

Mingxuan Li¹, Man Zhang¹,¹, Haiyan Huan¹, Gang Liu¹

¹Key Laboratory of Modern Precision Agriculture System Integration Research, Ministry of Education, China Agricultural University, Beijing, 100083, China
Corresponding author: Man Zhang e-mail: cauzm@cau.edu.cn.

**Abstract:** To make up the shortages of the existing vision navigation algorithms such as the noise interference and the processing speed, a new approach for navigation line detection is presented in this article. This new method was based on image boundary detection. The color images were transformed into binary ones after pre-processing. Then, three stages in this method were presented. At first, the image was divided into several sectors and the boundary of the crops was detected. Then a number of points indicating the centers of the rows were determined. At last, the navigation line was estimated by an improved linear regression algorithm. Compared to other navigation line detection algorithms, this new method could find the navigation lines more accurately. It need only about 89ms to process a picture with 640×320. So it can meet the need of agricultural real-time visual navigation when the speed is 1~2m/s.

**Keywords:** Navigation line detection, Machine vision, Agricultural implements navigation.

# 1 Introduction

Steering agricultural machinery within rowed crop fields is a tedious task for producers. Automated guidance of the machinery will not only reduce operator fatigue but also increase both the productivity and safety of the operation[1]. As a machine vision system can detect a path in relation to crop rows, vision-based guidance can be used to guide machinery travelling between crop rows to perform field operations such as cultivating, chemical spraying, and harvesting[2-4].

There are a lot of research have been made in the field of crop line identification. At present, the traditional ways of crop detection method is based on the Hough transformation largely[5-7]. Hough transformation algorithm is a shape matching technology put forward by Paul Hough in 1962. Duda and Hart established the Hough transformation linear testing in 1972. Hough transformation has several advantages such as strong robustness and small noise influence[8]. However, it also has
disadvantages such as a high computational complexity[9]. In order to improve the speed of Hough, Kultanen P. put forward the Random Hough Transformation (RHT) [6, 10] algorithm. RHT is a technology of multi-to-one. Compared to the traditional Hough transformation, RHT depend on the idea of randomness, reducing the computational complexity greatly. But for the RHT algorithm, random point selection is so important that it is unstable to detect the navigation line[11]. In addition, because the navigation line only depends on the two crops lines nearest to the image center, so it still brings some redundant calculation to extract all the crop lines.

In order to solve these questions above, a new method was introduced in this article. Multistage filtering is put forward to solve the detection problem, and result in a lower computational complexity with the similar detection stability than RHT and traditional Hough transformation. Two sections are contained in this new method, the first one is image pre-processing section and the second one is a new baseline extraction algorithm that designed in this research named Boundary Detection Navigation Algorithm.

# 2 Materials and method

## 2.1 Experimental Materials

The experimental platform, as shown in Fig.1, included several parts, such as vision sensor, computer, controller, hydraulic system and the agricultural implement. In this platform, the camera was used to real-time acquire the image of farmland; the computer was used to identify and position of crop line from the acquired image, and then calculate the lateral deviation of the farm machinery relative to the target path; Based on the lateral deviation, the controller is used to control the hydraulic system work and realize the automatic navigation of farm machinery.

<figure><img src="image_3.png" /><figcaption>Fig. 1. The experiment platform</figcaption></figure>

The visual sensor used in this paper is OK AC1310 camera made by Beijing Jiaheng Company. The pixel size is 4.65 µm×4.65µm; the resolution is 1300 × 1024; The frame rate is between 8~24Hz; The shortest exposure time of the electronic
shutter is 20µs. The data transmission between the video camera and the computer through a USB2.0 interface[12].

The navigation program is developed on the computer of Intel(R) Core (TM2) Quad CPU Q9400 @2.66GHz, 2.96GB in the XP operating system. And the development environment is Microsoft Visual C++ 6.0.

## 2.2 Methods

The testing method of the navigation line was divided two sections, the pre-processing section and the navigation line extraction section. The pre-processing section included graying, binarization and filtering, and the navigation line extraction section included division and detection (including image segmentation and boundary detection), points determination (including false points removing and navigation points estimation), and navigation line regression, as shown in the Fig.2.

<figure><img src="image_4.png" /><figcaption>Fig. 2. The flow chart</figcaption></figure>
# 3 Results and Discussions

## 3.1 Image pre-processing

Considering the complexity of the farmland environment, color images were chosen as research objects in this article because rich color information was contained in it[11]. However, color images also brought some problems such as the redundant information, the noise distribution and the complex nature. Thus, the images needed to be preprocessed before navigation line extraction. The image preprocessing contained three segments, namely as graying, binarization, and de-noising.

Usually, the green component G is much greater than red R and blue B components because the main pigment of the crops was chlorophyll. So the 2GRB method was taken to gray images in order to emphasize green component and restrain the rest two components[12]. The brightness and color will different if the image is acquired under different light condition. The OTSU method was used to transform the gray images into binary images[13].

There were a lot of noise points in the image after segmentation because of the characteristics of farmland environment, such as weeds, shadow. The method of corrosion-median filter-expansion was selected to reduce the noise according to the characteristics of the weeds. Firstly, the disc structure with 5 pixel radius was chosen to corrode the images. Secondly, the image was filtered through the 5×5 pixel size windows. Finally, the 5×1 pixel inflation structure elements was adopted to extrude the vertical part of the images. After that, most of the noise was filtered out and the stability of the navigation baseline extraction algorithm would be improved.

The images of bean, corn and winter wheat were collected to verify the effect of the image preprocessing. The results of the bean images were shown in the Fig.3, it showed that the preprocessing algorithm could effectively reduce the influence of weeds in images and get the boundary of the crops.

<figure><img src="image_5.png" /><figcaption>Fig. 3. The results of image processing</figcaption></figure>

## 3.2 The navigation line detection algorithm

Before detecting the row position, a number of points indicating the centre of the rows needed to be determined firstly. In order to obtain these points, the binary image
resulting from grayscale image was divided into a series of horizontal strips as shown in Fig.4, and the next step was to estimate where the rows were intersect each individual strip. In principle, the number of strips should equal to the number of pixel rows in the image. In order to reduce the amount of subsequent computation and enhance the stability of subsequent analysis, it was suggested to let each strip involve more than one pixel row. Thus, the number of strips should be kept at a reasonably low value, and this value was denoted as step. Practical experience showed that every 10 pixels divided into a strip was suitable when the system was working on the experimental field. Each image strip was processed as illustrated in Fig. 4. Mathematically, the process could be described as Fig. 5.

<figure><img src="image_6.png" /><figcaption>Fig. 4. Color image divided into horizontal strips</figcaption></figure>

<figure><img src="image_7.png" /><figcaption>Fig. 5. Steps of the navigation line extraction algorithm</figcaption></figure>

**Step 1: Image Segmentation**

$V_{i,j}$ was used as the binary value (0-black point and 1-black point) of the pixel in position (i,j) of the image strip. The indices i and j denote the pixel row and the pixel column, respectively, (i = 1, 2,..., N; j = 1, 2,..., M; N = Height / step; M = Width). Fig.6 was the first strip of binary image in Fig.4.

**Step 2: Boundary Detection**
$v = (v_1, v_2...v_M)$ was used as a row vector resulting from summing the array $V$ over the index $i$:

$$v_i = \sum_{j=1}^{N} V_{i,j} \qquad i = 1, 2...M \qquad (1)$$

The result of this summation is illustrated by the curve in Fig. 7. The position of non-zero value in Fig. 7 presents the position of crop in Fig. 6. So, Crop position can be determined by detecting the point that exchange between 0 and 1. The $i$th non-zero part is represented by a 2D vector like $P_i=(Pl_i, Pr_i)$. $Pl_i$ was the position of the $i$th non-zero part's left border in a strip, $Pr_i$ is the position of the $i$th non-zero part's right border in a strip.

<figure><img src="image_8.png" /><figcaption>Fig. 6. The first strip of binary image</figcaption></figure>

<figure><img src="image_9.png" /><figcaption>Fig. 7. The number of white points in the first strip</figcaption></figure>

### Step 3: False points removing

However, not all of these non-zero points represents the real crop because of the influence of weeds. An important difference between crop and weed is their scale in the image. By making use of this feature, a criterion to distinguish the crops and weeds could be set. This criterion describes as follows:

The $i$th non-zero part represents the crop when $(Pl_i - Pr_i) \ge \frac{1}{2K} \sum_{j=1}^{K} (Pl_j - Pr_j)$, ($K$ is the number of non-zero parts in a strip), or it represents the weed.

### Step 4: Navigation Points Estimation

After previous step the navigation points $(P_i^{navx}, P_i^{navy})$ can be estimated by the following formula:

$$ 
\left\{ 
\begin{aligned}
P_i^{navx} &= \frac{\min_0(|(\frac{Pl_i + Pl_r}{2}) - \frac{width}{2}|) + \min_1(|(\frac{Pl_i + Pl_r}{2}) - \frac{width}{2}|)}{2} \\
P_i^{navy} &= step * i - \frac{step}{2}
\end{aligned} 
\right. 
\qquad (2) $$
In this formula, the $\min_0(S)$ is the smallest value in $S$, the $\min_1(S)$ is the second smallest value in $S$, $i=1,2...N$, and the navigation point detection result is shown by Fig.8.

<figure><img src="image_10.png" /><figcaption>Fig. 8. The detected Navigation Points</figcaption></figure>

### Step 5: Navigation Line Regression

Based on these navigation points, the navigation line can be calculated by LSE (least-square estimation) algorithm. However, because of the existing of surplus plant or lack of one or more plant in a row, there are some outliers exist in navigation points, and not all of navigation points should be applied by LSE algorithm. We set another criterion to determine whether a navigation point is an outliers as follows:

The ith navigation point is an outliers when $P_i^{\text{navx}} > \alpha\sqrt{\text{var}(P^{\text{navx}})}$, and $\text{var}(P)$ is the variance of $P$. In our experiment the $\alpha$ is 1.5.

Assume the remainder navigation points after removed the outliers are $(P_1^{rx}, P_1^{ry}), (P_2^{rx}, P_2^{ry}), ..., (P_{N_r}^{rx}, P_{N_r}^{ry})$. Based on LSE algorithm, the navigation line parameters can be represented as follows (navigation line can be represented as $y = kx + b$):

$$ \left\{ \begin{aligned} k &= \frac{\overline{P^{rx} P^{ry}} - \overline{P^{rx}} \overline{P^{ry}}}{(\overline{P^{rx}})^2 - (\overline{P^{rx}})^2} \\ b &= \overline{P^{ry}} - k \overline{P^{rx}} \end{aligned} \right. \qquad (3) $$

Where:

$$ \overline{P^{rx}} = \frac{1}{N_r} \sum_{i=1}^{N_r} P_i^{rx}, \quad \overline{P^{ry}} = \frac{1}{N_r} \sum_{i=1}^{N_r} P_i^{ry}, \quad \overline{P^{rx} P^{ry}} = \frac{1}{N_r} \sum_{i=1}^{N_r} P_i^{rx} P_i^{ry}, \quad (\overline{P^{rx}})^2 = \frac{1}{N_r} \sum_{i=1}^{N_r} (P^{rx})^2 $$

The navigation line detection result is shown in Fig.9.
<figure><img src="image_11.png" /><figcaption>Fig. 9. The detected Navigation Line</figcaption></figure>

## 3.3 Evaluation test and discussion

This algorithm was named Boundary Detection (BD) algorithm because it partly depended on crop boundary detection. In order to evaluate the speed and accuracy of this BD algorithm, all traditional Hough Transform (HT) algorithm, Random Hough Transform (RHT) algorithm, and BD algorithm were used to analyze the wheat images. The aim of this test was to extract the center line of the image and calculate the parameters of the navigation line. The result was shown in Fig.10. The line parameter results were shown in Table 1. These lines were represented by the polar coordinate, and the $\rho$ represented the polar diameter, the $\theta$ represented the polar angle.

<table><caption>**Table 1.** The results of linear parameter comparison.</caption><thead><tr><td>Method</td><td>θ</td><td>R</td><td>Time</td></tr></thead><tbody><tr><td>HT</td><td>1.0922</td><td>368.32</td><td>228ms</td></tr><tr><td>RHT</td><td>1.2655</td><td>372.46</td><td>126ms</td></tr><tr><td>BD</td><td>0.8355</td><td>370.58</td><td>89ms</td></tr></tbody></table>

<figure><img src="image_12.png" /><figcaption>Fig. 10. The results of navigation line detection</figcaption></figure>
According to the above analysis, all of these methods could detect the navigation line successfully with acceptable error. From the time-consuming perspective, HT consumed the longest time and RHT algorithm was the second. BD algorithm needed the shortest time, which was 89ms and less than half of the HT. From the precision perspective, the difference of θ value between BD and HT algorithm was 0.2567°, and 0.43° between BD and RHT algorithm. And the difference of ρ value between BD and HT algorithm was 2.26mm, and 1.88mm between RD and RHT algorithm. The accuracy of all these methods could meet the demand of the agricultural navigation, but the BD algorithm required the least time.

## 4 Conclusion

A new navigation line extraction algorithm named BD algorithm was proposed to process the farmland environment images in this article. After images acquisition, firstly the color images were transformed into grayscale images by 2G-R-B method according to the characteristics of the farmland crops. Then the grayscale images were transformed into binary images by the OTSU method, meanwhile, a corrosion-median-expansion filter algorithm was adopted to filter the noise. Finally, the parameters of the navigation line were calculated by the BD algorithm. Theoretical analysis and experimental test results showed that the designed BD algorithm was more efficient than HT and RHT algorithm, which only needed 89ms to process an image in this article. The accuracy of the detected line was higher and could be used as the navigation baseline.

## Acknowledgements

This work was supported by the National Natural Science Fund (Grant No. 30900869), National Technology Support Program (Grant No. 2011BAD20B06), and 863 Program (Grant No. 2012AA101901).

## References

1. Marchant, J. A., Brivot, R.: Real time tracking of plant rows using a Hough transform. J. Real-Time Imaging. 1(02), 363–375 (1995)

2. Bakker, Tijmen, Wouters, Hendrik, Asselt, V., Kees et al: A vision based row detection system for sugar beet. J. Computers and electronics in agriculture. 60(1), 87–95 (2008)

3. Zhang, H.X., Zhang, T.Z., Chen, B.Q.: Detection Algorithm for Orientation Lines Based on Pattern Recognition. J. Transactions of the CSAM. 39(2). 107~111 (2008)

4. Jiang, G.Q., Ke, X., Du, S.F., Chen, J.: Detection Algorithm of Crop Rows Based on Machine Vision and Randomized Method. J. Transactions of the CSAM. 39(11), 85–88 (2008)

5. Chen, B., Tojo, S., Watanabe, K.: Detection algorithm for traveling routes in paddy fields for automated managing machines. J. Transactions of the ASAE. 45 (1), 239–246 (2002)
6. Fernandes, L., Oliveira, M.: Real-time line detection through an improved Hough transform voting scheme. J. Pattern Recognition.41(1),299-314 (2008)

7. Leemans, V., Destain, M.F.: Line cluster detection using a variant of the Hough Transform for culture row localization. J. Image and Vision Computing. 24(5), 541-550 (2006)

8. Jang, G.Q.: Research on Navigation and Localization of Agricultural Machinery Based on Machine Vision. D. Beijing: China Agriculture University (2009)

9. Cao, Q., Wang, K., Li, H.: Detection algorithm for crop target multi-lines of the field image based on machine vision. J. Transactions of the CSAE. 25(12), 107-113 (2009)

10. Chen, Y., Zhang, M., Ma, W.Q., et al: Positioning method of integrated navigation based on GPS and machine vision. J. Transactions of the CSAE. 27(3), 126-130 (2011)

11. Liu, Z.X.: Methods of Navigation Control and Path Recognition in Autonomous Agricultural Vehicle. D. Beijing: China Agriculture University (2011)

12. Chen, J., Jiang, G.Q., Du, S.F., et al: Crop rows detection based on parallel characteristic of crop rows using visual navigation. J. Transactions of the CSAE. 25(12), 107-113 (2009)

13. Soggaard, H., Olsen, H.J.: Determination of crop rows by image analysis without segmentation. J. Computers and Electronics in Agriculture. 38 (2), 141-158(2003)