<figure><img src="image_1.png" /></figure>

HAL
open science

# Design and Implementation of an Automatic Grading System of Diced Potatoes Based on Machine Vision

Chaopeng Wang, Wenqian Huang, Baohua Zhang, Jingjing Yang, Man Qian, Shuxiang Fan, Liping Chen

<figure><img src="image_2.png" /></figure>

## To cite this version:

Chaopeng Wang, Wenqian Huang, Baohua Zhang, Jingjing Yang, Man Qian, et al., Design and Implementation of an Automatic Grading System of Diced Potatoes Based on Machine Vision. 9th International Conference on Computer and Computing Technologies in Agriculture (CCTA), Sep 2015, Beijing, China. pp.202-216, 10.1007/978-3-319-48354-2_22. hal-01614202

HAL Id: hal-01614202

https://inria.hal.science/hal-01614202

Submitted on 10 Oct 2017

HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

<figure><img src="image_3.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Design and Implementation of an Automatic Grading System of Diced Potatoes Based on Machine Vision

Chaopeng Wang,<sup>1,2,3,4,5,a</sup> Wenqian Huang,<sup>2,3,4,5,b</sup> Baohua Zhang,<sup>2,3,4,5,c</sup>

Jingjing Yang,<sup>2,3,4,5,d</sup> Man Qian,<sup>1,2,3,4,5,e</sup> Shuxiang Fan,<sup>1,2,3,4,5,f</sup> Liping Chen,<sup>1,2,3,4,5,g,*</sup>

<sup>1</sup>College of Mechanical and Electronic Engineering, Northwest A&F University, Yangling712100,China;

<sup>2</sup>Beijing Research Center of Intelligent Equipment for Agriculture, Beijing 100097, China

<sup>3</sup>National Research Center of Intelligent Equipment for Agriculture, Beijing 100097, China

<sup>4</sup>Key Laboratory of Agri-informatics, Ministry of Agriculture, Beijing 100097, China

<sup>5</sup>Beijing Key Laboratory of Intelligent Equipment Technology for Agriculture, Beijing 100097, China

<sup>a</sup>wangcp_nwsuaf@163.com, <sup>b</sup>huangwq@nercita.org.cn, <sup>c</sup>baobaopost@126.com, <sup>d</sup>yangjj@nercita.org.cn, <sup>e</sup>qianman101504@163.com, <sup>f</sup>fanshuxiang1989@163.com, <sup>g</sup>chenlp@nercita.org.cn

**Abstract.** Potato is one of the most important crops in the world. In recent years, potato and its processed products have gradually become important trade goods. As an important semi-manufactured product, diced potatoes need to be graded according to their three-dimensional (3D) size and shape before trading. 3D information inspection manually is a time-consuming and labor intensive work. A novel automatic grading system based on computer vision and near-infrared linear-array structured lighting was proposed in this paper. Two-dimensional size and shape information were extracted from RGB images, and height information was measured in NIR images combined with structured lighting. Then, a pair of pseudo-color and gray level height map images fusing with 3D size and shape information was constructed. Finally, diced potatoes were classified into either regular or irregular class according to their 3D information and criteria required by the industry. The grading system and proposed algorithm were testified by a total of 400 diced potatoes with different size and shapes. The test results showed that the detection error was in the range of about 1 mm, and the classification accuracy was 98%. The results indicated that the system and algorithm was efficient and suitable for the 3D characteristic inspection of diced potatoes.

**Keywords:** Computer vision, Diced potatoes grading, Image processing, Structured light, Three-dimensional measurement

## 1 Introduction

Potato is the World's and China's fourth largest staple crop after rice, wheat and maize, it is also one of the most promising high-yield crops in China [1]. With the rapid development of global agriculture, potato and its processed products have great market potential and become a primary part of the global agricultural trade as well.
Consumers care more and more information about the products they bought, and well-informed high-quality products are easier to arouse consumers' interest and stimulate their purchasing desire [2]. For fruits and vegetables, appearance is one of the most important sensory quality attributes, it would not only influences the packaging and retail price, but also affects consumers' preferences and choice. Products with perfect appearance are always receive more favor of consumers favor and would have a better sales appeal[3,4]. Diced potato as a kind of semi-manufacture in trade has strict requirements in size and shape. The irregular shape of diced potato not only deteriorates its appearance, but also has effects on the market value. Therefore, it is necessary to make classification for diced potatoes according to the 3D information before packaging or trading. In practice, diced potatoes are classified by human visual inspection, and 3D information inspection manually is a labor intensive work, it was also time-consuming. Manual processing pose added problems of maintaining the consistency and uniformity in grading [5,6].

Currently, with the improvement of image processing, quality control with computer vision has been an important technology [7,8]. For the requirement of speedy and real time, computer vision system has being developed as a significant part in quality detection and evaluation [9,10]. Over the past several years, the on-line inspection system based on computer vision are widely used in realization of automatic detection of many different agricultural products, including the external and internal quality detection, but automatic classification for diced potato according to their 3D information is still not available. There is an increasing demand for on-line detection equipment based on computer vision that can mimic the human grading and realize accurate classification of diced potato with the unified standard to address the above issues with human visual inspection for food manufacturers.

With photo-electronics, image processing and computer technique rapid development, structured light vision technology has been widely used in computer vision system for 3D reconstructionandautomatic measurement successfully [11]. There have been several techniques proposed with quite different characteristics for accurate measurement. In order to get the height of seedling, Feng et al. designed a automatic inspection system based on structured light vision, and result shows the height measurement error was less than 5 mm for the normally straight seedling [12]. However, the detection precision could not satisfy the requirement of diced potato inspection. Binocular visual was also extensive used for 3D size measurement. A binocular structure-light scanner was constructed to acquire the surface detail information of the work-pieces by Liu et al., and it was suitable for precision and efficiency demands of the large work-piece measurement [13]. However, 3D measurement based on binocular stereovision systems is not suitable used for onlinedetection of diced potato just because it is time consume and the complexity of stereo matching. Coded structured light systems were commonly used for real-time acquisition of 3D surface data and it was considered as an important and widely used active shape acquisition technique [14,15]. The encoded pattern palys a dominate role in the system, it could affect all the measurement performances, including the time consumptionandaccuracy. Xu et al. proposed one-shot pattern method for 3D shape measurement result shows that the accuracy can achieve 0.18 mm. With the system,moving object could be inspected, and it can be implemented for automotive production lines[16]. A principle of uniquely color-encoded pattern projection was
proposed by Chen et al. [17] to design a color matrix for improving the reconstruction efficiency based on coded structured light system. By using such a light pattern, it could realize the 3D vision reconstruction from a single image and accomplish the reliable and accurate measurement for scene objects. It could also be used in dynamic environment for real-time application. However, encoded structured light system is complex and the light pattern has important effect on measurement accuracy.

To realize automatic on-line classification of diced potatoes, a grading system based on computer vision with high accuracy and reliability performance is needed. For this purpose, intensive research works are being conducted to design and build a flexible, reliable and effective computer vision system by using a monocular stationary camera and a near-infrared linear-array structured lighting.

## 2 Objectives

In order to develop a real time grading system for diced potato classification by using computer vision and near-infrared linear-array structured lighting. To actualize this objective, several steps have to be completed: (1) RGB and NIR images synchronous acquisition through the same optical path of the camera for 3D information inspection. (2) 2D size measurement in RGB images for width and length inspection. (3) 2D shape feature (rectangle degree) extraction from the RGB images for contour shape inspection. (4) Height map (pseudo-color image and gray level images) construction according to the height information extracted from NIR images by using near-infrared light. (5) Evenness evaluation by using Gaussian distribution in height maps (gray level images). (6) Developing an efficient image processing algorithm based on computer vision system and near-infrared structured light to classify the diced potato into either regular or irregular class.

## 3 Materials and methods

### 3.1 Diced potato samples

Fresh potatoes from a market in Beijing, were selected and processed into diced potatoes for study. The shape of some diced potatoes were influenced by the surface of potato, as a result, the contour shape of these diced potatoes would be triangle or other irregular shape, and the surface would be slant or rugged. Diced potatoes were classified into either regular or irregular class according to their 3D information and criteria required by the packinghouse. Namely, regular diced potatoes, which were traded at a higher price, can meet all the requirements of the geometric parameters, and irregular that doesn't satisfy at least one of the criteria. In our study, 17 potatoes were processed into 400 diced potatoes (length, width and height were about 15 mm for regulars, the geometries were uncertain for irregulars), including 270 regular and 130 irregular samples.
## 3.2 Computer vision system

All the samples were inspected and classified by using the vision system as shown in Fig. 1(a). The vision system used in our study issameas the system described in [2]. It mainly consists of a 2CCD camera (JAI AD-080GE), a near-infrared linear-array structured lighting (800nm, 200mw), a lighting system (LED light), a computer, a conveyor belt driven by a stepper motor. The multi-spectral camera installed right above the conveyor belt was connected to network ports of computer with two RJ45 twisted-pairs as show in Fig. 1 (b). It can acquire both NIR (800nm) and RGB image through the same optical path simultaneously. The structured lighting was mounted on the upper left of conveyor belt and in the same horizontal plane with camera. The pair of two visible LED light source were distributed symmetrically at the both upper sides of conveyor for light supplement. The whole system was placed in a black box to prevent the interference from outside. In the process of 3D information detection, the image acquisition, image processing, final diced potatoes classification and conveyor belt control panel proposed in this paper was developed in MFC combined with OpenCV.

The conveyor belt consists of inferior smooth material belt and stepper motor was used to transmit diced potatoes for the on-line detection. The specular reflection of projection light strip would be significantly reduced due to the inferior smooth surface of belt. And this would make it easy to extract the light strip and reduce the inspection error. The stepper motor was controlled by the driver and controller which were connected to computer through RS232 serial port to control the speed and other motion parameters of conveyor belt.

<figure><img src="image_4.png" /><figcaption>Fig.1.Schematic illustration of computer vision system
(a)Diagram of the computer vision system used in this research (b) Connection diagram of monocular camera and computer</figcaption></figure>

## 3.3 2D shape inspection methods

Contour shape detection plays a domination role in 2D shape extraction. The majority of diced potatoes are regular, and the contour shape of qualified diced potato is
regular rectangle. For others, they are influenced by the surface of potato, and the contour shape would be irregular. If the contour shape was not rectangle, the diced potato under detection would be judged as irregular directly and other shape information was not needed. Rectangle degree is a significant index to measure the contour shape of rectangular objects. Rectangle degree reflects the filling degree of an object to its external rectangle and rectangle factor can be used to distinguish whether the aim region is rectangle[18]. Therefore rectangle degree was used for the contour shape judgment with minimum circumscribed rectangle method in this paper.

$$R = S_0/S_{MER} (1)$$

Where, R represents the rectangle factor, S₀ represents the area of diced potato's surface imaged by camera, Sₘₐ₰ represents the area of minimum circumscribed rectangle. For rectangular objects, the rectangle factor is infinite closed to 1, but for other objects, the rectangle factors vary from 0 to 1.

### 3.43D size inspection methods

<figure><img src="image_5.png" /><figcaption>Fig. 2. The theory of the height measurement based on triangulation principle</figcaption></figure>

The inspection principle of height information based on triangulation theory is shown in Fig. 2. The conveyor belt is regarded as the reference plane, and it is configured to be parallel with the baseline of the camera and laser projector. The theory of height measurement can be explained by similar triangles △ABP and △CDP. According to the triangle similarity, height of object can be calculated by:

$$\frac{h}{L-h} = \frac{d}{s} (2)$$

Where, h represents the distance from point Q to C, and it also represents the height of object; d represents the distance between points C and D which can be extracted by image processing; L represents the vertical distance from the baseline of camera and laser projector to the conveyor belt; s represents the baseline distance from the laser projector to the CCD camera; L and s can be measured directly. Equation (2) can be equally transformed into the format as:

$$h = \frac{L-h}{s} d \qquad (3)$$

Generally, the height of diced potato is lower than 20 mm, but the distance between baseline of camera and laser projector to the conveyor belt has exceeded 500 mm, large difference existed between them in magnitude. So equation (3) can be simplified as:
$$h = \frac{L}{s} d(4)$$

For height measurement, the distortion distance *d*' caused by height of diced potato should be extracted for offset distance *d* detection. And it can be calculated by the distance between the projecting light stripes in inspected and reference images. The x-coordinate of point Q(x, y) is same of point C(x, y) due to the light strip is parallel to the y-axis of camera image plane, and it can be obtained before inspection. For points B, P, and D is collinear, and it is the same pixel for both reference plane point D and inspected part point P in the camera image plane, point P can be easily detected by image processing. A reference image with straight light strip should be acquired before inspection to obtain the reference coordinates Q(x, y), for which the coordinate position is fixed. In height measurement process, when diced potatoes were transmitted through the view of camera, NIR images with the distortional light strip should be acquired to get the information of detection coordinates P(x, y). The difference between x-coordinate of points P and Q could represent the distortion distance *d*' due to points P and Q were collinear, they had the same y-coordinate.

Both RGB and NIR images were used for 3D size detection. RGB images were processed for the 2D size measurement. With minimum circumscribed rectangle method, width and length of diced potato could be simply and conveniently detected in a small error range. NIR images were processed for height measurement. Fig. 3 shows NIR images acquired by the monocular camera. When nothing is under detection, the light strip would be continuous without any distortion for the near-infrared structured light is just throwing lighting onto reference plane as shown in Fig. 3(a). But when diced potato is transmitted and through the vision system, parts of the light strip would project on the surface of it, and the light strip would be disconnected as shown in Fig. 3(b). The offset distance of light strip would change with the height of diced potato, so it is important to get the relationship between diced potato's actual height and the offset distance of light strip for height measurement.

<figure><img src="image_6.png" /><figcaption>**Fig. 3.** NIR images acquired by AD080-GE
(a) Reference image (b) Detection image</figcaption></figure>

Before the height measurement, threshold segmentation was used to carry on binary processing for NIR image. And then, light strip in binary images should be refined for pixel coordinate extraction. An applicable image thinning algorithm not only need to provide high inspection accuracy but also should has high efficient that could complete the image processing in the time interval between two images was acquired (the maximum frame rate of AD-080GE is 30 fps). In our research, the centerline extraction method was used for the image thinning. The center
pointcoordinatesO(x₀, y₀)of light strip in each line of binary image can be calculated using equations (5):

$$x_0 = \frac{1}{A} \sum_{(x,y) \in R} x y_0 = \frac{1}{A} \sum_{(x,y) \in R} y(5)$$

Where, R represents the pixels set of light strip in each line of binary image; A represents the size of pixels set; x and y represents the x-coordinate and y-coordinate of each pixel in light strip.

NIR images were detected line-by-line to get pixel coordinates of the centerline which could represent the position of light strip. After thinning of the binary images, the centerline coordinates could be extracted, and distortion distance d' could be calculated by:

$$d' = d_0 - d_1(6)$$

Where, d₀ represents the centerline coordinates of reference image, d₁ represents the centerline coordinates of inspected image.

In our research, model materials (50 mm long, 30 mm wide and 2 mm high) were used layer by layer and formed different heights for the data sampling. Cubic spline interpolation was used to establish the relationship between offset distance d and distortion distance d' by using Matlab 2010. When d' were inspected by visual inspection system, the offset distance of light strip would be got by the fitting function, which could represent the relationship. Finally, the real height of diced potato h would be calculated with triangulation theory proposed above.

### 3.5 Surface evenness detection

For further research, height map, including pseudo-color and gray level images were constructed according to the distortion distance d' in the process of height measurement. The pseudo-color images could make it easier for human eyes to evaluate the performance of the height image, and the gray level images were used for the diced potato's surface evenness detection. The surface evenness was assessed based on the grey scale distribution of height map. The average and mean square error of d' could be calculated by the grey scale distribution, which could be extracted by the image processing of gray level image. Gaussian model was used for surface evenness detection, and the evenness was determined by the ucl (upper control limit) and lcl (lower control limit) of d' using equations (7) & (8):

$$ucl = \mu + \sigma(7)$$

$$lcl = \mu - \sigma(8)$$

Where, μ represents the average of d', σ represents the mean square error of d'. Due to different distortion distance d' corresponds to different gray scale in gray level image, μ and σ can be determined using equations (9) & (10):

$$\mu = \sum_{i=1}^{255} i * p_{(i)} \qquad (9)$$

$$\sigma = \sqrt{\frac{1}{255} \sum_{i=1}^{255} (i - \mu)^2} \qquad (10)$$
Where, *i* represents the gray scale of gray level image from 1 to 255, gray scale 0 is preclusive due to it is defined as the background (black) of the height image; *p*<sub>(i)</sub> represents the appearing probability of each gray scale appeared in height map image.

As shown in Fig. 4, the shadow represents the gray scales in the scope of *lcl* and *ucl*.
If the total probabilities of gray scales in the shadow area is higher than a specified
threshold, the surface evenness would be judged as eligible.

<figure><img src="image_7.png" /><figcaption>Fig. 4.Control limits for surface evenness detection</figcaption></figure>

**3.6Whole classification algorithm for 3D size and shape inspection**

In this paper, 3D geometry characteristics of diced potatoes were extracted by
using a combination of structured lighting method, triangulation theory and minimum
circumscribed rectangle method. The image processing and shape classification
algorithm mainly include the following steps: (1) Calibration: NIR image in which the
light strip was uninterrupted was acquired before shape detection to get reference
pixel coordinates; (2) 2D size measurement: RGB image was processed to get the
length and width in pixels with minimum circumscribed rectangle method. Then the
length and width of diced potatoes would be figured out with the parameters of
camera and other parameters. (3) 2D shape feature extraction: RGB images were
processed to get the rectangle degree of diced potatoes, according which to judge if
the contour shape of diced potato was regular rectangle. (4) Height map construction:
Pseudo-color images and gray level images which could represent the height
information of diced potato were constructed simultaneous depend on the distortion
distance *d*' between the projection light stripes in inspected and reference images. (5)
Surface evenness evaluation: The grey-histogram of height map (gray level image)
was created to get average of *d*' and mean square error of *d*', and then judge if the
surface of diced potato was even with Gaussian model. (6) Grade judgment: All the
shape parameters inspected through above steps were compared with the parameters
of 3D characteristic information set according to the factory requirements for
classification. If the geometrical information measured by computer vision system
satisfied requirements, Flag would be defined as TRUE, and diced potato under
detection would be regarded as regular. Otherwise, the Flag would be defined as
FLASE and the diced potato would be regarded as irregular. Finally, diced potatoes
were classified into either regular or irregular class. The flowchart of the whole
classification algorithm is shown in Fig. 5.
<figure><img src="image_8.png" /><figcaption>Fig. 5.Flowchart of the classification algorithm for diced potato</figcaption></figure>

## 4 Results and discussion

The automatic control software based on computer vision is shown in Fig. 6. The on-line inspection of diced potato would be conducted according to the classification algorithm mentioned above. Inspection results, including 3D size and classification information were shown in real time. Meanwhile, the height map images could be constructed with the moving of diced potatoes, and the pixels would be assigned to different color according to their height information and color bar.

<figure><img src="image_9.png" /><figcaption>Fig. 6. The inspection system developed in VC++ 2010 for diced potato grading</figcaption></figure>

### 4.1 Results of light strip thinning and smoothing

In the process of height measurement, it is important to extract the centerline of light strip in NIR image. Height information of each pixel in the light stripe could be calculated using the measurement principle of the triangulation method. The result of
light strip thinning and smoothing was shown in Fig. 7. Fig. 7(a) is the NIR image with light strip projecting on the surface of diced potato. As shown in Fig. 7(a), a light stripe with distortion was projected onto the scene and imaged by the camera, and it is obvious that the width of light strip on the surface of diced potato increases due to the light scattering. Fig. 7(b) shows the results of light strip thinning. As shown in Fig. 7(b), the light strip is thinned as a very thin line with the width of only one pixel using the centerline extraction method. Fig. 7(c) shows the distortion distance *d*′ of each pixel in the centerline. The 3D profile shape of diced potatoes could be constructed by connecting all the distortion distance of the pixels in the centerline. Veining defects were often observed around light strip on the surface of reference plane or diced potato due to the slight scattered light of conveyor belt and instability of the laser projector. As shown in Fig. 7(c), the data after image thinning were not smooth and sometime it would influence the detection results. For this reason, median filtering was used to smooth the original data of distortion distance *d*′ on account of the simple algorithm and better processing performance compared with other processing methods. Fig. 7(d) shows the result of data after media filtering. As shown in Fig. 7(d), most of the burrs are eliminated, especially the data in area of reference plane.

<figure><img src="image_10.png" /><figcaption>Fig. 7.The result of light strip thinning and data smoothing
(a)Detection image (b) Result of image thinning (c) Original data of distortion distance d' (d) Results data after median filtering</figcaption></figure>

## 4.2 Detection result

Contour shape and surface evenness inspection were used to judge if the diced potato satisfied the dimensional requirements was regular pattern. In order to detect the surface evenness, pseudo-color images and gray level images were constructed as shown in Fig. 8. Fig. 8(a) is the RGB image of regular diced potato, Fig. 8(b) and 8(c) are the pseudo-color image and gray level image constructed by the vision system according to the 3D information, Fig. 8(d) is the gray histogram which could indicate the grey scale distribution of the gray level image. Images in the second row of Fig. 8
is the relevant images of an irregular diced potato. As shown in Fig. 8(f) and 8(g), the height map were distorted due to the slant or rugged surface, and the surface area of irregular diced potato was larger than the surface area of irregular diced potato. Compared Fig. 8(d) with 8(h), it is obvious that, if diced potato under detection was regular, the grey scale distribution would be relatively concentrated and its' distribution trends are similar to Gaussian distribution. Otherwise, the grey scale distribution would be dispersive for irregular diced potato with the uneven surface.

<figure><img src="image_11.png" /><figcaption>**Fig. 8.** Detection results of Regular and irregular diced potato detection
(a) RGB image of regular diced potato; (b) Pseudo-color image of regular diced potato; (c) Gray level image of regular diced potato; (d) Gray histogram of regular diced potato; (e) RGB image of irregular diced potato; (f) Pseudo-color image of irregular diced potato; (g) Gray level image of irregular diced potato; (h) Gray histogram of irregular diced potato</figcaption></figure>

In order to observe the classification effect, and testify detection algorithm proposed in this paper, the inspection results of different classes of diced potatoes were randomly selected. Measuring error was extracted by comparing the 3D size detected by the grading system with the real value measured by digital caliper. The measurement results, including length, width and height show minor difference (in range of 1mm) contrasted with the real value.

Fig. 9 shows the detection results of geometric parameters. Fig. 9(a), 9(b) and 9(c) shows the relationship of real values and measured values of length, width and height, respectively. The corresponding $R^2$ (coefficient of determination) is 0.9072, 0.8939 and 0.8968, this could meets requirements for the classification of diced potatoes. The rectangle degree and Gaussian probability were the other two important parameters to judge if diced potato was regular or not. For most irregular diced potatoes, these two parameters were influenced by the surface of potato. There has a relationship between the contour shape and surface evenness, it can be represented by the correlation between rectangle degree and Gaussian probability. As shown in Fig. 9(d), it is obvious that the rectangle degree of regular diced potato is higher than 0.9. By contrast, the rectangle degree of irregular diced potato is disperse because of the various contour shape, and for majority of them, the rectangle degree is lower than 0.9. The Gaussian probability of regular diced potato is higher than 0.68, and it is lower than 0.68 for major irregular diced potatoes due to their uneven surface. Therefore, 0.68 can be used as the decision condition for the surface evenness judgment. For minority irregular diced potatoes, the Gaussian probability is higher than 0.68, but 3D
size and rectangle degree could not meet the requirements, it is greatly reduced the
inspection error probability. In Fig. 9 (d), there have many regular diced potatoes with
the same rectangle and Gaussian probability, therefore some overlap was also
observed.

<figure><img src="image_12.png" /><figcaption>**Fig. 9.Detection results of geometrical characteristics**
(a) Comparison of measured length and real length; (b) Comparison of measured width and real width; (c) Comparison of measured height and real height; (d) Distribution of Gaussian probability and rectangular ratio for regular and irregular diced potato</figcaption></figure>

The image processing mainly includes minimum circumscribed rectangle detection and height map construction aim at getting the 3D geometrical characteristics of diced potato. The intermediate process result images are shown in Fig. 10. Diced potatoes with different shape as shown in Row(a), all the geometrical information of sample 1 and 2 are regular and could satisfy the criteria, and other samples are irregular. Images in Row (b) are the detection results with minimum circumscribed rectangle method. Height map including pseudo-color and gray level image constructed in the process of height measurement are shown in Row (c) and Row (d), different height is denoted as different pixel values.

The irregular diced potatoes mainly include 3 types: (1) Geometrical feature is regular, but 3D size would not satisfy the requirements as shown in sample 3. Though the contour shape is regular rectangle and the surface is evenness, the height could not meet requirements. This is obvious when compared the height map of sample 3 with sample 1 or 2, especially in pseudo-color image. (2) Both of the contour shape and surface evenness could not satisfy the requirements as shown in sample 4, this would be classified as irregular directly with rectangle degree detection. (3) The contour shape of diced potato is regular rectangle, but the surface is slant or rugged, as shown
in sample 5 or 6. It is obvious to observe the trend of the surface height change with
the height map.

<figure><img src="image_13.png" /><figcaption>**Fig. 10.** Results of diced potatoes with deferent shape
(a) RGB images; (b) Results of minimum circumscribed rectangle method; (c) Pseudo-color images; (d) Gray level image</figcaption></figure>

**5Conclusions**

Real-time, on-line, low-cost computer vision system is urgent needed to replace the
manual inspection for the classification of diced potato. An automatic grading system
based on computer vision and near-infrared linear-array structured lighting was
proposed in this paper. In order to reduce the interference of external conditions and
make the inspection results more precise and stable, visible and near-infrared linear-
array structured lighting were used respectively for contrast tests to select the
projector with higher detection speed and accuracy. Meanwhile, convey belt with
inferior smooth material was used for diced potatoes transmission. The matt surface
would reduce the specular reflection of projection light strip significantly and this
would make it easy to extract the light strip. In this research, RGB and NIR images
acquired simultaneously by a monocular camera were processed for 3D geometrical
characteristics extraction, including physical dimension, contour shape and surface
evenness, which were used as the main decision conditions for the classification.
Minimum circumscribed rectangle method and structured light measurement method
were used for the 3D size inspection, meanwhile, height map include pseudo-color
image and gray level image were constructed according to the height of diced potato
for the surface evenness judgment with Gaussian module. The results demonstrate the
detection error of the grading system was in range of about 1 mm, and the
classification accuracy has reached 98%. This preliminary research verified the
possibility and feasibility of using computer vision combined with structure light
measurement method for the classification of diced potato.
## Acknowledgment

The authors gratefully acknowledge the financial support provided by National Key Technology R&D Program (No. 2014BAD21B01).

## References

1. Qu, D. Y., Xie, K. Y., Jin, L. P., et al. Development of potato industry and food security in China[J]. Scientia Agricultura Sinica, 2005, 38(2):358-362.

2. Zhang, B. H., Huang, W. Q., Wang, C. P., et al. Computer vision recognition of stem and calyx in apples using near-infrared linear-array structured light and 3Dreconstruction[J]. Biosystems Engineering, 2015, 139:25-34.

3. ElMasry, G., Cubero, S., Moltó, E., et al. In-line sorting of irregular potatoes by using automated computer-based machine vision system[J]. J Food Eng, 2012, 112(1-2):60-68.

4. Zhang, B. H., Huang, W. Q., Gong, L., et al. Computer vision detection of defective apples using automatic lightness correction and weighted RVM classifier[J]. J Food Eng, 2014a, 146:143-151.

5. Al Ohali, Y. Computer vision based date fruit grading system: Design and implementation[J]. Journal of King Saud University - Computer and Information Sciences, 2011, 23(1):29-36.

6. Blasco, J., Aleixos., N., Moltó., E. Machine vision system for automatic quality grading of fruit[J]. Biosystems Engineering, 2003, 85(4):415-423.

7. Razmjooy, N., Mousavi, B. S., Soleymani, F. A real-time mathematical computer method for potato inspection using machine vision[J]. Comput Math Appl, 2012, 63(1):268-279.

8. Kong, Y. L., Gao, X. Y., Li, H. L., et al. Potato grading method of mass and shapes based on machine vision[J]. Transactions of the Chinese Society of Agricultural Engineering, 2012, 28(17):143-148.

9. Gunasekaran, S. Computer vision technology for food quality assurance[J]. Trends Food Sci Tech, 1996, 7(8):245-256.

10. Zhang, B. H., Huang, W. Q., Li, J. B., et al. Principles, developments and applications of computer vision for external quality inspection of fruits and vegetables: A review[J]. Food Res Int, 2014b, 62:326-343.

11. F., Z., G., Z. Complete calibration of a structured light stripe vision sensor through planar target of unknown orientations[J]. Image and Vision Computing, 2005, 23(1):59-67.

12. Feng, Q. C., Liu, X. N., Jiang, K., et al. Development and experiment on system for tray-seeding on-line measurement based on line structured-light vision[J]. Transactions of the Chinese Society of Agricultural Engineering, 2013, 29(21):143-149.

13. Liu, J. W., Liang, J., Liang, X. H., et al. Industrial vision measuring system for large dimension work-pieces[J]. Optics and Precision Engineering, 2010, 1:126-134.

14. Chen, Y. J., Zuo, W. M., Wang, K. Q., et al. Survey on Structured Light Pattern Codification Methods[J]. Journal of Chinese Computer Systems, 2010, 31(9):1856-1863.

15. Salvi, J., Fernandez, S., Pribanic, T., et al. A state of the art in structured light patterns for surface profilometry[J]. Pattern recognition, 2010, 43(8):2666-2680.

16. Xu, J., Xi, N., Zhang, C., et al. Real-time 3D shape inspection system of automotive parts based on structured light pattern[J]. Optics & Laser Technology, 2011, 43(1):1-8.

17. Chen, S. Y., Li, Y. F., Zhang, J. Vision processing for realtime 3-D data acquisition based on coded structured light[J]. IEEE Transactions, 2008, 17(2):167-176.

18. Rosin, P. L. Measuring shape: ellipticity, rectangularity, and triangularity[J]. Machine Vision and Applications, 2003, 14(3):172-184.