# Design of Agaricus Bisporus Automatic Grading System Based on Machine Vision

Jiye Zheng, Wenjie Feng, Bingfu Liu, Fengyun Wang

► To cite this version:

Jiye Zheng, Wenjie Feng, Bingfu Liu, Fengyun Wang. Design of Agaricus Bisporus Automatic Grading System Based on Machine Vision. 10th International Conference on Computer and Computing Technologies in Agriculture (CCTA), Oct 2016, Dongying, China. pp.388-395, 10.1007/978-3-030-06155-5_39. hal-02179964

HAL Id: hal-02179964

https://inria.hal.science/hal-02179964

Submitted on 11 Jul 2019

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Design of Agaricus Bisporus Automatic Grading System Based on Machine Vision

Jiye Zheng¹, Wenjie Feng¹, Bingfu Liu¹, Fengyun Wang¹(✉)

¹ S&T Information Institute of Shandong Academy of Agricultural Sciences (SAAS), Jinan
250100, China;
jiyezheng@163.com, {34941269, 370623831}@qq.com, wfylily@163.com

**Abstract.** Aim at the agaricus bisporus postharvest automatic classification problem, this paper designed a kind of agaricus bisporus grading system based on machine vision, the system is mainly composed of machine vision system, mechanical system, automatic control system three parts, and analyzed the key technologies involved in every part. Extracted the feature parameters from the mushroom cap color, mushroom cap area and mushroom stem three aspects, combined with the classification standard, the final classification result is given by using edible fungus intelligent recognition platform, and then control the robot grabbing the agaricus bisporus into the corresponding classification box, the rate of accuracy reached over 88%. The results show that using machine vision based automatic grading system for the agaricus bisporus classification is feasible.

**Keywords:** machine vision, agaricus bisporus, grading system, image processing, quality detection

## 1 Introduction

Edible fungi not only has high nutritional value, but also has the very good pharmacological action, is more and more popular with consumers' health ingredients. China is the world's largest producer and exporter of mushroom, edible fungus production ranked sixth in the planting industry, products are exported to 119 countries and regions of the world, earning hundreds of millions of dollars in foreign exchange, therefore edible fungus has become an important industry to create economic benefits for farmers. Because of fresh edible fungus has the very high metabolic activity and high moisture content[1], this makes it vulnerable to microbial corruption and makes the surface shows brown spots and brings important economical losses in China. Currently the agaricus bisporus sorting is done manually, but artificial separation has many shortcomings such as big workload, low efficiency, poor sorting accuracy [2]. With the development of machine vision technology, it has been more and more used in the agricultural product quality automatic detection and classification [3-10], but the use of machine vision technology to construct the agaricus bisporus automatic grading system has not been reported. Therefore, the research tried to use machine vision technology to identify the agaricus bisporus so as to achieve automatic classification purpose.
## 2 Agaricus Bisporus Automatic Grading System

Machine vision based Agaricus bisporus classification system is mainly includes the machine vision system, mechanical system, automatic control system three parts. The overall structure is shown in Fig.1. Machine vision system is responsible for completing the agaricus bisporus image acquisition, image processing and the communication with control system; mechanical system completes the convey of agaricus bisporus and classification actions; automatic control system is used to realize the transmission of orders between the visual system and the mechanical system[11], it includes robot private servers controller, conveyor belt controller and camera controller etc.

<figure><img src="image_2.png" /></figure>

1. Upper computer2. Image acquisition device3. Top light source4. Robot5. Gripper6. Pneumatic control line7. Air compressor8. Agaricus bisporus arrangement system9. Conveyor belt10. Photoelectric sensor11. Agaricus bisporus 12. Bottom light source13. Classification box

<caption>Fig.1. Diagram of general scheme</caption>

The computer vision system is mainly composed of color camera, camera lens, light source system, image acquisition card, image processing software, in order to improve the quality of the acquired image, Microvision[12] produced light source were used in light source system. Using the production of circular low Angle LED AFT-RL12068R as positive light source and LED light AFT-BL100R as a light source on the bottom. Positive light source makes the agaricus bisporus surface texture pattern clear, back light can highlight the agaricus bisporus contour. Mechanical system uses the Japanese Mitsubishi company production of six axis robot RV-4FL-D, the robot can be controlled by software programming using RT ToolBox2 integrated development environment.
RT ToolBox2 is a powerful robot programming control software, apply to all Mitsubishi electronic production of robot products. Combining MELFA BASIC programming language, it enables users to quickly create their own complete program in a short time. This version also provides simulation function, making the program run in the offline world.

The software can send compiled program to the robot controller through the Ethernet interface or a serial port. And also can test the robot state parameters at any time in the process of the program running, such as real-time speed, load, input and output status. In addition, it can detect program running in a real-time, this feature can help users find the mistakes in the program.

The work flow of the agaricus bisporus grading system is as follows: the agaricus bisporus are delivered to the feeding inlet, the purpose is to achieve the agaricus bisporus go through the photoelectric sensor in sequence, when the agaricus bisporus moved to the photoelectric sensor, photoelectric sensor will perceive the objects, conveyor belt controller controls object continues to move forward to the camera vision scope and stops the belt. Then camera acquires agaricus bisporus image and send the image to the upper computer for image recognition, if recognition is successful, the agaricus bisporus image center coordinates and classification number will send to private servers controller via TCP/IP protocol, private servers controller converts the image coordinate into a machine coordinate and control the robot to move to the position above the agaricus bisporus, and then pneumatic gripper will open and grab the agaricus bisporus to the corresponding classification box. The specific process is shown in Fig.2.
<figure><img src="image_3.png" /><figcaption>Fig.2. Agaricus bisporus classification system work flow chart</figcaption></figure>

3 Key Technology of Classification System
## 3.1 Visual Localization Algorithm

The machine vision can locate the position of the agaricus bisporus, and report it to the robot, but it cannot guide the robot moves to the position, because the robot cannot use the image coordinates correctly. We must transform the image coordinates into robot can accept unit. This can be done by calibration, after calibration the machine vision system can use the real world unit (such as millimeter) to report the position, which is the cartesian coordinate system used by the robot.

Converting the image coordinate to robot coordinate consists of two steps, first need to uniform the coordinate units, put metal wafer, 1cm in diameter, or other standard shape object as a standard parts in filming place, use software to calculate pixels the image contained, then the proportion coefficient k can be obtained through actual size in millimeter divided by pixel size, therefore coordinate conversion can be completed in the subsequent detection through pixel coordinates multiplied by coefficient k. Secondly, need to uniform the coordinate system origin, the image coordinate system and the robot coordinate system have the same axis direction, but the X-axis and Y-axis on the contrary, as shown in Fig.3. Take a standard objects (best for circular), placed below the camera, get the coordinates of the object in the image coordinate system (x0, y0), at the same time use the teaching device to move the robot claw to coincide with the object center location to get the coordinates of the object in the robot coordinate system (x1, y1), set the coordinate difference between the two coordinate system is Δx, Δy, then the value of Δx, Δy can be calculated by formula 1.

$$ 
\begin{aligned}
\Delta x &= x1 - k * y0 \\
\Delta y &= y1 - k * x0
\end{aligned}
\quad (1) $$

Therefore when a new object image coordinates (x01, y01) is obtained, and its new robot coordinates (x11, y11) can be calculated by formula 2:

$$ 
\begin{aligned}
x11 &= y01 + \Delta x \\
y11 &= x01 + \Delta y
\end{aligned}
\quad (2) $$

<figure><img src="image_4.png" /></figure>
<caption>Fig. 3. The relation between image coordinate and robot coordinate</caption>

## 3.2 Image Recognition Algorithm

The acquired image will be send to the upper computer, due to the agaricus bisporus arbitrary position in the field of view, pretreatment such as rotating, set the region of interest(ROI) should be done before calculating the length of the mushroom stem, and makes the long axis of the ellipse which has the same second moment with the mushroom region in horizontal direction. Then the original image with the commonly used median filtering, average filtering, wiener filtering method for filtering processing, after the test, the wiener filtering achieved a better filtering effect, fully preserved the image edge and detail information. Next using the formula 3 to convert the RGB color image into a gray image, then the mushroom area was segmented from the background with the automatic threshold selection algorithm of Otsu[13], which picture segmentation method is nonparametric and unsupervised, based on space information and statistical of histograms.

$$ \text{Gray} = 0.299 * r + 0.587 * g + 0.114 * b \quad (3) $$

By comparing gray histogram of the R, G, B component, find that mushroom stem and the cap can be easily distinguished from the background in G component image, so choose G component image for adaptive threshold segmentation to get stipe images, use the original binary image minus the stipe image to get the cap image. Extracting the cap Color, cap area and stipe length as grade criterion parameters, the parameters are calculated based on the pixel points.

## 3.3 Classification Standard

Existing agaricus bisporus classification standards and specifications cannot fully adapt to the machine vision based agaricus bisporus classification criteria, combining with the existing agricultural industry standard agaricus bisporus specifications(NY/T 1790-2009)[14] as well as the common method to describe the characteristics of the image, set the color of the mushroom cap and its area, stipe length as the image feature parameters as shown in table 1.

<table><caption>Table 1. Feature definition of agaricus bisporus</caption><thead><tr><td>Characteristic parameters</td><td>Definition</td></tr></thead><tbody><tr><td>Cap color</td><td>Agaricus bisporus cap area's average color value</td></tr><tr><td>Cap area</td><td>Pixels included in the agaricus bisporus cap image</td></tr><tr><td>Stipe length</td><td>Stipe's minimum enclosing rectangle width</td></tr></tbody></table>

# 4 Results and Discussion

In order to manage the collection of a large number of images, and convenient to the image processing and classification, build edible fungus intelligent recognition
platform, as shown in Fig.4. After the system started, click the select equipment button, if a computer is connected to multiple image acquisition devices, first select a particular device for image acquisition and display. Then set the selected image acquisition device attributes such as the exposure time, the external trigger function, video display format. When click the start monitoring button, the program began monitoring the conveyor belt, when agaricus bisporus go by photoelectric sensor, after a certain delay, the acquisition device will take a picture for the mushroom, according to different applications and different processing requirements, you can manually save the image as BMP or JPEG formats. Using the relevant image information extraction method mentioned above calculate the cap Color, cap area and stipe length, according to the characteristic values of the image and the classification criteria to determine the current level and specifications of the agaricus bisporus.

<figure><img src="image_5.png" /><figcaption>Fig. 4. Edible fungus intelligent recognition platform</figcaption></figure>

In order to verify the reliability of agaricus bisporus automatic grading system, with reference to China's agricultural industry standard the agaricus bisporus grade (NY/T1790-2009), choose 60 white, no mechanical damage agaricus bisporus as premium; 60 white with mild speckled caused by mechanical damage as top grade; 60 ivory with mechanical damage or mushroom cap shape is irregular as secondary grade. During the trial, the image acquired is color image, image size is 1600 x 1200 pixels, the running speed of the conveyor belt is 0.5 m/s, stop waiting for grabbing time sets 3s.The test results are shown in table 2, automatic classification accuracy reached over 88%. The damaged mushrooms are perfectly segregated from the rest of the categories. The premium's classification accuracy bellows 90% due to the fact that all those mushrooms were free of defects and their color characteristics with slightly differences. This is where the work needs to be improved in the future.
<table><caption>Table 2. Agaricus bisporus grading results</caption><thead><tr><th rowspan="2">Agaricus bisporus rank</th><th rowspan="2">Sample number</th><th colspan="3">Classification results</th><th rowspan="2">accuracy</th></tr><tr><th>Premium</th><th>Top grade</th><th>Secondary grade</th></tr></thead><tbody><tr><td>Premium</td><td>60</td><td>53</td><td>6</td><td>1</td><td>88.3%</td></tr><tr><td>Top grade</td><td>60</td><td>1</td><td>56</td><td>3</td><td>93.3%</td></tr><tr><td>Secondary grade</td><td>60</td><td>0</td><td>3</td><td>57</td><td>95%</td></tr></tbody></table>

## 5 Conclusion

Automatic classification of agaricus bisporus based on machine vision technology was studied, and the agaricus bisporus automatic grading system was constructed. Through the image analysis to obtain 3 kinds of agaricus bisporus's differences in color, texture, etc., make the automatic grading system overall recognition correct rate above 88%. The results show that automatic classification of agaricus bisporus based on machine vision technique is feasible, and also provides a new way for agaricus bisporus grading test method research.

## Acknowledgment

Funds for this research was provided by Shandong Academy of Agricultural Sciences(SAAS) Youth Scientific Research Funds Project(2015YQN58), the Key Research and Development Plan of Shandong Province (2016CYJS03A01-1).

## References

1. Aguirre L, Frias JM, Barry-ryan C, et al. Modelling Browning and Brown Spotting of Mushrooms (agaricus Bisporus) Stored in Controlled Environmental Conditions Using Image Analysis[J]. Journal of Food Engineering, 2009, 91(2): 280-286.

2. Huang Xingyi, Jiang Shuang, Chen Quansheng, et al. Identification of defect Pleurotus Geesteranus based on computer vision[J]. Transactions of the CSAE, 2010, 26(10): 350-354. (in Chinese with English abstract)

3. Zhou Zhu, Huang Yi, Li Xiaoyu, et al. Automatic detecting and grading method of potatoes based on machine vision[J]. Transactions of the CSAE, 2012, 28(7): 178-183. (in Chinese with English abstract)

4. Zhan Hui, Li Xiaoyu, Wang Wei, et al. Determination of chestnuts grading based on machine vision[J]. Transactions of the CSAE, 2010, 26(4): 327-331. (in Chinese with English abstract)

5. Zhang Zhiqiang, Niu Zhiyou, Zhao Siming, et al. Weight grading of freshwater fish based on computer vision[J]. Transactions of the CSAE, 2011, 27(2): 350-354. (in Chinese with English abstract)
6. HU Fa-huan, LIU Guo-ping, HU Rong-hua, et al. Quality Grade Detection in Navel Oranges Based on Machine Vision and Support Vector Machine [J]. JOURNAL OF BEIJING UNIVERSITY OF TECHNOLOGY, 2014, 40(11): 1615-1620.

7. Li Ming, Fang Junlong, Qiao Yibo, et al. Automatic Grading Method of Cucumber Fruits Based on Machine Vision[J]. Journal of Agricultural Mechanization Research, 2016, (11): 229-233.

8. Wang Hongjun, Xiong Juntao, Li Zouzou, Deng Jianmeng, Zou Xiangjun. Potato grading method of weight and shape based on imaging characteristics parameters in machine vision system[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2016, 32(8): 272 — 277. (in Chinese with English abstract)

9. Riquelme M, Barreiro P, Ruiz-altisent M, et al. Olive Classification According to External Damage Using Image Analysis[J]. Journal of Food Engineering, 2008, 87(3): 371-379.

10. Gaston E, Frías JM, Cullen PJ, et al. Visible-near Infrared Hyperspectral Imaging for the Identification and Discrimination of Brown Blotch Disease on Mushroom (agaricus Bisporus) Caps. [J]. Journal of Near Infrared Spectroscopy, 2010, 33(8): 327-336.

11. GE Liang, CHEN Hong, REN Jin-hai, et al. The Application of Machine Vision in the Grading of Mushrooms [J]. EDIBLE FUNGI OF CHINA, 2011, 30(1): 8-9, 13.

12. Microvision. http://www.microvision.com.cn/.

13. Otsu N. A Threshold Selection Method From Gray-scale Histogram[C]//Ieee Trans. Syst. Man Cybern, [S.l.]: [s.n.], 1978: 62-66.

14. NY/T 1790—2009. Agaricus bisporus grades and specifications [S].