SUGISAKAMASANORI

Journal of Robotics, Networking and Artificial Life
Vol. 9(3); December (2022), pp. 285–288
ISSN (Online):2352-6386; ISSN (Print):2405-9021
https://alife-robotics.org/jrnal.html

Research Article

A Survey of Machine Vision Applications for Fruit Recognition
Tianyi Zhang1, Fengzhi Dai1,2College of Electronic Information and Automation, Tianjin University of Science and Technology, 300222，China
Tianjin Tianke Intelligent and Manufacture Technology CO., LTD, China

12

ARTICLE INFO
Article History
Received 22 November 2021
Accepted 22 September 2022
Keywords
Machine vision
Image processing
Edge detectionMaturity inspection

ABSTRACT

Machine vision has been widely used in the field of fruit picking in recent years, and its main
application directions include fruit identification, fruit quality detection, fruit maturity detection
and grading, etc. Among them, fruit maturity detection technology is of great significance for
improving the quality and market competitiveness of fresh and stored fruits. This paper focuseson the application of machine vision in fruit identification, fruit ripeness detection and grading in
the past three years, and the application is more mature in many fruits such as citrus, blueberry,cherry, etc. They use some algorithms to accurately identify the fruit, process its image, and feedit back to control the robotic arm for picking and other operations...

© 2022 The Author. Published by Sugisaka Masanori at ALife Robotics Corporation Ltd
This is an open access article distributed under the CC BY-NC 4.0 license(http://creativecommons.org/licenses/by-nc/4.0/)

1. Introduction

identification in recent years [3]. This paper mainly
describes the application of machine vision in the field offruit recognition, and makes a detailed understanding anddescription of the development trend and application
research of its technology. What can be learned is that themain technologies in this area are image recognition,
image signal conversion, image processing, systemcomputing, etc. These operations consist of picking,defect identification and sort identification of citrus,
navel oranges and other fruits.

In recent years, with the steady development of society,agriculture has become more and more rapid and bigger.
Consumers also need more and better fruits to meet their
daily needs. In order to reduce human losses during fruit
picking and to save labor for more efficient picking, more
and more machine vision technologies are being used in
fruit picking and identification detection, etc.
Machine vision is the use of computers to simulate thevisual function of the human eye, extracting informationfrom images or image sequences, and identifying theform and motion of three-dimensional objects in theobjective world [1]. The end result to be achieved bymachine vision is the use of machines to fully interpret,
simulate, replicate and process human vision. In thefuture intelligent development of agriculture, machine
vision technology will play a very important supportingrole. And it will also face many technical challenges [2].
This paper is based on the application of machine visionin agriculture, especially in the field of fruit picking and

2. Fruit (Citrus) Maturity Recognition
Citrus Research Institute of CAAS (Chinese Academyof Agricultural Sciences) has proved that citrus peelcolor can be used as an indicator of citrus fruit maturity
after experimental research. Based on this conclusion,
one can complete the key identification part of theautomatic detection of citrus maturity.
Therefore, Yibin Ying of Zhejiang University in
China designed an image recognition system that canCorresponding author’s E-mail: daifz@tust.edu.cn URL: www.tust.edu.cn285

extract the edges of the fruit, and uses a straight-line
projection method to eliminate corner points. This step isdone to improve the accuracy. Finally, the circles aredetected by the least squares method to identify the fruit.
The algorithm can be divided into the following foursteps, which are:
1. Image denoising process
2. Color segmentation
3. Occlusion problem processing
4. Fruit recognition

non-destructively detect the maturity of citrus by color.
The machine vision system consists of CCD camera,
image acquisition card, monitor, light box and computer[2].
According to the contrast of light source effects, thechoice is to capture images of citrus against a whitebackground. Then, a variety of color models wereestablished, and the image acquisition method wasdetermined.
This image acquisition method obtains a color
picture in RGB format through a camera, and this color
picture is stored and displayed in a computer terminal.
Then the RGB format pictures are converted to HIS
format pictures for the computer to perform featureprocessing. The relationship between the RGB and HIS
can be expressed as
𝐼𝐼 = (𝑅𝑅 + 𝐺𝐺 + 𝐵𝐵)⁄3[𝑀𝑀𝑀𝑀𝑀𝑀(𝑅𝑅, 𝐺𝐺, 𝐵𝐵)]𝑆𝑆 = 𝐼𝐼 −3⎨
2𝑅𝑅 − 𝐺𝐺 − 𝐵𝐵
⎪𝑊𝑊 = 𝑐𝑐𝑐𝑐𝑐𝑐 −1 ��⎩
2√𝑅𝑅 2 + 𝐺𝐺 2 +𝐵𝐵2 − 𝑅𝑅𝑅𝑅 − 𝐺𝐺𝐺𝐺 − 𝑅𝑅𝑅𝑅

3.1. Image denoising processing
For easier post-processing, the image is first processedfor noise reduction. The algorithm uses a Gaussianfiltering algorithm to reduce the impact of image noise.
Gaussian filtering is shown in Fig. 1.

⎧⎪

𝐻𝐻 = 𝑊𝑊, if B≤G
𝐻𝐻 = 2𝐶𝐶 − 𝑊𝑊, if B＞G(1)This experiment showed that the color of citrus peelchanged from green to orange during the maturity
process, and the color change was used to determine
whether the citrus was mature or not.

Fig. 1. Gaussian filtering

3.2. Color segmentation

3. Fruit (navel orange) Identification Processing

Ripe navel oranges are warm colors such as orange or red,and the fruit background is mostly cool colors such as
green and cyan.
The fruit and background colors are clearly
distinguished, so the color segmentation can achieve
good results. There uses normalized chromatic aberration
for image segmentation. The relationship between the Sand R, G and B can be expressed as

In the process of navel orange picking, therobot captures images in real time through the camera,
and the field of view includes the sky, branches, leaves,fruits, and the earth, etc.
Moreover, the captured real-time images containoverlapping, blocking, and uneven lighting, and thebackground is very complex, which puts the robustnessof the recognition algorithm to a severe challenge.
To solve such problems, Guangli Chu designed amachine vision-based method for sphere-like fruit
recognition. The method first uses an image
segmentation algorithm based on color normalizationdifference. It separates the fruit from the background [3].
Then uses a single-edge pixel tracking algorithm to𝑠𝑠 =255(2𝑅𝑅+𝐵𝐵)
2(𝑅𝑅+𝐺𝐺+𝐵𝐵)(2)

When S≥T ( the fixed threshold ), the pixel valueof the point remains unchanged, and when S＜T, thepoint becomes black. Next, compare the images before
and after processing. The unprocessed image is shown inFig. 2, and the processed image is shown in Fig. 3.
286

Fig. 2. Unprocessed image

Fig. 4. Circle detection result
Finally, the data is transmitted to the actuator forpicking. The study used a machine vision algorithm torecognize 500 similar images and it achieved aprocessing time of 83.548ms and an accuracy of 95%.
4. Fruit (cherry) Defect Detection
Wang Zhao from Beijing Forestry University usedmachine vision technology to extract image informationfrom the cherry surface. He proposed the corresponding
image processing algorithm to achieve defect detection
and recognition on the cherry surface [4].
To complete the defect identification task on thecherry surface, two steps are required: cherry image
background removal processing and cherry defect feature
extraction.

Fig. 3. Processed image
3.3. Occlusion problem processing
Considering the position of navel orange fruits on the tree,
they will overlap with other fruits or leaves, thus blocking
each other.
The mutually blocked fruits or leaves must havecross corner points, here use the straight-line projection
method to eliminate the corner points, so that the edge ofthe two fruits at the cross point break. This allows forbetter identification of the navel orange.

4.1. Cherry image background removal processing
The first choice is to extract its R component to enhance
the contrast between the cherry image and thebackground image. To make the image features more
visible, it is converted into a binary image, andmorphological processing is used to augment the imagefeatures. Afterwards, an edge extraction algorithm wasapplied to extract the edges to obtain the cherry anddefective contour edges.
Subsequently, the cherry contour is filled to get thecomplete cherry contour, the contour is filtered to remove
the non-conforming contour. Finally, the original color
image is filled.

3.4. Fruit recognition
Based on the fact that navel oranges are mostly round in
shape, this study uses least squares to achieve circledetection. After the circular detection process, the noncircular edge graphics are filtered out, and then thecircular fruits are identified. Circle detection result isshown in Fig. 4.

287

4.2. Cherry defect feature extraction

References

There are four main types of common cherry defects,
which are rotten, bird peck, scratches, deformed fruit.
Among them, rotten in the image acquired by machine
vision is manifested as a large number of spots in thebinary map; bird peck and scratches are manifested as
black irregular lines inside the edge of the image andother defect characteristics. And deformed fruits havevarious forms, such as twin fruits and stimulated growth
fruits, which are characterized by irregularly shaped
images instead of a circle or oval.
In order to achieve the cherry defect feature
extraction, the image must be processed.
Firstly, the obtained de-background map isweakened by the brightness of the reflected area, andtransformed into a binarized image. The cherry can bejudged by the large number of spots in the map whether
the cherry is rotten or not. Secondly, the image grayscale
map is processed by custom multi-thresholding, andwhite defects and edges can be obtained. After color
inversion, the defects and flips become black, and it canbe observed whether there are black lines inside thecherry in the map and whether the cherry edge contour isregular round. Based on these, bird pecks, scratches anddeformed fruits can be detected.

1. Hongkun Tian, Tianhai Wang,Yadong Liu, et al., Computer
vision technology in agricultural automation — A review.
Information Processing in Agriculture, 2020, 7(1), pp. 1-19.
2. Ying Yibin, Rao Xiuqin, Ma Junfu. Methodology for
nondestructive inspection of citrus maturity with machine
vision (in Chinese). Transactions of the Chinese Society ofAgricultural Engineering (Transactions of the CSAE), 2004,
20(2), pp. 144-147.
3. Chu Guangli, Zhang Wei, Wang Yanjie ,et al., A method offruit picking robot target identification based on machinevision. Journal of Chinese Agricultural Mechanization, 2018,39(2), pp. 83- 88.
4. Wang Zhao, Cheng Pengle, Huangpu Yilong, et al.,Detection of Cherry Defects Based on NI Vision Assistant.
Computer Science and Application, 2017, 07(12):1163-1173.

Authors Introduction
Ms. Tianyi Zhang

5. Conclusion

Dr. Fengzhi Dai

She is a second-year mastercandidate in Tianjin University ofScienceandTechnology,majoringin
brain-computerinterface, machine learning.

He received an M.E. and Doctor ofEngineering (PhD) from theBeijing Institute of Technology,
China in 1998 and Oita University,
Japan in 2004 respectively. His
main research interests are artificial
intelligence, pattern recognition
and robotics. He worked inNational Institute of Technology,
Matsue College, Japan from 2003 to 2009. Since October
2009, he has been the staff in College of Electronic
Information and Automation, Tianjin University of Scienceand Technology, China.

This paper describes the application of machine vision
for recognition and detection on fruits such as citrus,
navel orange and cherry, including some methods forimage processing in machine vision applications, such as:image enhancement, image edge detection, etc. At thesame time, it also further understands the specificmethods of machine vision in fruit detection andrecognition.
With the refinement of neural networks, fuzzy control
and other intelligent algorithms and the development ofartificial intelligence, the richness and speed of image
processing technology is greatly enhanced. The
development of 5G networks has also accelerated thedevelopment of machine vision technology, especially inagricultural production and fruit identification andpicking.

288