SUGISAKAMASANORI

Journal of Robotics, Networking and Artificial Life
Vol. 9(1); June (2022), pp. 13–19
ISSN (Online): 2352-6386; ISSN (Print): 2405-9021
https://alife-robotics.org/jrnal.html

Research Article

Toward Smart Tomato Greenhouse:
The 6th Tomato-Harvesting-Robot Competition and Regulation
Changes Aiming at Practical Application
Takayuki Matsuo1, Yasunori Takemura2, Takashi Sonoda2, Yuya Nishida3, Shinsuke Yasukawa3, Kazuo Ishii3

National Institute Technology, Kitakyushu College, 2-20-1, Shii, Kokuraminami-ku, kitakyushu-shi, Fukuoka, Japan
Department of Engineering, Nishinippon Institute of Technology, 2-11, Aratsu, Kanda-town,Miyako-gun, Fukuoka 800-0396, Japan
Department of Life Science and System Engineering, Graduate School of Kyushu Institute of Technology, 2-4, Hibikino, Wakamatsu, Kitakyushu-city, Fukuoka
808-0196, Japan

1
2
3

ARTIC LE INFO
Article History
Received 16 November 2020
Accepted 17 June 2022

Keywords
Smart agriculture
Agricultural robot
Tomato-Harvesting-Robot competition

AB S T RA C T
Agriculture is one of the most important primary industries, however the agricultural workforce
in Japan has been decreasing due to the aging, shortage of successors and their heavy work.
Smart agriculture and farms, which aim to automate farm work, have been attracting attention
for the future sustainable society. We have been organizing the Tomato-Harvesting-Robot
Competitions since 2014 to introduce robotic technology and AI into agriculture and promoteagricultural technology. The competition consists of a Junior League for young students and a
Senior League for general competitors. In this paper, we report the results of the 6th TomatoHarvesting-Robot Competition and the regulation changes toward practical applications..
© 2022 The Author. Published by Sugisaka Masanori at ALife Robotics Corporation Ltd
This is an open access article distributed under the CC BY-NC 4.0 license(http://creativecommons.org/licenses/by-nc/4.0/).

work, knowledge transfer from experts to younger
workers, the large initial investment when starting
farming, and a small income compared to the investment,
etc. Recently smart agriculture using AI, IoT, big data
and robot technologies has been attracting attention, and
it is expected to improve the agricultural works inefficiency and contribute to labor-saving by automatedoperations, parameterization and visualization of expert
farmers and sharing. In the automation of agriculturalwork, research on agricultural harvesting robots has been
actively carried out, e.g., Hayashi et al. developed a

1. Introduction
According to statistical information by the Ministry ofAgriculture, Forestry and Fisheries of Japan (MAFF)[1], the agricultural workforce in 2020 is 1.4 million
workers, which is 22.5%, 400 thousand less compared tothose of 2015 in spite of world population increase. In
addition, the rate of older workers (over 65 years old)increased 4.9% to 69.8%. The decrease of the number ofagricultural workers and their aging have become one of
the social problems in Japan, whose reasons are heavy

Corresponding author’s E-mail: matsuo@kct.ac.jp, takemura@nishitech.ac.jp, sonoda@nishitech.ac.jp, s-yasukawa@brain.kyutech.ac.jp,
y-nishida@brain.kyutech.ac.jp, ishii@brain.kyutech.ac.jp http://www.lsse.kyutech.ac.jp/~sociorobo/ja/tomato-robot201913

strawberry harvesting robot with obstacle avoidancecontrol algorithm to approach the target fruit without
damaging the neighboring fruits [2], Henten et al.
developed an autonomous robot for removing unwanted
cucumber leaves [3] and Lehnert et al. realized
autonomous harvesting of sweet pepper using mobile
robot with 6-DOF robot arm, end-effector and RGB-D
camera[4].
We have been organizing the Tomato Harvesting Robot
Competition since 2014 with the aim of developing Agrirobots, arousing the interests of young researchers ofrobotics in agriculture, and giving back the developed
technology to the agricultural field [5]. The competition
is divided into two leagues, Junior league for outreachactivities to junior and senior high school students andSenior league for actual harvesting demonstrations. In the
junior league, each team develops a robot that harvested
small tomatoes using LEGO Mindstorm. In the seniorleague, competitors demonstrate the harvesting
performances of their developed robot(s) using tomato
plants for commercial production. Yaguchi et al.
developed an end-effector that grasped a fruit using
grippers and plucked it from the separation layer in thepeduncle using the results of the competitions. They
described the results of harvesting experiments
conducted on an actual farm, demonstrating that theharvesting time was shortened by improving theharvesting motion [6]. In addition, they developed
tomato harvesting robots and carried out evaluation
experiments on three robot systems with different endeffectors for cutting and an end-effector or birdview
stereo cameras [7]. Fujinaga et al. also report the mosaic
image of tomato plants using IR and RGB-D images for
estimation of maturity of tomato fruits, which are also
developed through the competitions [8][9].
In this paper, we report the results of the 6th TomatoHarvesting-Robot Competition and discuss theregulation changes toward actual applications. Chapter 2
provides an overview of the junior and senior leagues of
the Tomato Harvesting-Robot Competition. Chapter 3
describes the history of the rule revision in the seniorleague. Chapter 4 presents the results of the 6th Tomato
Robot Competition. Section 5 concludes.

League are supposed to have and automated and selfcontained robots, and the Junior League are for young
students, where they make teams with 2 to 4 students andmake a robot with LEGO Mindstorms.
2.1 Senior League
In the Senior League, each team developed one or two
robots that harvest medium tomatoes (approximately 60
to 120 grams) provided by Hibikinada green farm Ltd..
In the first stage, there are tomatoes hanging one by one
in line, and the basic robot performances, such as
mobility, tomato handling and recognition are evaluated.
In the second stage, bunches of tomatoes are hung in a
cage, and each team competes the number of harvested
tomatoes and total performance as a system is evaluated.
In the final round, each team harvests tomato fruits fromplants which are the same as commercial ones.
Each team chooses one of two areas depending on
moving the method, the rail-style area (Fig. 1(a))
assuming the operation in the greenhouse with rails or thefree-style area (Fig. 1(b)) assuming the outdoor
cultivation. In particular, the rail style area reproduces therails installed in the Dutch-light greenhouse as shown in
Fig.2. The rails of the Dutch-light greenhouse are about
𝜙𝜙50 mm, and the distance between the centers of the rails
is 600 mm. All greenhouses use rails with almost thesame specifications. From the second round, theharvesting point is calculated based on the number oftomatoes harvested multiplied by robot class coefficient
C by degree of automation and working area shown in
Tables 1 and 2. In Table 1, "Directly" means that theoperator controls the tomato by directly looking at thespot, and "Indirectly" means that the operator controls therobot remotely and manually using visual informationand others.

(a)(b)
Fig.1 Working fields of senior league(a)Free-style area (b)Rail-style area

2. Tomato-Harvesting-Robot Competition
The Tomato-Harvesting-Robot competition started in2014 and the 6th competition was held in the winter of
2019, which consists of two leagues, the Senior League
and the Junior League. The target competitors in Senior
14

box, the robot is to return to the course line. Then, therobot carries the box to the specific storage location.
(4) Object detection
The robot should return to the course line after putting
the tomato-boxes. Then, the robot is to go to the charging
station which is put at a certain distance away from thecourse line. The robot has to stop by the distancemeasuring sensor.
Each team consists of 3 or 4 students and makes arobot using LEGO Mindstorms. The sizes of the robot arewithin 300 x 300 [mm] as the basic specification for the
robot. The robot can have structural change after starting,so that the initial robot structure is judged as the robot
size.

Fig. 2 Dutch-light greenhouse
Table 1 The class number and coefficient C at choose
remote control
Method of View
AreaClass number
Coefficient C

Directly
Rail
Free
T1
T2
1
2

Indirectly
Rail
Free
T3
T4
2
4

Until the 4th Tomato-Harvesting-Robot Competition, thetomatoes were stored in a transparent box "Tomato-Box",
from the 5th competition, the tomatoes themselves wereplaced without boxes. Therefore, more careful handlingof tomatoes is needed when the robot manipulates andtransports them. For participants not to forget tomatoes
food products, and if they are handled improperly in thiscompetition, they will not be scored for storage points.

Table 2 The class number and coefficient C at choose
autonomous control
Area
Class number
Coefficient C

Rail
T5
8

Free
T6
16

Criteria for improper handling of tomatoes are:

2.2 Junior League

1. The liquid inside the tomato is released to the outside
or not.

The competition field of Junior League is illustrated inFig. 3. We set the subjects to make a tomato harvesting
system of an autonomous transport such as thetransportation and sorting of harvested tomatoes.
Competition subjects are shown as following [10].

2. The tomato inside is visible or not
3. The tomato is clearly dented from their pre-race
condition or not.

(1) Line Trace
The robot moves along the black course line using thecolor sensor, which is designed to guide to tomatoes andother landmarks.

4. The tomato is transported in contact with the floor ornot.

(2) Color Identification
The robot explores the working field and recognizes
color signs along the course line, and finds three sets oftomatoes in the boxes (tomato-box). In the harvest field,
there are red, green and yellow Tomato-squares. The
tomato-boxes are arranged in the corresponding squares
so as to find them if a robot moves along the line.
(3) Mechanism Design and Control
Participants are expected to design and make a robot
arm to get a tomato-box for picking up the tomato-boxes.
The robot is required to store, transport and relocate theboxes to the assigned storage area. After picking up theFig.3 Robot Jr. League competition Field

15

3. Regulations Changes in the Senior League

where C is the coefficient of robot class, 𝛼𝛼 is the numberof successful tomatoes, 𝛽𝛽 is that of damaged ones, and 𝜀𝜀
is the plant damage. When harvesting the target tomatoes,
some teams dropped neighboring tomatoes as the tomato
grows up as a cluster and turns red from upper fruits,
moreover, some teams intentionally dropped immature
tomatoes to harvest the target tomatoes. Therefore, Eq. 2
was employed from the fourth competition, and two
points were deducted for each dropped tomato. Tomatoes
remained in the plant and damaged without being
harvested were also counted.

The regulation of the competition is changed byevaluating the achievement of teams toward actual
greenhouse toward smart agriculture. Table 3 shows thehistory of regulation changes. The first competition washeld in 2014 with the Senior League only. From thesecond competition, Junior League started as theoutreach activity of agricultural robotics. From the thirdcompetition, a bump was put on the free-style field tomotivate the improvement of mobility for rough terrain.
From the fifth competition, the black board shown in
Fig.4, which was placed in the image processing easy,
was removed and the robots were asked to find tomatoesinvariouscoloredenvironments.

(2)

𝑃𝑃 = 𝜂𝜂𝜂𝜂(2𝛼𝛼 + 𝛽𝛽) − 2(𝛾𝛾 + 𝛿𝛿) − 𝜀𝜀(3)𝛾𝛾 is the number of dropped tomatoes and 𝛿𝛿 is thenumber of damaged tomatoes without being harvested.
The current evaluation function is modified to Eq. 3 forconsideration of the inequality point per successful
harvesting between manual operation and autonomous
operation. The autonomous harvesting team got higher
points and less effect of point reduction due to dropping
tomatoes. Therefore, the harvest rate 𝜂𝜂 as shown in Eq. 4
was adopted from the fifth competition. As a result, themore tomatoes the robot dropped, regardless of its class,the less points it received for each tomato it harvested.
The robots should deal with tomatoes gently as
commercial products.

(1) Mobility: Working field
The Senior league of Tomato-Harvesting-Robotcompetition consists of mainly two classes depending on
working fields, the rail-style and the free-style fields. The
rail style field is designed assuming the Dutch-lightgreenhouse where a pair of rails is laid for temperature
control and carts transferring tomato maintenance, andthe free-style field is assuming an open-field culture fieldand currently the flat ground covered with artificial grass
instead of soil. In order to give the robots incline to theground, 10mm or 16mm height bumps are put on the fieldand currently slope is arranged near tomato plants toimitate ridges of open-field culture (see Fig.5).

𝜂𝜂 =(2) Automation and Image Processing
The black board was placed behind the tomato plants inthe first competition to reduce the computational cost ofimage processing for harvesting tomatoes autonomously.
To make the competition close to the actual harvesting
gradually, teams select tomato printed background (x2point/tomato) or black boards in the third competition.
Most teams for automated harvesting classes utilized
RGB-D cameras like Intel Realsense device to obtain
RGB image and depth information, and no backgroundboard from the 5th competition.
(3) Scoring for gentle manipulation as products
The damages to tomato fruits and the planets are crucial
evaluation as commercial products, and the degree ofmaturity is directly connected to the shipping time. The
harvesting robots should take the proper color tomatoes
without any damage to fruits and plants. Until the thirdcompetition, the score P was calculated mainly
depending on the color and damage conditions of theharvested tomatoes as shown in Eq. 1.
(1)
𝑃𝑃 = 𝐶𝐶(2𝛼𝛼 + 𝛽𝛽) − 𝜀𝜀

𝑃𝑃 = 𝐶𝐶(2𝛼𝛼 + 𝛽𝛽) − 2(𝛾𝛾 + 𝛿𝛿) − 𝜀𝜀𝛼𝛼𝛼𝛼 + 𝛽𝛽 + 𝛾𝛾 + 𝛿𝛿(4)

At the beginning of the competition, only the number oftomatoes harvested was reflected in the score (Eq. 1).
However, in order to improve the robot's harvesting
technique, we added a section that deducted points basedon the number of dropped and damaged tomatoes (Eq. 2).
This was done to reduce the number of unsalable
tomatoes. Finally, to improve the harvesting accuracy,
we introduced the concept of harvest rate (Eq. 3 and 4).
Table 3 History of regulation changes

16

inclined and difficult to control the robot by sliding down
the slope or collisions as intended. The results of the
Senior League are shown in Table 4 and the winner the
robot of Hibikino-Tom's is shown in Fig.6, second placerobot of Sugar-lab is in Fig.7.
The winning robot had succeeded in harvesting five
tomatoes with no damage and two damaged tomatoes
remaining in the plant in the first trial, and then five
tomatoes succeeded and four dropped, two ones
remained in the plant in the 2nd trial. It takes 2 min per
one harvesting, which is almost 10 times slower thanhuman harvesting.
For Junior League, this competition was the second
one after the regulation changes to require the teams tohandle fresh tomatoes. Although some teams damaged
tomatoes in the previous competition, in this competition,
most teams were able to instill an awareness of theimportance of handling raw plants through their crafts
with ingenuity in handling the tomatoes with care andfew damages to tomatoes. Overall, the competition
scores improved from the previous year, and one team
had succeeded in making a system to harvest andtransport all the tomatoes in one week. Basically, we will
not change the basic rules, however, some additional
subjects for the next competition should be considered.
Some comments from students are related to thedifficulties from tomato size variation, which often
happens in natural environments. That encourages
ingenuity in the devices and algorithms due to theinconsistent environment of handling natural objects.

Fig.4 A black board set up behind the tomatoes.

(a)

(b)
Fig.5 (a) A bumps set up in the freestyle area. (b) A
slope set up in the freestyle area

4. Result of 6thCompetition

Tomato-Harvesting-Robot

The Senior League regulation was revised in the 5thcompetition, including changes to the scoring equationand the slopes of the free-style area. Equation 4
introduces the concept of harvest rate, where dropping
tomatoes and scratching tomatoes have a significant
effect on the score. In actual use, the robot needs toharvest more tomatoes within a certain time, therefore, it
is desirable for the robot to work for the entire
competition (10 min). However, many teams quit thecompetition before 10 min not to make the harvest rate
worse. As for the free-style slope, some robots become

Fig.6 The winner robot Kyushu Institute ofTechnology team.

17

Science and Technology (FAIS), IT and Mechatoronics
Chapter and Kyushu Chapter of Japanese Society ofAgricultural Machinery and Food Engineers, The
MathWorks Co. Ltd., and Ministry of Agriculture,
Forestry and Fisheries(MAFF).
Appendix
Results of the 6th Tomato Harvesting Robot
Competition are as follows. Both teams competed with a
T5 class robot that operates in a rail-style area andharvests tomatoes autonomously. Table 5 shows theresult of the senior league. Table 6 shows the result of thejunior league.

Fig.7 Second place robot of Nagasaki Institute ofApplied Science team
Table 4 Result of Senior League

Table 5 Result of Senior League
Team

Ranking

Winner
2nd Place
5. Conclusions
In this paper, we report on the 6th Tomato Robot
Competition, which aimed at the social synthesis ofrobots. In the Junior League, 24 teams participated, andin the Senior League, 9 teams. We also summarized thechanges in the rules of the Senior League since the firstcompetition. The operating environment was modified tobe similar to that of a real farm, and the calculation
method of the score was modified to aim at harvesting"sellable tomatoes".

Hibikino-Toms(Kyushu Inst. Of Tech.)Syuga-Lab(Nagasaki Inst. Of
Applied Science

Rankin
1st Place

Table 6 Result of Junior League
Team
Tangokinasu

2nd Place

Double Lycopin

3rd Place(Fukuoka Joto High School)(Fukuoka Joto High School)

NiASience(Nagasaki Sogo
High School)

Ver.2

FuzokuSpecial Award
Best Presentation Award G-AdvanceAcknowledgements
This works was supported by Kyushu Institute ofTechnology, Center for Socio-Robotics Synthesis, The
University of Kitakyushu, Nippon Bunri University,
Nagasaki Prefecture University, Kyushu Polytechnic
Collage , Nishinippon Institute of Technology(Thissupport was including The Ministry of Education,
Culture, Sports, Science and Technology(MEXT),
Center of Community), National College of Technology,
Kitakyushu College, Hibikinada-Saien, Afrel Co., Ltd.,City of Kitakyushu, The Kitakyushu Chamber ofCommerce and Industry, Institute of Systems,
Information Technologies and Nanotechnologies (ISIT),
Japan Chapter of IEEE Society on Systems, Man, andCybernetics, The Japan Society of MechanicalEngineers(JSME), The Robotics Society of Japan(RSJ),
Kitakyushu Foundation for the Advancement of Industry

Perfect Award
Special Judges Award
Idea Award
Challenge Award

18(Fukuoka Joto Highschool)

Tangokinasu(Fukuoka Joto High School)

Kako Tomato Curry(Kashii
school)Technical

High

Revolutionary(Fukuoka Joto High School)

SyokugyoMochi(National institute ofTechnology, Kitakyushu
college)References
1.
2.

3.

4.
5.

6.

7.

8.

9.
10.

Dr. Takashi Sonoda

2020 CENSUS of Agriculture and Forestry in Japan,
2020
Hayashi et. al, Collision-free control of a strawberryharvesting robot by recognition of immature fruits,
Journal of Science and High Technology in Agriculture,
Vol.25, No.2, pp. 29-37, 2013
Henten et. al, An Autonomous Robot for De-leafing
Cucumber Plants grown in aHigh-wire Cultivation
System, Biosystems Engineering, Vol.94, No.3, pp. 317–
323, 2006
Lehnert et. al, Autonomous Sweet Pepper Harvesting forProtected Cropping Systems, IEEE Robotics and Automation

Dr. Yuya Nishida

Letters, Vol.2, No.2, pp. 872–879, 2017

Matsuo et. al, Toward Smart Tomato Greenhouse: The
Fourth Tomato Harvesting Robot Competition, Journal of
Robotics, Networking and Artificial Life, Vol. 6, No.2,
pp. 138-142, 2019
Yaguchi et. al, Development of An Autonomous Tomato
Harvesting Robot with Rotational Plucking Gripper, 2016
IEEE/RSJ Int. Conf. on Intelligent Robots and Systems,
pp. 652-657, 2016.
Yaguchi et. al, A Research of Construction Method forAutonomous Tomato Harvesting Robot focusing on
Harvesting Device and Visual Recognition, J. Robot. Soc.
Jpn., Vol. 36, No.10, pp. 693–702, (in Japanese withEnglish summary), 2018.
Takuya Fujinaga et al., Image Mosaicing Using MultiModal Images for Generation of Tomato Growth State
Map, J. Robot. Mechatron., Vol.30, No.2, pp. 187-197,
2018.
Takuya Fujinaga, et al., Tomato Growth State Map for theAutomation of Monitoring and Harvesting, J. Robot.
Mechatron., Vol.32, No.6, pp. 1279-1291, 2020.
The 6th Tomato robot competition junior rule, 2019, 7

Dr. Shinsuke Yasukawa

Prof. Kazuo Ishii

Authors Introduction
Dr. Takayuki Matsuo

Dr. Yasunori Takemura

He is an Associate Professor at
Department Creative Engineering,
National College of Technology(KOSEN), Kitakyushu College,
Japan. His research area isunderwater robots and biomimetic
robots.

He is an Associate Professor at
Department Integrated System
Engineering,Nishi-Nippon
Institute of Technology, Fukuoka,
Japan. His research area is aboutmachine learning, data mining andRobotics

19

He is an Associate Professor at
Department Integrated System
Engineering,Nishi-Nippon
InstituteofTechnology,
Fukuoka, Japan. His research
are underwater robotics androbot manipulator systems

He is an Associate Professor at
Graduate School of Life
ScienceandSystemEngineering, Kyushu Instituteof Technology, Japan. His
research area is about filed
robotics, its application, anddata processing.
He is an Associate Professor at
Graduate School of Life
ScienceandSystems
Engineering, Kyushu Instituteof Technology. His researchinterests include information
processinginbiologicalsystems and their applications
in robotics.
He is a Professor at Graduate
School of Life Science andSystem Engineering, Kyushu
Institute of Technology, Japan.
His research area is about field
robots and intelligent robot
systems