KIKS Extended Team Description for
RoboCup 2020
Yusei Naito, Shin Ohno, Yuta Imaeda, Akihito Odanaka, Yasutaka Tsuruta,
Ryoma Mitsuoka, Taisuke Tane, Masato Watanabe, and Toko Sugiura1
National Institute of Technology, Toyota College, 2-1 Eisei-cho, Toyota, Aichi
471-8525, Japansugi@toyota-ct.ac.jp,
URL: www.ee.toyota-ct.ac.jp/~sugi/RoboCup.html

Abstract. This paper is used to qualify as participation to the RoboCup
2020 soccer small size league. Our team’s robots and systems are designed under the RoboCup 2019 rules. In this paper for hardware, it isdescribed about the various idea of dribble mechanism, whose importance was again emphasized in the ﬁnal of RoboCup 2019 Sydney. In
software, the implementation of UCT (Upper Conﬁdence bound applied
to Trees) in MCTS (Monte Carlo Tree Search) method was performed
to achieve the action-decision of the motion for the robots. Moreover, we
tried to use Deep Learning Library (Sony products) and Neural Network
using GUI (Graphical User Interface) for layered design, learning andevaluation in AI system.
Keywords: RoboCup, small size league, autonomous robot, global vision, engineering education

1

Introduction

KIKS has been continuously developing the higher-performance hardware andsmart AI system. In hardware, we worked on improving ball control performance,
and implemented and examined them . In software, we examined and improved
path generation for the opponent robots. In addition, Neural Network Libraries
and Neural Network Console were introduced, and action-decision algorithm
using neural networks and Monte Carlo tree search for robots were inplemented.

2

Mechanical design

In RoboCup SSL, to realize a strategy based on AI, it is necessary to havea mechanism to accurately kick a ball and a mechanism to accurately receive
a ball. Our present robots, however, did not have suﬃcient performance of amechanism for holding a ball, and often bounce oﬀ the ball during the game.
Also, when a ball was kicked, the ball sometimes rolled in a diﬀerent direction
from the AI command because the ball was not held at the center of dribbling

2

Yusei Naito et al.

bar. If we can improve the ball holding ability, the dribbler will be an eﬀective
tactics like ZJUNlict in ﬁnal game of RoboCup 2019 Sydney. In addition, when
the dribbler’s rotation can be suﬃciently transmitted to the ball, a curve shot
may be realized, which will be eﬀective in expansion of AI strategies. Therefore,
in this section, it focuses on dribbling performance which improves speed andstability for a ball, and describes an idea of new dribblers to improve the ballholding ability.
2.1

Idea of ball stabilization in present dribbler

In the RoboCup 2019 Sydney ﬁnal, the importance for dribbling performance
was once again demonstrated. In order to keep possession of a ball from opponent team and also to make a success for Auto Ball Placement, the dribblingmechanism that can continue to give a stable rotation of a ball is required. In
our present dribble device, however, when a backspin is applied to the ball, even
if the ball is stable at ﬁrst, it gradually begins to vibrate and moves away fromthe roller. It is causally related to the fact that the ball bites too much into thedribbler and generates a large pressing force on the dribbler. As the result, thedribbler cannot withstand the mechanical load and it pushes the ball back. To
solve this problem, for example, ZJUNlict team applied 3-point-touch system
as shown in Fig.1(a)[1], and OP-AmP team removed the the chip-kick bar andattached a freely rotating roller as shown in Fig.1(b)[2].

(a)

(b)

Fig. 1. Image of typical dribble structure for (a)ZJUNlict and (b)OP-AmP [2].

Our idea of dribbling device is shown in Fig.2. It has two dribbling rollers.
By mounting the roller on a higher position, the ball is more strongly pressed

KIKS Extended Team Description for RoboCup 20203

Fig. 2. Schematic drawing of idea for present dribbling device

against the carpet, and as the result, makes it diﬃcult for the ball to leave theroller. Therefore, the roller can eﬃciently transmit rotation to the ball. But,
as the contact point between the roller and the ball raises up, the forth withwhich the ball enters into the dribbling device increases. Thus, another roller isplaced at the lower position that does not touch the straight-kicking bar. Thisroller stops the ball entering into the dribble device. The idea of this dribbling
device will be able to stabilize a high-speed spin ball while keeping the chipkick function. In addition, a gap between the ball and the chip-kicking bar play
as a role for no bouncing oﬀ the ball when catching it. The prototype dribbler
used for the test is shown in Fig.3. The Material and diameter of roller used inprototype is tabulated in Table. 1.

Fig. 3. Prototype dribbler used for the test.

4

Yusei Naito et al.
Table 1. Material and diameter of roller using for the prototype dribbler

Two small rollers
Power wheel

Material
Urethane
Rubber

Diameter
8mm
16mm

Two small rollers and a power wheel shown in Fig.2 transmit the force by friction contact with rubber. In fact, we tried to transmit rotation to the ball using
a prototype. As the result, when the rotation speed exceeded around 2000[rpm],
the ball and the roller slipped, and the rotation of power roller could not betransmitted suﬃciently to a ball. This is probably because the contact surface
between the ball and the rollers was small due to the small roller diameter, andthe roller material was too hard. At present, there is no mechanism to adjust
the angle of the dribble bar according to the position of the ball. Hereafter, we
would like to make a conﬁrmation of eﬀectiveness of this idea by changing theroller diameter and material.
Furthermore, in order to realize the precise pass-play, we improved the accuracy for kicking by adding the spiral structure on dribbling roller as shown in
Fig.4. This is similar to the idea of the ZJUNlict [1] and TIGREs Mannheim[3].
It allows to center the ball automatically and instantly. Because of this change,
the accuracy for kicking as well as the probability of the Auto Ball Placement
were improved.

Fig. 4. Dribbler’s surface with a spiral structure.

2.2

Idea of new dribbling device

Here is another approach to receiving a ball at the center of the dribble bar. The
speed of the dribble roller is proportional to the radius. Therefore, by making
the center thicker and the ends narrower, the velocity of the ball will add a force
towards the center. In this subsection, we propose a mechanism that can change
the dribbling-bar angle by using a parallel link mechanism and a servomotor.
The 3D CAD image is shown in Fig.5. In this mechanism, when the ball is held,
the position of the ball is recognized by infrared sensors, and the angle of thedribbling bar is adjusted based on the result. This makes it possible to keep the
position of the ball at the center of the dribbling bar at any time.

KIKS Extended Team Description for RoboCup 2020

5

In addition, even when the robot is dribbling, the angle of the dribbling-bar
can be properly adjusted by using the parallel link mechanism. Therefore, it isconsidered possible to keep dribbling without releasing a ball. Since the eﬀectiveness of those performances has not been conﬁrmed yet, detailed veriﬁcation
will be carried out in the future.

Fig. 5. 3D CAD concept image of new dribbler with parallel link mechanism.

2.3

Application to diagonal kick using movable dribbler

In RoboCup SSL, a wide choice for kicking mechanisms including straight kick,chip kick and curve kick is eﬀective in expanding the range of AI strategies.
The mechanism of the curve kick has been already reported by OP-AmP [4] andRoboTeam Twente [5], but we also considered its introduction. We propose a new
method for curve kick using a movable dribbler. Using the prototype mechanism
shown in Fig.6, we tried to realize a curve kick by applying a diagonal force to therotating ball. It is possible to achieve by changing the angle of the dribbling-bar
with a servomotor before kicking. It was conﬁrmed that the trajectory of the ballafter kicking can be controlled by adjusting the angle of the movable dribbler.
But, it does not implement yet, because of some problems are occurred for
mounting a dribbling motor and link mechanism compactly and for considering
the shape and material as same as conventional dribbler. In the future, due tocontrol the rotating speed of dribbler, the orbital radius of the ball will be able

6

Yusei Naito et al.

Fig. 6. Experimental prototype dribbler for the test.

to change and expand the range of tactics. It will also give advantage in penalty
kick.

3

Software design

We examined and improved the path generation and ball competition, whichhave been problems since last year. We aim to realize a smart game. In recent
years, the ﬁeld size and the number of robots are expanding in RoboCup SSL.
The conventional decision-making algorithm has problems such as not considering future prospects and poor expandability. It is impossible to realize cooperative actions such as passes during in play, and resulting in a low score rate.
Therefore, we tried to apply a technique that combines Monte Carlo tree searchand neural network, which has been successful in the ﬁeld of game AI in recent
years. We tested this algorithm in practice games.
3.1

Improvement of path planner

In previous system, robots were moving by path generation, but their performance was poor. Therefore, there were many collisions with opponent robots. In
this subsection, we describe about improvement for our route generation. The
current route generation method only generates a path in one frame, and does
not consider the moving direction and speed of the partner robot. Therefore, themoving position is predicted from the speed and the distance of the obstacle, andthe path is generated based on the predicted position. First, a prediction time
T is considered based on the distance to the obstacle closest to the target robot.
As the distance to the obstacle approaches, the time to collision decreases. If T
is ﬁxed, the risk of collision increases. Next, assuming that the obstacle moves
linearly at a constant speed, the position after T seconds is predicted. Finally,a path planner is examined using the predicted position as an obstacle. As the

KIKS Extended Team Description for RoboCup 2020

7

result, the motion corresponding to the action of the obstacle will be realized,
as shown in Route B instead of Route A in Fig.7.

Fig. 7. Improvement of path planner.

3.2

Introduction of Neural Network Libraries / Neural Network
ConsoleWe have introduced the Neural Network Libraries (NNabla) [6] and Neural Network Console[7] provided by Sony Network Communications Inc. to implement
Deep Learning. The neural network is designed, learned and evaluated using the
Neural Network Console, and inference is performed using C++ API of NNabla.
Neural Network Console can be operated with GUI, and trial and error is possible eﬃciently.
3.3

Decision-making algorithm using Monte Carlo Tree Search and
Neural Network

When a robot plays soccer, there are various action options such as pass, shoot,and dribble. Signiﬁcant rule changes in recent years have increased the number of robots and the size of the ﬁeld. Rule changes may continue in the future.
Therefore, decision-making for motion of the robots considering future prospects
will be more important in strategy. We implemented an algorithm that combines Monte Carlo Tree Search (MCTS) and Neural Networks with reference toAlphaZero[8, 9] developed by DeepMind. MCTS is a search method based on
simulation results. This method has been used in the ﬁeld of Go etc.[8, 9], but
we applied it to soccer by inferring the success probability of action.
Implementation of MCTS The nodes of the game tree have information of
the position, elapsed time, Action, Reward, and Visit-count for the robot in theﬁeld. The node transitions by Action. At this time, the transition probability
to each node is derived by inferring the success probability of Action using a
Neural Network. This inference is also applied to the MCTS Rollout (random

8

Yusei Naito et al.

simulation), and it is possible to obtain the expected value of the reward and to
search for promising actions. Typical game tree is shown in Fig.8.

Fig. 8. Typical game tree for action-decision.

MCTS is an algorithm that expands and searches a tree based on random
sampling. Compared to the αβ method generally used in Japanese chess (Shogi),
it has the advantage that there is no need to create and adjust complex evaluation functions. MCTS consists of ﬁve elements, i.e., Selection, Expansion,
Evaluation, Backup, and P lay for the process as shown in Table2. First, thealgorithm creates a root node with the state of the observed ﬁeld. Next, selection, expansion, evaluation, and propagation from the root node are repeatedaccording to the search situation until a certain time has elapsed. After that,the node with the highest number of selections is selected from the child nodes
connected to the root node, and the action to execute is made decision.
Table 2. Process elements on the Monte Carlo Tree Search
P rocess

Deﬁnition

Selection

Select the child node with the highest UCB1 value.
When a ball is in playing ﬁeld even if the search reaches a leaf node,
the node is extended in case that the Visit-count is greater thana certain value.
When a ball is in playing ﬁeld even if the search reaches a leaf node,
the result of Rollout (random simulation) from that state is added to
the reward of the node in case that the Visit-count is less thana certain value.
When a ball is not in playing ﬁeld and the search reaches a leaf node,
the state evaluation value is propagated to the parent node.
After the search is completed, the node with the highest Visit-countis selected, and the action to be performed is determined.

Expansion

Evaluation

Backup
P lay

KIKS Extended Team Description for RoboCup 2020

9

The UCB1 (Upper Conﬁdence Bound) [10] described in eq.(1) was used as
the selection criteria for child nodes to be searched. UCB1 is well known for themulti-armed bandit problem, and it is possible to search in consideration of theevaluation value and accuracy for decision branches. where, wi and ni show the
Reward and the Visit-count for nodes i, respectively. The t indicates the Visitcount for all of child nodes, c is the constant. In eq. (1), the ﬁrst term indicatesthe average of the reward, and the second term means the value that gives thereliability of the evaluation. The second term becomes larger as the Visit-count(evaluation value) to the node is smaller than the total Visit-count, and is easily
selected. That is, the evaluation value increases as the reliability of the ﬁrst term
decreases.
√wi
2log(t)U CBi =
+c
(1)
ni
ni

(a)Creation of a root node representing the current state(b)Extension from root node to node(c)Selection of the largest value of UCB and evaluation by Rollout(d)Extention from a node whose evaluation number reaches a certain level to a nodeFig. 9. MCTS execution procedure10

Yusei Naito et al.

The ϵ-greedy method, in which the evaluation value is randomly selectedwith a certain probability, was also studied. In this study, however, UCB1 thatmathematically guarantees convergence to the optimal solution was used. The
execution procedure of MCTS is shown in Fig.9. The update of the evaluation
value and the expansion of the tree are repeated in Fig.9 (c) and (d). Finally, theaction that transitions to the child node with the highest Visit-Count is executed
except the root node. For example, in Fig.9, an action to transition to any ofnodes B, C and D is executed.
Neural network design The success probability of actions such as pass andshoot is inferred by a neural network. The processing for the enemy robots is
performed one by one, and the inferred values of all the robots existing on theﬁeld are combined. Also, when creating teaching data and performing inference,
it was assumed that there was no enemy robot farther than the target positionat the start position of the action. That is, the inference was based only on the
distance to the robot located in front of the target position. For the input data,
it was used the ball speed, the distance between the ball and any enemy robot,
and the angle between the ball and any enemy robot. The output is that the ally
robot can either act or not, without intercepting the ball by the enemy robots.
The reachability inference NN described here was composed of all connected
layers, and several NNs with diﬀerent numbers of layers, nodes, and activation
functions were compared.
The success probability of Action is inferred by a Neural Network structurewith all layers of connections as shown in Fig.10(a). This network infers theprobability that an enemy robot will get a moving ball. This inference is applied
to all enemy robots on the ﬁeld to calculate the probability that the ball canmove at any position and any speed.
Learning for Neural Networks More than one hundred learning data wereobtained from the game simulation on grSim[11], and sent to Neural Network
Console. Execution example of learning process is shown in Fig.10(b). For thelearning, it was performed with a batch size of 32 and 2000 epochs.
Performance evaluation of neural network Figure10 shows the structure
and learning curve of the introduced reachability inference NN. In Fig.10 (b), thehorizontal axis shows the number of epochs (generations), the left vertical axis
shows the output of the cost function, and the right vertical axis shows the error
rate. TRAINING ERROR and VALIDATION ERROR indicate error rates forlearning data and for veriﬁcation data, respectively. The introduced NN has two
hidden layers. The ﬁrst hidden layer has three nodes, and the activation function
is ReLU. The second hidden layer has three nodes, and the activation function is
Swish. The time until completion for learning was 36.2sec. Obtained probability
are tabulated in Table.3. The accuracy of the inferred value was about 84%,
which was enough for use in the actual games.

KIKS Extended Team Description for RoboCup 202011(a) Neural Network structure(b) Typical Learning Curve
Fig. 10. Neural Network structure (a) and learning process (b).

Table 3. Probability predicted by Neural Network on Confusion Matrix (50 samples)
Data \ Predicted probability
FALSE
TRUE
Precision
F-Measures
Accuracy
Average of Precision
Average of Recall
Average of F-Measures

<0.5
18
1
0.94730.81810.84
0.86070.84
0.8376≥0.5
7
240.7741
0.857

Recall
0.72
0.96

12

Yusei Naito et al.

On the other hand, Table4 shows the accuracy of NNs with diﬀerent numbersof node in the hidden layers. As the result, it can be conﬁrmed that the accuracy
is lower than that of introduced NN.
Table 4. Comparison of Accuracy for NN under the various conditionsFirst hidden layer: 8,
First hidden layer: 5,
First hidden layer: 3,
Activ. func.: ReLU
Activ. func.: ReLU
Activ. func.: ReLU
Condition
Second hidden layer: 4, Second hidden layer: 3, Second hidden layer: 3,
Activ. func.: Swish
Activ. func.: Swish
Activ. func.: ReLU
Accuracy
0.76
0.76
0.64

Performance evaluation of decision making algorithms An example forthe action of this algorithm on grSim is shown in Fig.11 (Observation with
AutoReferee [12]). The circle in the lower center represents the ball, and themarkers numbered B or Y indicate the robots of each team. First, at the lower
left in Fig.11, it can be seen that the B5 robot is moving toward the opponent’s
goal after passing the ball toward B1. After that, by receiving the ball again
from B1 in front of the goal (ducking and weaving the opponent robots, Y1 andY4), it is found that the B5 robot will attack toward the goal. We introduced
this algorithm to real game (RoboCup JapanOpen 2019) and conﬁrmed that aneﬀective pass play could be realized. Furthermore, we plan to use the selection
probability and evaluation value for Action as the inference value of Neural
Network. As the result, the eﬃciency of the tree search is improved, and therange of the decision branch for the action will be wider.

Fig. 11. An example for the action of this algorithm on grSim.

Planned Neural Network structure is shown in Fig.12. In this network, we
consider a matrix of 1m intervals. It receives three channels as input, i.e., theKIKS Extended Team Description for RoboCup 202013

positions for a ball and both team’s robots, and outputs the expected score
and the distribution of the selection probabilities for action candidates. One ofthe problems of this method is that the computational complexity is greaterthan conventional methods. As the solution, now we are studying about the
introduction of GPU for Neural Network processing, and the separation andparallelization of this method.

Fig. 12. Planned Neural Network structure for inferring expected score and the distribution of the selection probabilities for action candidates.

3.4

Game reporter for the audience

In SSL game, there are many complicated and speciﬁc rules for the league, andbecause of the oﬀense and defense change quickly, it is diﬃcult to understand thegame situation for the audience. Therefore, we studied about the game reporter14

Yusei Naito et al.

system that automatically explains the game situation of SSL to the audienceby artiﬁcial voice. Rough process ﬂow for the game reporter is shown in Fig.13.

Fig. 13. Rough process ﬂow of game reporter.

(a)

(b)

Fig. 14. Typical image on the game reporter, for counts of events in each team (a) and
explanation to speech (b).

Parsian reports the game reporter function as one of the log analyzers in2019 ETDP[13]. We have created a smartphone application that provides game
event information to the audience. To explain the game situation, it is necessary
to create a rule base corresponding to the situation. It is especially important tounderstand the features of the SSL game and the diﬀerences from human soccer.

KIKS Extended Team Description for RoboCup 202015

We investigated the number of events such as goals, passes and rule violations
from the game log. For example, the number of events in each team at the
SSL Final at RoboCup 2019 is shown in Fig.14(a). This result will be helpful
in explaining the diﬀerence from the actual human soccer game. Figure14(b)shows an application screen image for visually displaying the game situation. In
the future, we will introduce a function to explain the reason of the foul, thesituation of the game and competing teams introduction by local language, andcreate a rule base for giving an appropriate explanation according to the progressof the game. We think that the explanations in local language are particularly
eﬀective in attracting children’s interest.

References
1. Zheyuan Huang, Lingyun Chen, Jiacheng Li, Yunkai Wang, Zexi Chen, LichengWen, Jianyang Gu, Peng Hu, and Rong Xiong: ZJUNlict Extended Team Description Paper for RoboCup 2019, https://ssl.robocup.org/wp-content/uploads/2019/03/2019 ETDP ZJUNlict.pdf (2019).
2. Takamichi Yoshimoto, Takato Horii, Shoma Mizutani, Yasuyuki Iwauchi, and Shota
Zenji: OP-AmP 2019 Extended Team Discription Paper, https://ssl.robocup.org/
wp-content/uploads/2019/03/2019 ETDP OP-AmP.pdf (2019).
3. Nicolai Ommer Andre Ryll and Mark Geiger: TIGERs Mannheim Extended Team
Description for RoboCup 2019, https://ssl.robocup.org/wp-content/uploads/2019/03/2019 ETDP TIGERs Mannheim.pdf
4. Takamichi Yoshimoto, Takato Horii, Shoma Mizutani, Yasuyuki Iwauchi,Yutaka
Yamada, Kousei Baba, and Shota Zenji: OP-AmP 2017 Team Discription
Paper,
https://www.robocup2017.org/ﬁle/symposium/soccer sml size/
Robocupssl2017-ﬁnal9.pdf (2017).
5. Cas Doornkamp, Zahra van Egdom, Gaël Humblot-Renaux, Leon Klute,
Anouk Leunissen, Nahuel Manterola, Sebastian Schipper, Luka Sculac, Emiel
Steerneman, Stefan Tersteeg, Christophe Vanderwalt, Wouter van Veelen,
Haichuan Wang, Jeroen Weener, Jelle Zult: RoboTeam Twente 2018 Team Description Paper,https://ssl.robocup.org/wp-content/uploads/2019/01/2018 TDP
RoboTeam Twente.pdf (2018).
6. Neural Network Libraries, https://nnabla.org
7. Neural Network Console, https://dl.sony.com
8. David Silver, Julian Schrittwieser, Karen Simonyan, Ioannis Antonoglou, Aja
Huang, Arthur Guez, Thomas Hubert, Lucas Baker, Matthew Lai, Adrian
Bolton, Yutian Chen, Timothy Lillicrap, Fan Hui, Laurent Sifre, George van den
Driessche, Thore Graepel, Demis Hassabis: Mastering the game of Go without
Human Knowledge, nature, 2017. https://deepmind.com/research/publications/mastering-game-go-without-human-knowledge
9. David Silver, Aja Huang, Chris J. Maddison, Arthur Guez, Laurent Sifre, George
van den Driessche, Julian Schrittwieser, Ioannis Antonoglou, Veda Panneershelvam, Marc Lanctot, Sander Dieleman, Dominik Grewe, John Nham, Nal Kalchbrenner, Ilya Sutskever, Timothy Lillicrap, Madeleine Leach, Koray Kavukcuoglu,Thore
Graepel & Demis Hassabis: Mastering the game of Go with deep neural networks and tree search, nature, 2016. https://storage.googleapis.com/deepmindmedia/alphago/AlphaGoNaturePaper.pdf

16

Yusei Naito et al.

10. Peter Auer, Nicolò Cesa-Bianchi, Paul Fischer: Finite-time Analysis of the Multiarmed Bandit Problem, https://link.springer.com/article/10.1023/a:1013689704352
11. Monajjemi, Valiallah (Mani), Ali Koochakzadeh, and Saeed Shiry Ghidary; grSim– RoboCup Small Size Robot Soccer Simulator, In Robot Soccer World Cup, pp.
450-460. Springer Berlin Heidelberg, 2011.
12. TIGERs Mannheim AutoReferee, https://github.com/TIGERs-Mannheim/
AutoReferee
13. Kian Behzad, Elham Daneshmand, Nadia Moradi, Mohammad Reza Kolani, Mahdi
Hajimohammadi Onidin, Yasamin Alizadeh Gharib, Atiyeh Pirmoradi, Mohammad Mahdi Rahimi, Mohammad Mahdi Shirazi, and Mohammad Azam; PARSIAN 2019 Extended Team Description Paper https://ssl.robocup.org/wp-content/
uploads/2019/03/2019 ETDP Parsian.pdf (2019).