HfutEngine2022 Simulation2DTeam Description PapeYansong Yu,Gongfang Zhao
Hao Wang, BaoFu Fang
April 3, 2022
Abstract
This paper mainly describes the related work of HfutEngine2D in 2021 -2022, including using
genetic algorithms to adjust parameters and trying to use Neural Networks to make more rational
decisions. Combined with the characteristics of HfutEngine2D, this paper expounds the al-gorithm
concept, design intention, application process and progress in detail. In our recent test with other
RoboCup strong teams, the strength of HfutEngine2D team has been sig nificantly improved, andfor the long-term development of the team, these im provements have also played a great benefit.

1Introduction

HfutEngine2D was founded in 2002 and began to participate in the China robot competition in the sameyear. In the next few years, HfutEngine2D developed rapidly and participated in various competitions.
Since 2003, we have been us-ing agent-2d as our bottom code, with the update of server version, weuse BP neural network algorithm in intercepting learning, value-based learning algorithm in action
selection, reinforcement learning algorithm and a series of machine learning algorithm in passing action
learning, which has achieved good results. We have won the runner up of 2007 RoboCup China open,
the seventh place of 2008 World Cup RoboCup, the bronze medal of 2008 Iran open, the fourth place of2009 RoboCup China Open Simulation Group 2D, the cham-pion of 2009 Iran open, and the simulationgroup 2D of 2011 RoboCup China open.We use agent2d as our underlying code to build HfutEngine2Din 2014.I n the 2015 World Cup, we finished eighth in the simulation 2D group. We also participatedin the 2016 and 2017 World Cup. In recent years, we won the cham-pionship in the RoboCup robot
world cup in China in 2019, the 10th place in the RoboCup robot world cup in 2019, and the runner
up in the RoboCup robot world cup in China in 2021.

2

model and approach

2.1

genetic algorithms

In our robot team, there are tens of thousands of parameters that need to be adjusted, because of thelarge number of parameters, the ”brute force method” is expensive. In this case, genetic algorithm ist he best choice. The selection of criterion, the setting of variation and the initial value are all very
important in genetic algorithm, otherwise it is easy to be trapped in local optimization.
In HfutEngine 2022, we divide the modification targets into three categories :1. Whether a pieceof functional code exists; 2. Parametric data. Then we put all the parameters in a set of code as
an individual, a group of such an individual as a species, and they have different genes, we put on adifferent team winning percentage as fitness function evaluation standard, first record the match ineach iteration and the corresponding parameters, and then according to the fitness eliminate a certain
number of teams, And let all individuals produce a certain amount of variation, in the process ofcontinuous iteration, we gquet better individuals.
The genetic algorithm uses 10 individuals as a generation, and uses the 100- game winning rate
for different teams as the fitness to eliminate underperforming individuals, and adjusts the individual1differences through simple random up and down. We use about 50 generations of variation. It is foundthat the win rate peaks, and then the win rate stabilizes around this value.

Figure 1: Flow-process diagram of genetic algorithm

2.2

neural network

We once used the deep learning method to optimize the action and decisions of a single player, butthe overall decision. We hope to evaluate the scoring function of high-level actions (passing, dribbling,shooting) through the neural network, learn strategy, do not improve personal behavior, but improveteam strategy.
To get a better strategy, we let the stronger teams play with us, get the log data, record theposition of the positive and negative action ball before the strong teams, find the offensive and defensive
positions of the strong teams, and thus imitate the strategy of the strong teams We trained the neuralnetwork through supervised learning, setting positive and negative events with a target value of 1 / 0.
We built neural networks using the pytorch framework,The input layer is the position of the next
kick, and the fragments extracted from the log are converted to the training data, with the input
features (x, y) and positive and negative target values.The output layer is the probability of a sigmoid
function —— entering the opponent’s penalty area, assuming that team members will consider teamstrategy to approach experts by modelling each player’s action selector as an expert action selector.Take
the ball into the penalty area and have a low action score with low success probability.The neuralnetwork structure is shown in the Figure 2.
We import the libtorch library at compile time by rewriting the cmakelist to deploy the pytorchtrained model. Finally, the results of neural network evaluation and our already existing evaluation
functions combine the strategy of optimizing the overall system by changing the estimator.

2Figure 2: Neural network structure

3

partial optimization

3.1

Tackle optimization based on tackle rate

We take the probability of tackle success and the probability of foul as input vectors, and the valuesof T and F of the input vectors are output online by the player agent and saved to a file. Saves theperiod at which the tackle action is performed, the defensive player number, the enemy offensive player
number, the T value, and the F value. The cycle and offensive player numbers are used to quickly
locate player-related information when analyzing Rcg files. According to whether the player performs
a dash atomic action after being shoveled, the tackle action is labeled. In the action information inRcg, there is a statistical number of atomic action dashes dash count. If the tackle fails, the shoveled
player will generally continue to carry the ball forward, and if the tackle is successful, the shoveled
player will not immediately perform the dash action.
Train the model using the logistic regression method to decide whether to tackle or not. The
network framework is pytorch, the model structure is shown in the Figure 3, the model deployment
mode is the same as above, the number of iterations T = 1500, and the learning rate is α=0.03.

Figure 3: Neural network structure33.2

Defensive optimization

When the opposing player enters our half, the most important thing is to intercept the passing ballof the opposing team and disrupt the passing strategy of the opposing team. In order to improve thesuccess rate of interception, it is a key step to choose the interception position in advance.
1.The position of the goalkeeper was moved back according to the position of the ball.
2.Added condition judgment and position selection for defenders to intercept shots in front of goal.
According to the characteristics of the attack, the distribution of the ball, goalkeeper position selection,and scoring.
3.Added pass-block interception mode, which predicts the next possible receiving opponent basedon the distribution of the opponent holding the ball and other opponents, and selects the interception
position and scores accordingly.
4.According to the position selection of ball-block interception mode, the position of interception
is predicted and scored by the state of the ball. Conclusion: Increasing the more likely positions
and reducing the less likely positions through judgment and conditional screening, and increasing thesuccess rate of interception after scoring. Future direction: Optimize the evaluation model to increase
the success rate of ball interception.

4

Experiment

Conducting 50 game tests to count the number of tackle successes and failures of Full Back , This isshown in the chart.the tackle failure rate dropped from 7.39% to 5.87%, indicating that the full-back’s
new tackle strategy had some effect, reducing the possibility of tackle failure.

we collected the deployment of HFUTengine2022 after the genetic algorithm, the winning rate of
HFUTengine2022 for each domestic team, after 200 tests, the test results are as shown in the chart, it
can be seen that after the genetic algorithm is adjusted, the winning rate of each team has improved.

5Conclusions

HfutEngine2D has undergone some optimizations based on neural networks and genetic algorithms.
We briefly describe the basic principles and implement them according to their own characteristics. Atthe same time, we have done some work on the optimization of the strategy and the optimization of
the power model. HfutEngine2D is now a big improvement over the previous version. Tackle successrates have improved and attacks in the penalty area are more efficient. In the next phase, we will try

4

to train the model to analyze the opponent’s offense to improve defensive ability, and optimize thecurrent imitation learning model to construct a search tree so that it can achieve multi-step evaluation.

References[1] Hidehisa Akiyama and Tomoharu Nakashima. Helios base: An open source package for the robocup
soccer 2d simulation. In Sven Behnke, Manuela Veloso, Arnoud Visser, and Rong Xiong, editors,RoboCup 2013: Robot World Cup XVII, pages 528–535, Berlin, Heidelberg, 2014. Springer Berlin
Heidelberg.
[2]Fang Baofu Robot soccer simulation [M] Hefei University of Technology Press, 2011[3]Wang Junyang, Cao Mingkuan, Wang Hao and Fang BaofuHfutEngine2021 Team Description Paper2021[4]Li long Value based machine learning method and its application in RoboCup Simulation 2D [D]Hefei University of technology, 2009[5]Mimicking an Expert Team Through the Learning of Evaluation Functions from Action Sequences[C]//
Takuya Fukushima, Tomoharu Nakashima, Hidehisa Akiyama,2018[6]. Zhou ZhiHua.Machine learning[M].tsinghua university press, 2016.
[7]. Peter Harrington. Machine learning practice[M]. People’s Posts and Telecommunications Publishing House, 2013.
[8]. Toby Segaran. Collective Intelligence Programming [M]. Publishing House of Electronics Industry,
2015.
[9]. Ian Goodfellow, Yoshua Bengio. Deep Learning [M]. Posts and Telecommunications Press.2017[10]. Zhuang Yanbin, LI Chunguang, Yang Xiaoyun. Research on the new Method of RoboCup RobotTackle based on BP neural Network[11].Chen rongya Research on player cooperation strategy in simulated robot football [D] Universityof science and technology of China, 2015[12]RoboCup Official Site, https://www.robocup.org/.

5