HfutEngine2021�Team Description Paper
Wang junyang, Cao Mingkuan, Wang Hao and Fang Baofu
Artificial Intelligence and Data Mining Lab
School of Computer and information
HeFei University of Technology, P.R.China
HfutEngine2D2d@gmail.com

Abstract. This paper mainly describes the related work of HfutEngine2D
in 2019-2021, including the optimization of running mode, the optimization of passing and the application of logistic regression model. Combinedwith the characteristics of HfutEngine2D, this paper expounds the algorithm concept, design intention, application process and progress indetail. In our recent test with other RoboCup strong teams, the strengthof HfutEngine2D team has been significantly improved, and for the longterm development of the team, these improvements have also played agreat benefit.
Keywords: Point - frame running pattern, breakthrough pass, logistic
regression model

1 Team overview
HfutEngine2D was founded in 2002 and began to participate in the China robot
competition in the same year. In the next few years, HfutEngine2D developed
rapidly and participated in various competitions. Since 2003, we have been using UVA Base2003 as our bottom code, with the update of server version, weuse BP neural network algorithm in intercepting learning, value-based learningalgorithm in action selection, reinforcement learning algorithm and a series ofmachine learning algorithm in passing action learning, which has achieved goodresults. We have won the runner up of 2007 RoboCup China open, the seventh
place of 2008 World Cup RoboCup, the bronze medal of 2008 Iran open, thefourth place of 2009 RoboCup China Open Simulation Group 2D, the champion of 2009 Iran open, and the simulation group 2D of 2011 RoboCup Chinaopen. We use agent2d as our underlying code to build HfutEngine2D in 2014.
In the 2015 World Cup, we finished eighth in the simulation 2D group. We alsoparticipated in the 2016 and 2017 World Cup. In recent years, we won the championship in the RoboCup robot world cup in China in 2019, the 10th place in
the RoboCup robot world cup in 2019, and the runner up in the RoboCup robot
world cup in China in 2021.

2

HfutEngine2021�Team Description Paper

2 Running mode
2.1 existing mode
As a sport of 11 players, football can’t win without mutual cooperation, whichis also the case on our simulation platform. In our attack, although the decisionmaking of the players with the ball is very important (the rewriting of the action
chain is very important), no matter how good the decision-making is without
the cooperation of other players and making the corresponding running position
without the ball. The corresponding performance of this situation on the court is- we have very few offensive methods, the ball can not run between the offensive
players, unable to form a subtle cooperation. And that’s the problem we’re facing
now. In recent versions of the team, we’ve been improving the running mode.
In HfutEngine 2019-2021, we constantly modify the running mode of thebottom layer, but it is basically based on the following mode: the running modeof the bottom layer is based on the standard point determined by the formation
file, and a unique standard point can be obtained according to the distribution ofplayers on the field and the coordinates of the ball. On the basis of the standard
point, different additional displacements are introduced to classify the current
situation on the field to get the final result The goal is to move the position, sothat the formation can be maintained in the game and the players can be evenly
distributed.

Fig. 1. running mode based on formation points

This mode has a good effect at the beginning, but with the deepening of
the improvement, its disadvantages are gradually exposed. When a run is not
good, you can only separate this situation and make a separate strategy, but
the situation on the court is unlimited. With the refinement of classification,
the number of codes will increase greatly. If there is no detailed comments, thereadability is extremely low, which brings some problems, the effect is very little.
And the basis of this model is that we need to preset an optimal running point.
However, we are not professional football players. In many cases, we can notdetermine the optimal running point or even an area. At this time, we need tocontinue to explore and supplement this model.
2.2 improvement ideas
In order to solve these problems, we use a new mode to determine the runningpoint, which is based on the scoring mode. First, according to the coordinatesHfutEngine2021�Team Description Paper

3of the ball and players to determine the running frame, and then do discrete
processing to the running frame, traverse the points in the frame, into the scoringmachine, so as to select the best running point. Using this mode can get rid of thetrap of constantly refining classification, and it is more flexible, controllable, andthe code can be more concise, which is more conducive to the future development.

Fig. 2. point frame running mode

As shown in (Fig. 3 shows an example3)., at this time, No. 7 of our teamholds the ball, and the next running position of No. 11 player is very important.
According to the new mode, first determine the running position box, which isthe black box on the graph, and discretize the running position box, which isshown as the white point on the graph. Replace all the white points into thescorer, and the one with the highest score is the target running position point.
Of course, No. 9 and No. 10 are also doing this Active running without the ball,compared with the previous formation point mode, this mode is easier to findthe opponent’s defense flaws.

Fig. 3. point frame running mode

4

HfutEngine2021�Team Description Paper

2.3 Application of improvement in exploration
The key of this model is the establishment of scorer. Although every good ballhas something in common, the definition of good ball is different for different
players and different situations, so the classification is still necessary. However,the classification here only needs to be clear about the major categories, andit does not need to be accurate to the details. However, the workload is stilllarge. At present, we have established a basic scorer, which only aims at theformer The front runs without the ball when attacking. After the improvement,
we tested several RoboCup teams. Although we have made some improvements,
we think it has unlimited potential, especially after the application of machinelearning algorithm, there should be more improvements.
Table 1. comparison of test results
Opponent Original mode New modevs TJPU-Base78.67%
81.31%
vs Miracle_2D
64.37%
70.69%
vs Apollo2D
63.58%
80.21%vs MT2020
47.98%
55.49%
vs YuShan2020
29.31%
36.56%

3 Application of logistic regression model in passing
3.1 Question raising
In RoboCup Simulation 2D game, the process of bringing the ball from our half
court to the other side’s restricted area can best reflect a team’s offensive ability
and comprehensive strength. Because only when the ball is successfully brought
into the penalty area of the opponent’s court can we have the chance to break
the goal, and keeping the ball in the penalty area of the opponent is also very
helpful to enhance our attack threat. Because of this, the designers of all teams
will try their best to prevent the other side from bringing the ball into theirown restricted area on the defensive side and bring the ball into the other side’srestricted area on the offensive side when designing team strategies.
In our previous design version, the usual defensive strategy to face this problem is to intercept or tackle the ball to prevent the opponent from entering thepenalty area. It is difficult to find an effective attack strategy. The attacking
party should try its best to break through the defense and transport the ballto the other side’s restricted area. This is a very difficult thing, but once it isreached, the dividend is huge. In the team version of HfutEngine2017, we try tobring the ball into the opponent’s penalty area by designing the pass strategy fora specific area of the course. In developing HfutEngine 2019, we try to improveHfutEngine2021�Team Description Paper5

the scoring of specific areas in the course in the evaluator to keep the ball in theenemy’s restricted area as much as possible.
In last year’s HfutEngine2020 version, we tried to use the formation editor
to fine adjust the running position in the game, so that our players can bring
the ball into the enemy’s restricted area as much as possible. However, in theactual video observation of the game, we found that often some can penetrate
the enemy formation and pass the ball of several enemy players at one time
(we call it ”breakthrough pass”) is the main reason to bring the ball near theenemy’s restricted area. In a game, it is often such several opportunities thatlead to goals, even when there are multiple breakthroughs The percentage ofwinning games is very high. This makes us have to start to explore whether
there is a connection between breakthrough passing and goal difference, andbetween breakthrough passing and victory in the game, and how to improve theproportion of breakthrough passing in the game.
3.2 further research on breakthrough passing
Through the analysis of the game log, we can find out which passes are breakthrough passes. We use regular expressions to process the log information, andthen use some criteria to determine whether each pass is a breakthrough pass.
Then, in order to explore whether there is a relationship between breakthroughpassing and the number of goal difference, we analyze the correlation between
the number of net breakthroughs and the number of goal difference, that is, toobtain the Pearson correlation coefficient of the two. In 100 games randomly
selected, the scatter diagram of breakthrough pass and goal difference is as follows:
After calculation, the correlation coefficient between them is 0.7654. Whenlooking for a breakthrough pass, we only consider the situation of midfield breakthrough, and whether we can score after breakthrough is also affected by manyfactors, such as the offensive strategy of the attacker and the defensive strategyof the defender in the restricted area, and the defensive strength of the goalkeeper all affect the final result, which explains why the correlation coefficient
has a certain gap from 1. However, it can be seen from the existing results that
there is a great positive correlation between the number of breakthrough passes
and the number of goal difference.
3.3 logistic transformation and logistic regression model
When we establish the regression equation, the value range of the dependent
variable is the real number set; when we feel whether to pass the ball, the valueof these dependent variables is between 0 1, such as 0.1, 0.5, 0.8, etc., so weneed to do logit transformation on the value of the dependent variable (targetprobability).
Let p be the probability of occurrence of an event and 1−p be the probabilitypis called the occurrence ratio of an event, which isof non occurrence, then 1−precorded as odd

6HfutEngine2021�Team Description PaperFig. 4. point frame running mode

(logit (p) = lnp1−p)(1)

Obviously, when p = 1 ,logit(p) is + ∞; when p = 0.5 ,logit(p) = 0, whenp = 0 ,logit(p) is − ∞. In this way, the value range of dependent variable isextended from 0 ∼ 1 to real number set, and the regression analysis using thismethod is logistic regression.
There is a binary dependent variable y, when the value is 1, it means that theevent happened, when the value is 0, it means that the event did not happen; thedependent variable has m influencing factors (independent variables):x1 , x2 , · · · , xm .
The conditional probability P (y = 1|xi ) = pi . Then the logistic regression modelof pi (the ith observation) was as follows�(logit (pi ) = lnpi1 − pi)= β0 +β1 x1 +· · ·+βm xm = β0 +m∑βj xj , j = 1, 2, · · · , m

j=1(2)
Where βj is the independent variable, when xj changes a unit, the change oflogit(pi ) can be understood as the weight coefficient of each influencing factor.
Through transformation, logistic regression model can also be written as follows:∑mpi =eβ0 +1+e

j=1 βj xj∑
β0 + mj=1 βj xj(3)

HfutEngine2021�Team Description Paper

7

3.4 application and result exploration
We first extract the position and speed of players and balls in each cycle fromthe log. And according to the direction of the pass, the number of passes in thepass and other standards, judge whether the pass is a breakthrough pass. After
that, take the data of breakthrough pass as positive example, and those notjudged as breakthrough pass as counter example. Use the position and speed of
the current team member, the passing team member and the opponent on thepass route as input, and whether it is breakthrough pass as output to train thelogistic regression model. After training, the accuracy of the model is 87.34
Then, when applying the model, we modify the relevant part of the evaluatorin the team source code, modify the score of the breakthrough pass, and if thecurrent pass is predicted to be a breakthrough pass, improve the score of thisaction, so as to improve the number of breakthrough passes. After adding theclassifier, the test is carried out. In the face of the 2021 version of some RoboCupteams, the test winning rates before and after the modification are as follows:
Table 2. comparison of test results
Opponent Logistic regression was not used Logistic regression was used
vs TJPU-Base
80.00%
89.31%
vs Miracle_2D
47.50%
65.97%
vs Apollo2D
67.00%
84.81%vs MT2020
50.34%
53.67%
vs YuShan2020
25.00%
33.46%

4 Further work
In recent years, more and more progress has been made in intensive learning andhas solved some problems such as GO, DOTA, StarCraft, etc. From simulation
games, human-computer games to robot arm control, automation system, andeven to recommendation engine and natural language dialogue system on Web
system, the application of reinforcement learning is everywhere.
RoboCup is a sequence decision-making problem, and it is also Suitable touse reinforcement learning to improve. but the current problems in Reinforce
learning include credit distribution, large scale and low sampling efficiency. In
order to better promote RoboCup soccer and strengthen the learning community to carry out cooperation and exchange. based on RoboCup Simulation 2D
platform SoccerMARL is proposed to develop a team coordination strategy intelligent decision system through deep reinforcement of learning and multi-agent
cooperation. The project uses the deep neural network trained by actor criticalmethod to evaluate the behavior selection in the robot tactical cooperation, andmakes the analysis and decision of the action sequence. Finally, the feasibility

8

HfutEngine2021�Team Description Paperand application value of multi-agent cooperation are explored through experiments.

References
1. Hidehisa Akiyama and Tomoharu Nakashima. Helios base: An open source package
for the robocup soccer 2d simulation. In Sven Behnke, Manuela Veloso, Arnoud
Visser, and Rong Xiong, editors, RoboCup 2013: Robot World Cup XVII, pages
528–535, Berlin, Heidelberg, 2014. Springer Berlin Heidelberg.
2. Fang Baofu. Robot soccer simulation [M]. Hefei University of Technology Press,2011
3. Xu Xiaoxing. Formation and pass coordination of 2D simulation soccer robot system[D]. Nanjing: Nanjing University of Posts and telecommunications, 2016
4. Nagae N, Igarashi H, Ishihara S. Return Pass in RoboCup 2D Simulation League:
State Prediction and Evaluation Function of the Chain Action[C]// Proceedings ofthe Fuzzy System Symposium. 2015. Iran:Shiraz University.2016.
5. Zhou Zhihua. Machine learning: = machine learning [M]. Tsinghua University Press,
2016
6. Long Defu. Mining and application of RoboCup Simulation 2D game log [D]. Guangdong: Guangdong University of technology, 2016
7. Li Xuejun. Local pass and tactical cooperation of soccer robot [D]. Nanjing: Nanjing
University of Posts and telecommunications, 2016
8. Peng Jun, Wu Min, Cao Weihua, key technology of RoboCup robot soccer simulationcompetition [J]. Computer Engineering, 2004,30 (4): 49-51
9. Chen rongya. Research on player cooperation strategy in simulated robot soccer[D]. Hefei: University of science and technology of China, 2015
10. Circus, logistic regression model, blog Garden, [EB] / [OL], https://www.cnblogs.
com/zm-pop-pk/p/11668294.html