Persepolis Soccer 2D Simulation TeamDescription Paper 2021
Morteza Noohpisheh1 , Maryam Shekarriz1 , Fatemeh Zaremehrjardi2Farnaz Khademi Ardekani3 , and Seyedeh Afsoon Khorsand4
Dept. of Computer Engineering, Shiraz University, Iran
Dept. of Computer Science, St. Francis Xavier University
Dept. of Biomedical Engineering, Kazeroon University
Dept. of Industrial Management, Zand University
Morteza.Noohpisheh@gmail.com
m.shekarriz31@gmail.com
X2020fpvv@stfx.ca
KhademiFarnaz4@gmail.com
AfsoonK94@gmail.com

Abstract. This paper includes some explanations about algorithms implemented by Persepolis team members. We will introduce algorithms
that are used for pass, shoot, dribbling and marking, and in particularwith the ranking algorithm for the states of chain action. The base codeused by Persepolis is agent-2d 3.1.1[17].
Keywords: RoboCup · Soccer Simulation · AI.

1Introduction

The goal is to implement intelligent agents, that have the ability to decide on
using soccer individual skills as well as the ability to work with other agents inthe soccer field. In addition, programming has been done by agents to improve
individual skills (shoot, pass, dribble, positions of players on the field). Razi teamis formed from students, has started its activities from November 2012 and has
made plan in order to achieve its goal, participating in World Cup competitions.
This team has participated in prestigious competitions and has won the third
place in IranOpen2017 and IranOpen2018. This team has begun to use artificial
intelligence in its decision making. It continues its work with the aim of using
artificial intelligence algorithms and online analysis.
Note : The name of this team was changed from Razi to Persepolis after theJapan Open 2020.

2Activities Done in Other Iranian Teams

In the recent years, the following tasks have been done in other Iranian teams: Aformation 2 detection system as well as a software called Tournament Planning2and Analyzing Software(TPAS) have been developed by Namira[5][6][7]. Dynamic formation changing, field evaluation system for offense, field evaluation
system optimization, Optimizing defensive decision making by using message
passing between agents, Shooting behavior optimization using Deep Neural Network, Predicting opponent’s behavior using Rough Neural Network, Predicting
the behavior of the agents with ball possession, Defensive Decision Making using Deep Reinforcement Learning have been developed by Cyrus[8][9][10][11][13].
Nexus has implemented Reinforcement Learning for decision making in penalty
area[14].

3Previous Works in Team

2018: Razi has organized its own system for rating states in chain action[20].
2019: Use the Gravity Strategy[21].

4

Offensive Strategy

In the 2D soccer simulation league, the field state has many properties describing
offence and defense, such as the relative positions, the distance from the ball,the members in a certain region and so on. It is easy to see that the field stateis a continuous space. To deal with the state data quickly and effectively, weneed to divide them into many equivalent states by a process of discretization.
In the course of a soccer game, the attacking strategy varies with the state.
That is to say, attacking strategy is a piecewise function: the best attack action
may be different if the field state changes. Furthermore, we must identify thebest action for each state. In this strategy we set a path for each action in eachstate, and found the best action in a given state via the preferences of the paths.
We should regard each attack action as a path in the solution space, consider
each agent’s training in the same way we would foraging behavior of an ant, andtreat an effective attack as success foraging. In order to solve the problem of “asolution per state”, we set an ant group for each equivalent state. Within thissetting, each ant lays down a trail pheromones on a path (corresponding to thepreference value of an attack action), and each ant group may have a path thathas the highest pheromone level (corresponding to the best preference value andthe best attack action).
4.1

Offensive strategy algorithm

We can simulate different attacking environment by setting agents at different
points on the field. Regarding three attack actions (shooting, passing and dribbling) as three different paths for foraging, we can obtain the best attack action
by using the preference values of ants’ foraging behaviors. The following twosteps should be made in order to find the best attack action: 1) train the preference values of actions in each environment, and 2) select the best attack action

Persepolis Soccer 2D Simulation Team Description Paper 2021

3based on preference value. The first step runs offline in the background for along time to obtain the preference value; the second step is designed using thedecision module of the attacker and runs online to select the best action. The
state of attacking environment is a continuous space. In order to deal with thestate data quickly and effectively, we should transfer them into many discrete
equivalent states. For every attacking environment (state), we generated a groupof ants and used pheromone concentration to record the dynamic success-rate ofeach attack action.

5

Experimental evaluation

After being created, the 2D soccer simulation team with the MACO[16] wasrun in the RoboCup simulation platform to evaluate our new algorithm. Firstly,
every agent took random attack actions. The results of these actions have beenrecorded and used for the next training session. The underlying code and underlying database are Agent2d-3.1.1 and Librcsc, respectively. The positions of
all players (two attackers, two defenders and one goalkeeper) were set randomly
by an offline coach. After five cycles of competition, the results of two teams arerecorded as the format of “αshoot, dshoot, αpass, dintp, ddrib, dintd, aa, suc”.
The training will stop after 150,000 times. The training scene is shown in Fig.
1. In order to show the situations of the attack and defense sides more clearly,
only the penalty area is depicted.

Fig. 1. Screenshot of training scene arranged randomly.

In the following experiments, the maximum value for each state data (αshoot,dshoot, αpass, dintp, ddrib, dintd) was defined as (π/2, 15, π /2, 15, 3, 3),
respectively, with each state discretized into 30 elements. If state data are larger
than the defined maximum, the discretized data will be the maximum (e.g., if
the value of αshoot is equal to 2π/3, then the discretized value of αshoot willbe 30). The number of states was 2,700 (3*302), the number of groups was
2,700 and the number of ants was 8,100 in total. The adjustment factors forpheromones and heuristics were set as follows: α=2, β=4. In order to keep the4previous experiences obtained during the pre-event training, we set the coefficientof pheromone evaporation at a low level (ρ=0.1).
5.1

Competitions with baseline strategy

Application effect in competition is the sole criterion for evaluating an offensivestrategy. This section concerns our creation of a 2D soccer team named Persepolis, which employs the MACO algorithm as the offensive strategy. Meanwhile,using the underlying code Agent2d-3.1.1, which is an open source by Helios (former world champion in the RoboCup 2D simulation league), we created anotherteam, which adopted the searching-space offensive strategy. The only difference
between the Persepolis team and the BASE team is the attacking selection strategy. There were 10 games in total in the test, with each game running 6,000
standard clock cycles. The results of the games are presented below: As listed
in Table 1, the Persepolis team dominated in the games with 53.4% possession
and 82.61% passing-success on average. As a result, the Persepolis team enjoyeda 100% winning advantage with 3.4 goal differences per game, as well as scoring
three times as many goals as its opponent on average.
Games Possession Passing Success Goal Scored Goal Against Result Goal Difference
153.2%
80.1%
51
Win 4
2
54.1%
82.8%
41
Win 3
352.8%
81.7%
62
Win 4
453.9%
83.1%
42
Win 2
554.2%
82.6%
31
Win 2
651.7%
82.1%
51
Win 4
753.4%
83.2%
42
Win 2
852.4%
84.2%
42
Win 2
954.6%
83.6%
63
Win 3
1053.7%
82.7%
52
Win 3
Table 1.

6Conclusions

The offensive strategy is an important factor in the attacking capability of asoccer team. Given the unpredictability of the environment and the influenceamongst agents, it is very difficult to select action for the ball-handler. Attacking strategy has becomes a hotly discussed issue in the domain of multi-agent
cooperation in a real-time, asynchronous and noisy environment. We analyzed
the principal factors of the main attack actions for the ball-handler in the 2D
simulation league and presented a novel offensive strategy algorithm based on
multi-group ant colony optimization (MACO-OS) in order to improve attackingefficiency. Using the pheromone evaporation mechanism, the algorithm creates

Persepolis Soccer 2D Simulation Team Description Paper 2021

5an AIT, which is the basis of decision-making action. Our future work will focus on defensive strategy and defense conversion strategy to enhance the team’s
overall performance.

References
1. Frieder Stolzenburg, OliverObst, Jan Murry, Qualitative Velocity and ball Interception, 2002
2. Hausknecht, Matthew, and Peter Stone. ”Deep reinforcement learning in parameterized action space.” arXiv preprint arXiv:1511.04143 (2015)
3. Gabel, T.,Riedmiller, M.Trost, F.:A Case Study on Improving Defense Behavior inSoccer Simulation 2D:The NeuroHassle Approach.In:Springer-Verlag Berlin Heidelberg (2009) 61-72 An Introduction to Genetic Algorithms , Jenna Carr , May 16,
2014
4. ”RoboCup Soccer Simulation”, http://en.sourceforge.jp/projects/rctools, 2017
5. E. Asali, F. Negahbani, S. Tafazzol, M. S. Maghareh, S. Bahmeie, S. Barazandeh,
S. Mirian and M. Moshkelgosha, ”Namira Soccer 2D Simulation Team Description
Paper 2018,” in RoboCup 2018, Monreal, Canada, 2018.
6. E. Asali, A. Moravej, S. Akbarpoor, O. Asali, M. Katebzadeh, S. Tafazol, F. Negahbani, M. Valipour, S. Mirian, H. Mehran, A. Bitaraf Haghighi and Y. Khoshghadam
Borazjani, ”Persian Gulf Soccer 2D Simulation Team Description Paper 2017,” inRoboCup 2017, Nagoya, Japan, 2017.
7. E. Asali, M. Valipour, N. Zare, A. Afshar, M. Katebzadeh and G. Dastghaibyfard,”Using Machine Learning approaches to detect opponent formation,” Artificial Intelligence and Robotics (IRANOPEN), pp. 140-144, 2016.
8. N. Zare, M. Sadeghipour, A. Keshavarzi, M. Sarvmaili, A. Nikanjam, R. Aghayari,
A. Firouzkoohi, M. Abolnejad, S. Elahimanesh and A. Akhgari, ”Cyrus 2D Simulation Team Description Paper 2018,” in RoboCup 2018, Montreal, Canada, 2018.
9. N. Zare, A. Najimi, M. Sarvmaili, A. Akbarpour, M. NaghipourFar, B. Barahimi
and A. Nikanjam, ”CYRUS 2D Simulation Team Description Paper 2017,” in 21thannual RoboCup International Symposium, Nagoya, Japan, 2017.
10. R. Khayami, N. Zare, M. Karimi, P. Mahor, A. Afshar, M. S. Najafi, M. Asadi, F.
Tekrar, E. Asali and A. Keshavarzi, ”CYRUS 2D simulation team description paper
2014,” in RoboCup 2014 Symposium and Competitions: Team description papers,
Joao Pessoa, 2014.
11. R. Khayami, N. Zare, H. Zolanvar, M. Karimi, P. Mahoor, F. Tekrar, E. Asali andM. Fatehi, ”Cyrus Soccer 2D Sumilation Team Description Paper,” in 17th annual
RoboCup International Symposium, Eindhoven, Netherlands, 2013.
12. M. Hausknecht and P. Stone, ”Deep Reinforcement Learning in Parameterized
Action Space,” in International Conference on Learning Representations (ICLR),
San Juan, Puerto Rico, 2016.
13. Nader Zare , Mahtab Sarvmaili , Omid Mehrabian , Amin Nikanjam , Seyed Hossein Khasteh , Aref Sayareh, Omid Amini, Borna Barahimi, Arshia Majidi, Aria
Mostajeran , “CYRUS 2D Simulation Team Description Paper 2019”14. M. Abolfazli Esfahani, M. Ghafouri, M. Jamili, S. Askari, R. Etemadi, H. Mohebbi,M. Dara, M. Erafani and M. Malekabadi, ”Nexus2D Team Description Paper,” inRoboCup 2017, Nagoya, Japan, 2017.
15. Genetic Algorithm Optimized GravityBased RoboCup Soccer Team ,Tory Harter,July 28, 2015

616. Chen, S., Lv, G., Wang, X. (2016). Offensive Strategy in the 2D Soccer Simulation League Using Multi-Group Ant Colony Optimization. International Journal ofAdvanced Robotic Systems, 13.
17. Akiyama H, Nakashima T. Helios base: An open source package for the robocup
soccer 2d simulation. InRobot Soccer World Cup 2013 Jun 24 (pp. 528-535).
Springer, Berlin, Heidelberg.
18. Fukushima T, Nakashima T, Akiyama H. Similarity Analysis of Action Trajectories
Based on Kick Distributions. InRobot World Cup 2019 Jul 2 (pp. 58-70). Springer,Cham.
19. Akiyama, H., Nakashima, T., Fukushima, T., Suzuki, Y., Ohori, A.: Helios2019:
Team description paper. In: RoboCup 2019 Symposium and Competitions, Sydney,
Australia (2019)20. M. Noohpisheh, M. Shekarriz, S. Barzegar, D. Borzoo, A. Kariminia, ”Razi Soccer
2D Simulation Team Description Paper 2018,” in RoboCup 2018, Montreal, Canada,
2018.
21. Noohpisheh M, Shekarriz M, Bordbar A, Liaghat M, Salimi A, Borzoo D, Zarei
A. M. Noohpisheh, M. Shekarriz, A. Bordbar, D. Borzoo, M. Liaghat, A. Salimi,A. Zarei, ”Razi Soccer 2D Simulation Team Description Paper 2019,” in RoboCup2019, Sydney, Australia, 2019.