OXSY 2022 Team Description
Sebastian Marian, Dorin Luca, Bogdan Sarac, Ovidiu Cotarlea
OXygen-SYstems laboratory, Str. Constantin Noica,Bl.5, Sc.C, Ap.36, C.P. 550169, Sibiu, ROMANIA
Email: (sebastian.marian@oxsy.ro) Web: (http://oxsy.ro)

Abstract. Oxsy team was founded in July 2002 for a graduation project of one
student, Sebastian Marian, in the field of Multi-Agent Systems [1], at the Department of Computer Science of Lucian Blaga University (Sibiu - Romania).
After graduation he continued the work on this project and so was born Oxsy
team. As we started from scratch, our ideas, concepts and beliefs, was implemented year by year and today, we are happy to see that we are on the rightway, as our team was growing in these years, more than we expected from thebeginning. If we will qualify to the competition, this year we’ll reach at the 18 thconsecutive participation, in RoboCup [4] Soccer Simulation League.

Keywords: RoboCup, 2D Soccer Simulation, Coach, Neural Network, Offside,
RSA accumulator.

1Introduction

In July 2003 at RoboCup [4] competition, which was held in Padua - Italy, we won
the first round and for us it was a good surprise for first year of participation. Then, inthe next year, we participated in Lisbon - Portugal for the second time, and again weobtained a good result (the 11th place). In 2005 in Osaka – Japan, we participated forthe third time and finally we entered in the first 8 teams of soccer simulation league,
as we won (the 8th place). In 2006 the competition was held in Bremen – Germany
and we won (the 7th place). In 2007 we went to Atlanta – Georgia (U.S.A), where weobtained (the 5th place), the same result which we achieved in 2008 in Suzhou –China. Finally, in 2009 in Graz, we entered in the first 3 teams in the soccer simulation league, as we won (the 3rd place), the same result which we achieved in 2010 inSingapore. In 2011 we came back from Istanbul - Turkey with (4th place). In 2012 wewere in Mexico City, where we had a bad experience as we made some majorchanges in our defensive system, and also many others overall our team strategy,changes which was not very well balanced at that time, with all others characteristics
of our team, as we were not qualified for finals, from the second round groups. In
2013 we came back in top, as we won (the 6 th place), in Eindhoven – Netherlands. In
2014 the competition was held in Joao Pessoa – Brazil, and we entered on the stage
for the third time in our participation history, as we won again (the 3 rd place). In 2015
we won the 4th place as we played the semifinals in Hefei – China. In 2016 the competition was held in Leipzig – Germany, we missed the semifinals and we came backwith (5th place). In 2017 we came from Nagoya with (3rd place) for the fourth time inour participation history. In 2018 the competition was held in Montreal – Canada andwe won the 4th place as we played in the semifinals. In 2019, unfortunately, our teammissed the competition which was held in Sydney – Australia. In 2020 the competition has been postponed for 2021 and last year it was held online due to pandemic situation. Our team won the 6th place. This year the competition will be held in Bangkok- Thailand. As we already have a very good experience in 2D Soccer Simulation
league, we hope that our new ideas and improvements will be reflected in the competition where we will also test other tactical elements developed.
This year we further developed and extended our architecture, based on a very powerful concept that we have used in our early times in 2D Soccer Simulation Competition, by using the concept of “Holon”. Multi-agent (intelligent) systems (MAS) aresoftware systems composed of several autonomous software agents running in a distributed environment. Besides the individual goal of each agent, global objectives areestablished committing all or some agent groups to their completion. This concept of“Holon” was proposed by the Hungarian philosopher A Koestler (1967,1978) and explains the importance of the hierarchy of a system. Each organ is an element of the organic system while the organ is itself a system composed of multiple tissues. This relationship appears at every level of the system. This means that a system element islocated at a hierarchy node and has both characteristics as a whole and as a part.
Koestler named the node of hierarchy "Holon". (Hino R. 1999)[6] based on the combination of the Greek word "holos" that means "whole", and the suffix "on" meaning
particle or part. Accordingly a "holarchy" is a hierarchy of self- regulating control
building blocks holon, which function:(a) As autonomous wholes in supra-ordination to their parts;(b) As dependent parts in subordination to controls on higher levels;(c) In coordination with their local environment;2

Why holarchies are suitable for Soccer Simulation?

By definition, the holon is an excellent concept for modeling a soccer team because ofits main characteristic: it can act as a part (modeling a player) or as a whole (modelingthe team). One of the most important characteristics of the holarchies is the capacity
to modify themselves, i.e. to create temporary hierarchies (Giebels, M. et. al. 1999)[7]. Like modern industry, the soccer is very dynamic, i.e. not only that each teamcomes with its own style and game strategy, but also each game phase has a dose ofnovelty. In this way, it isn’t possible to create only one command scheme that will
work correctly for each adverse team and each game phase. The holarchies aren’t
rigid, they change themselves according to necessities (strategic and tactic) and thestructure of the decisional schema will dynamically change (by modifying momentarypriorities). Another advantage of the use of holons is providing a balance between thetwo possible approaches of the leading process: the hierarchical control (fixed, static,pre-established) and the heterarchial (autonomous, decentralized, flexible but not very
efficient), it can be seen in (Fig. 1).

Fig. 1 Holarchical model offers balance between hierarchical and heterarchical approches
An holon is the fundamental part of any holarchy, and it can be referred to as an agent
who has outspoken cooperatively and autonomy characteristics. The holon has two
communication channels with the outside world, as it can be seen in (Fig. 2). One ofthis channels is used for communication with the others holons and the other one foracquiring information from the environment. The reactive layer – obviously indispensable – underscores one the main common features of holons and agents. The
most important part of the holon is the planning process set up in the deliberativelayer. The planning process uses data acquired on the two communication channels –a) with the environment (receiving stimuli and sending response to them), and b) withthe other holons (taking advice from the higher level, negotiation with adjacentholons monitoring subjacent holons)Fig. 2 Generic holon structure and logical layers

The negotiation part (implemented in the communication layer) is activated in two situations. The first situation is when it receives some advice that can’t be accepted (executed); in this situation it can inform the caller what can be done (i.e. it has othertasks more important for it or it can’t implement the advice because of lack of capaci ties). The second situation is when it sends an advice to another holon and receives arejection. Based on this refuse it can begin the negotiation process, which consists inredefining the advice (less requirements) or in task decomposition (it will try to assign
more holons to solve the initial task).

3Architecture

Our architecture has been extended based on the PROSA model, developed atPMA/KULeuven as a reference model for Holonic Manufacturing Systems (VanBrussel at al. 1998, Wyns 1999)[5]. The acronym PROSA came from Product-Resource-Order-Staff Architecture, the holons type used. The resource holon contains aphysical part namely a prodution resource of the manufacturing system, and an information processing part that controls the resource. The product holon holds the processand product knowledge to assure the correct making of the product with sufficientquality. The order holon represents a task in the manufacturing system. It is responsible for performing the assigned work correctly and on time. The staff holon is implemented in the idea to assist the rest of three holons in performing their work. The concept and it’s assignment can be seen in (Fig. 3).

Fig. 3 The PROSA model architecture
At the (level 0) we have the “RoboCup environment” containing the soccer server
rules (Itsuki Noda et al. 1998)[3].
At the (level 1) we have holons who implements opponent strategy analyzing, opponent modeling without noise, make some player changes (heterogeneous). Communication with others holons from others players can be made through (Say) command.
At the (level 2) we have holons who implements some game schemes, strategy basedon opponent modeling and also support for asking spares players (heterogeneous).
Communication with others holons from others players can be made through (Say)command.
At the (level 3) we have holons who implements the skill (i.e. pass, dribble, shoot),
the goal oriented behavior and the strategy.
At the (level 4) we have holons who implements effectors (like body components)that can do actions like (kick, dash, turn, etc.).
All these levels should be understood like a holarchy not as a hierarchcal organization. Please see the (Fig. 4).

Fig. 4 Holarchical levels with the PROSA assignment

4Implementation

This year we extended our architecture based on the PROSA model where each holon
has a well-defined functionality. We have added the functionality for the product andorder holons at the team level. The holons that we are using have similar functionality
with the original (Table 1).

Table 1

5Future work

In this paper we describe our new extended architecture that tries to combine in a synergistic manner the approaches applied in two, up to now insufficiently related domains: MAS and HMS. The preliminary results suggest that we are in a very promising starting point for future work. This validates the approach of adding functionality(first of all, flexibility) stepwise, i.e. improving the old architecture. The results showthat at this stage the most useful is the product holon. For the game strategy the orderholon is still insufficiently adapted. The staff holon will be useful when the evolutionof the game rules will give an even more substantial role to the coach. Another problem for the future is improving negotiation between holons.

References
1. Peter Stone Layered Learning in Multi-agent System [D]. Pittsburgh: school of computerscience, Carnegie Mellon University, 1998.
2. M. Riedmiller and Artur Merke, “ Using machine learning techniques in complex multiagent domains,” In I. Stamatescu, W. Menzel, M. Richter and U. Ratsch, editors, Perspectiveson Adaptivity and Learning, LNCS, Springer, 2002.
3. Itsuki Noda et al, Soccer Server Manual, RoboCupFederation. http://www.robocup.org.
4. Y. Jinyi, C. Jiang, and S. Zengqi. An application in RoboCup combining Q-learning with
Adversarial Planning. In the 4th World Congress on Intelligent Control and Automation(WCICA’02), Shanghai, China, June 2002.

5. Wyms, J. and Van Brussel, H. and Bogaerts, L. (1999) Design Pattern for Integrating centralised scheduling in distributed holonic manufacturing control systems. Proceedings of thesecond International Workshop on Intelligent Manufacturing Systems, Leuven, Belgium,
pp.75-82
6. Hino, R. and Moriwaki, T.(1999) Decentralized Scheduling in Holonic Manufacturing System. Proceedings of the second International Workshop on Intelligent Manufacturing Systems, Leuven, Belgium, pp.41-47
7. Giebels, M. and Kals, H. and Zijm, H. (1999) Building Holarchies for Concurrent Manufacturing Planning and Control. Proceedings of the second International Workshop on Intelligent Manufacturing Systems, Leuven, Belgium, pp.49-56