<figure><img src="image_1.png" /></figure>

HAL
open science

Artificial Swarm Intelligence and Cooperative Robotic
Systems

Iaroslav Omelianenko

<figure><img src="image_2.png" /></figure>

To cite this version:

Iaroslav Omelianenko. Artificial Swarm Intelligence and Cooperative Robotic Systems. 2019. hal-01997091

HAL Id: hal-01997091

https://hal.science/hal-01997091

Preprint submitted on 28 Jan 2019

**HAL** is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_3.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Artificial Swarm Intelligence and Cooperative Robotic Systems*

Iaroslav Omelianenko

NewGround Advanced Research
Kiev, Ukraine

Email: yaric@newground.com.ua

**Abstract**—In this paper, we look at how Artificial Swarm Intelligence can evolve using evolutionary algorithms that try to minimize the sensory surprise of the system. We will show how to apply the free-energy principle, borrowed from statistical physics, to quantitatively describe the optimization method (sensory surprise minimization), which can be used to support lifelong learning. We provide our ideas about how to combine this optimization method with evolutionary algorithms in order to boost the development of specialized Artificial Neural Networks, which define the proprioceptive configuration of particular robotic units that are part of a swarm. We consider how optimization of the free-energy can promote the homeostasis of the swarm system, i.e. ensures that the system remains within its sensory boundaries throughout its active lifetime. We will show how complex distributed cognitive systems can be built in the form of hierarchical modular system, which consists of specialized micro-intelligent agents connected through information channels. We will also consider the co-evolution of various robotic swarm units, which can result in development of proprioception and a comprehensive awareness of the properties of the environment. And finally, we will give a brief outline of how this system can be implemented in practice and of our progress in this area.

**Keywords**—artificial intelligence, neuro-evolution, swarm intelligence, robotic swarm, cooperative robotics, free-energy principle, active inference, evolutionary computation, novelty search

## I. INTRODUCTION

With current advances in machine learning and robotics it's safe to assume that in near future we will be witnessing deep penetration of cooperative robotic systems into everyday life of millions of people as well as into many industrial processes. Close cooperation with fragile human beings suggests that robotic systems should evolve to achieve deep understanding of the environment in order to safely use their powerful actuators. Such level of environmental awareness can be achieved by considering robotic system as not a single unit, but rather as a distributed swarm of robotic units connected by peer-to-peer mesh network with powerful central command and control (CNC) unit. Such structure allows to join effective decision making (by powerful CNC unit) with high awareness of the environmental dynamics achieved through spatially distributed sensorium.

In this work we consider how to effectively build such distributed intelligent system which is able to adapt to the specific environmental conditions due to inherent capabilities for a lifelong learning. We will show that there are plausible parallels with biological systems can be made and apply biologically inspired paradigms to the design of such system.

This paper is organized as follows: In Section II, we give an overview of the structure of the swarm system at the physical level. It is followed by Section III, describing the hierarchy of

the swarm system at the logical level, as well as describing its main components, which make it's cooperative intelligence possible. In Section IV, we discuss implementation of swarm learning model and optimization methods to be used for initial training of artificial swarm intelligence. Section V provide description of simple swarm architecture and how it can be implemented. Finally, in Sections VI and VII we discuss our findings and describe our current and future works in the area.

## II. SWARM STRUCTURE - PHYSICAL LEVEL

In this paper, we consider swarm as a tightly integrated robotic environment in which many semi-autonomous types of robotic organisms cooperatively interact to achieve specific goals. The robotic units in the environment organized in well-defined hierarchy, where particular types of units provide specific sensory inputs for the swarm, while others are capable of affecting the environment. In addition, the swarm will include powerful units (*command centers*) to perform energy- and computationally-intensive tasks for strategic decision-making and general knowledge accumulation.

Our idea is to have specific types of units to evolve in the process of swarm maturation from some kind of basic units. Each basic unit has unique combination of sensors and actuators, allowing it to perceive and affect the environment in a certain way. Later in the evolutionary process, some sensors and actuators of basic units will be inactivated to create an unique configuration for the new derived unit type. The evolutionary process will be bounded by environmental factors and will be optimized to ensure cooperative behavior of various types of robotic units to achieve the desired results. Such cooperative integration allows a sensorium of one type of units to provide guidance for another type, e.g. aerial units will provide reconnaissance data about environment, that is not directly perceived by ground-based units.

The continuous communication with the *command center* is an important part of coordinating collective behavior between units of the swarm. However, in case of an emergency, it is assumed that all units can operate autonomously to complete the current task, even when the communication link is down.

The autonomous behavior of each unit is assisted by specialized tactical decision making software, which consist of modular/hierarchical neural networks trained to operate within sensorimotor framework of that specific robotic unit.

## III. SWARM HIERARCHY - LOGICAL LEVEL

We regard the robotic swarm as a single sensorimotor logical unit, which has an unified spatial awareness of the environment, distributed over all its constituent parts. But

<footnote>*This work was supported by NewGround LLC</footnote>
the internal structure of the swarm's consciousness is not a monolith, but rather consist of a complex hierarchy of micro-intelligent agents that interact with each other to perform data collection, data processing, and decision-making, according to the current environmental conditions and tasks to be completed.

The dynamics of the swarm will be controlled with the help of *attractors* - the active components of the system that will lead the cooperative passive components. *Passive components* are implemented in such a way as to assist any external action initiated by a friendly active unit or person. *Active components* are dedicated swarm units that have improved sensorimotor capabilities and higher level of autonomy due to the more powerful cognitive models implemented in them.

Thus, while each component of the swarm has a certain level of autonomy, the full potential of the system is revealed in the synergistic interaction of all its components. As a result, the *Artificial Swarm Intelligence* (ASI) [14] system is emerging, which has a cognitive potential that exceeds the sum of all its components [15].

An appropriate swarm hierarchy will emerge in the process of evolution of the system from *seeds of basic units*. Each seed of a specific robotic unit describes its basic sensorimotor capabilities/configuration in terms of the topology of the *Artificial Neural Network* (ANN) controlling it. The neuro-evolution [11], [12] process will be applied to the mentioned seed configurations to create specialized ANN modules [13], depending on the sensorimotor dynamics of the system and the observed environmental characteristics.

The evolutionary potential of each ANN module will be bounded by the hardware on which robotic unit operates. Thus, as in biological systems, the most potent units will be evolved where the most of the computing power and energy is available. Such inherent limitations will contribute to a targeted evolution of the *hierarchical modular system*, where each part has certain processing capabilities in accordance with its place in the swarm hierarchy.

## A. The Command and Control Center

Strategic planning of the general behavior of the swarm is carried out by the *Command and Control* (CNC) center, where sensory data from all swarm units are collected and processed. The decision-making process of the CNC center use collected data to model the next state of the environment in which swarm operates. After that, the modelling error is corrected using actual sensory inputs received from the swarm units (*posterior probabilities distribution*) after performing a specific action based on the model predictions (*prior probabilities distribution*). Thus, it becomes possible to create a highly adaptive system that is able to quickly learn the properties of novel environments and quickly adapt the behavior of the swarm to effectively achieve the goals. In addition, with each new environment learned, the *knowledge base* of the CNC system will be expanded, which will allow it to easy master novel environments using *transfer learning* methods.

Our idea is to use the *Minimal Criterion Novelty Search* [4] optimization method for a comprehensive exploration of the environment. With this method of learning, the exploring

agent will be rewarded depending on the level of novelty of solution that was found. At the same time, the *minimal fitness criterion* is introduced, which allows to optimize the training time and avoid agent sticking in the some noisy parts of the environment, which have a high potential for the novelty but low objective fitness.

The ultimate goal of the swarm system's training is to minimize sensory surprise (*free-energy*) [9] of the CNC module while roaming in the environment. The minimal value of surprise implies that the CNC system is fully aware of the properties of the surrounding environment in the context of the actual sensory inputs and the probability distributions of the causes of the inputs. Thus, the CNC system is able to maintain a reliable model of the environment and make the right inference about the outcomes of actions that will be performed. And with this awareness, it becomes possible to build an optimal strategy for using the observed properties of the environment to achieve specific goals and accomplish the objectives.

Another important aspect of the CNC module is that it maintains a library of already explored environments and can use this prior knowledge as a stepping stone for improving performance in a different environment.

The CNC module will control the overall dynamics of the swarm through the *attractors* - the dedicated active units. But at the same time, to perform micro-tasks that require high accuracy, it will be able to directly control each unit of the swarm.

## B. Micro-Intelligent Agents and Information Channels

Recently researchers have shown [21] that in simple models of neural networks, the amount of *effective information* [26] increases as you coarse-grained the model of neuron interactions in the neural network, that is, consider groups of them as a whole. The possible states of these interconnected units form a *causal structure* in which transitions between states can be mathematically modeled using so-called *Markov chains* [22]. At a certain macroscopic scale, *effective information* reaches a peak. This is the scale at which states of the system have the *most causal power*, predicting future states in the most reliable and effective manner. By increasing the scale further, you begin to lose important details about the causal structure of the system. The neural groups with most causal power at the optimal macroscopic scale we refer to as *Micro-Intelligent Agents* (MIA).

MIA agents are the foundation of the *cognitive hierarchy* of the swarm system. They are able to process tiny amounts of input data (bit sets, words) and aggregate them into higher-order information units that will be passed to the next level of processing. Physically, MIA agents are represented as highly specialized software modules evolved to process certain kinds of input data received from specific set of sensors or from *Information Channels* (ICH) [21].

Information Channels, in turn, facilitate the exchange of information between different parts of the swarm and encode the casual structure of the sensory inputs. Information Channels evolve along with *Micro-Intelligent Agents* in the process of maturation of Artificial Swarm Intelligence and become
an important part of the swarm system configuration, which determines its ability to ingress environmental information from its sensorium.

<figure><img src="image_4.png" /><figcaption>Fig. 1: The Information Channel connecting two MIA agents, one of which encodes the signal on the input side and the other decodes on the output side, respectively. Thus, ICH with associated MIA agents becomes an informative representation (coding) of a particular sensory input for a particular type and state of the environment.</figcaption></figure>

The Information Channel consist of two finite sets, X and Y, and is a collection of transition probabilities p(y|x) for each x ∈ X, such that for every x and y, p(y|x) ≥ 0 and for every x, $\sum_y p(y|x) = 1$ (set, known as the channel matrix). X and Y are the input and output of the channel, respectively [23]. The channel is governed by a channel matrix, which is a fixed entity represented by a specific Artificial Neural Network, emerging during the corresponding neuro-evolutionary process. Both channels and causal structures can be represented as Transition Probability Matrices (TPM) [22], and in a situation where the channel matrix contains the same transition probabilities for a certain set of state transitions, the TPMs will be identical. The causal structure is a matrix that transforms previous states into future ones.

The TPM associates each state $s_i$ in S with the probability distribution of past states ($S_P|S = s_i$), which could led to it, and the probability distribution of future states to which it could lead to ($S_F|S = s_i$). In these terms, the $E_D$ (Effect Distribution, which is an ASI model's internal states describing sensory inputs) can be formally expressed as expectation of $(S_F|do(S = s_i)\forall i \in 1...n)$ given some $I_D$ (Intervention Distribution, which is the sensor casuals, respectively). Where do(x) is an operator that formalizes interventions [29], which set a system (or variables in a causal model) into a specific state, which facilitates the identification of the causal effects of that state on the system, disrupting its relationship with the observed history.

To send a message through a channel matrix with these properties, it is necessary to define some encoding/decoding function implemented through the specialized MIA agents in our architecture (see Figure 1). A message can be some binary string like {110111010001} generated via the application of some $I_D$ (Intervention Distribution). The encoding function $\phi: \{message\} \rightarrow \{encoder\}$ is a rule that associates some

channel input with some output along with some decoding function $\psi$. The encoding/decoding functions together create a code-book, which is a concise, informative description of the environmental properties sampled by the ASI agent from its sensory inputs.

According to the Shannon's theory of communication [25], communication channels have a certain capacity, that is, the ability of the channel to convert inputs into outputs in the most informative and reliable way. Also he found that the rate of information transmission over the channel correlates with the changes in the input probability distribution $p(X)$. Thus, the channel capacity (C) is determined by a set of inputs that maximizes the mutual information $I(X;Y)$. By maximizing channel capacity, it is possible to effectively increase its transmission rate:

$$C = \max_{p(X)} I(X;Y) \quad (1)$$

where $I(X;Y)$ is the mutual information for representing the transmission rate of information over the channel:

$$I(X;Y) = H(X) - H(X|Y) \quad (2)$$

where $H(X)$ represents the total possible entropy of the source and the conditional entropy $H(X|Y)$ indicates how much information is left about X, once Y is taken into account. Therefore, $H(X|Y)$ has a clear causal interpretation as the amount of information lost during the processing and transmission of sensory inputs.

Taking into account the discoveries of Shannon, it can be seen that the use of MIA agents as a encoder/decoder for input/output of the data bits from the information channel, effectively increases its capacity and transmission rate. Which is crucial given the distributed nature of the swarm intelligence.

To draw parallels with biology, swarm intelligence can be viewed as full body consciousness, in which MIA agents can be viewed as individual cells or cell conglomerates (sensing and pre-processing organs, etc) [1], [2] and information channels as a signaling pathways in the form of peripheral neural system or information exchange routines through chemical markers.

## IV. SWARM LEARNING MODEL

It is assumed that swarm intelligence is not fully determined by a constant learned set of skills - a priori pre-trained knowledge, but rather is a combination of prior knowledge with continuous lifelong learning abilities. The lifelong world exploration is based on a certain kind of visual consciousness, mediated by inherent knowledge of the swarm system about its sensorimotor contingencies [5] and proprioception. Here, the visual consciousness is not only about the sensation of visible electromagnetic specter, but rather full gamma of sensations collected from the environment and processed as a visualization model, where internal visualization of the environment is compared with actual data received from sensors. In this aspect, it is important that swarm system evolve to maturity with all its constituents incorporated in order to learn its unique sensorimotor configuration, which combines the perception of
external world with the available actuating mechanisms to act on it. This unique configuration will be encoded by control ANN emerged during evolution. Thus, the experience gained during evolution of the swarm system becomes its unique bubble of experience [6], which serves as a starting point for self-awareness development.

The learning model will be optimized to minimize surprise (negative log probability) of some sensory data, given a model of how this data were generated, which is called *active inference* [7]. The bounds of sensory surprise in a quantitative terms can be expressed as a *free-energy* [8]. Free-energy measures the difference between the probability distribution of environmental factors affecting our system and an arbitrary distribution encoded by the swarm system's configuration, i.e. the topology of the control ANN, its connection strengths, and modules included. The swarm intelligence can minimize free-energy (sensory surprise) by changing its configuration (topology of the control artificial neural network) to affect the way it samples the environment (e.g. when we move through in a dark room, we build a mental model of what to expect in the next step and then compare what we feel with these prior expectations). Thus, the adaptive exchange with the environment will be established allowing swarm intelligence to construct prior expectations in a dynamic and context-sensitive fashion.

According to the recently proposed *free-energy principle framework*, the following key points characterize the model of *adaptive learning* (biological):

* Adaptive agents must occupy a *limited repertoire of states* and, therefore, minimize the long-term average value of surprise associated with sensory exchanges with the world. Minimizing surprise allows them to resist the natural tendency to disorder.

* Surprise is based on predictions about sensations that depend on the internal generative model of the world. Although surprise cannot be measured directly, the free-energy bound on surprise can be, suggesting that agents minimize free-energy by changing their predictions (perception) or by changing the predicted sensory inputs (action).

* Perception optimizes predictions by minimizing free-energy with respect to the activity of neural units (perceptual inference), efficacy (learning and memory) and gain (attention and salience). This gives Bayes-optimal (probabilistic) representations about what caused the sensations (providing a link to the Bayesian brain hypothesis).

* Learning under the free-energy principle can be formulated in terms of optimizing the connection strengths in hierarchical models of the sensorium. This is based on associative plasticity for encoding causal regularities and address the same synaptic mechanisms as those underlying the formation of a cell assembly.

* Action under the free-energy principle is reduced to the suppression of sensory pre-

diction errors that depend on the predicted (expected or desired) movement trajectories. This provides a simple account of motor control, in which action is enslaved by perceptual (proprioceptive) predictions.

* Perceptual predictions are based on prior expectations regarding the trajectory or movement through the agent's state space. These priors can be acquired (as empirical priors during hierarchical inference) or they can be innate (epigenetic) and, therefore, subject to selective pressure.

* The predicted movements or state transitions realized by the action correspond to policies in the optimal control theory and reinforcement learning. In this context, value is inversely proportional to surprise (and implicitly free energy), and the rewards correspond to innate priors that constrain policies.

( [9])

In this work, our idea is to integrate this biologically inspired model of adaptive learning into a field of artificial intelligence in order to achieve our goal - to develop an intelligent distributed swarm system that can maintain its sensory states within the physical bounds of its components in face of constant environmental flux. In short terms, this means avoiding sensory surprise, which relates not only to the current state of the swarm system (which cannot be changed), but also to the transition from one state to another (which can be changed). The guiding principle for the mentioned transitions between states, which is compatible with the swarm survival (e.g. the flight of a swarm of drones within a small margin of error), can be considered as a global *random attractor* [10]. In the learning process we will be optimizing the movements of this attractor by minimizing free-energy bound on the sensory surprise of the swarm.

<figure><img src="image_5.png" /><figcaption>Fig. 2: Information exchange between the environment and the ASI agent. The swarm learning model will be represented as a set of internal states that control actions and make prediction error corrections in accordance with sensations caused by external environmental states.</figcaption></figure>

The dependencies between the inputs/outputs of a swarm,
along with its internal states and the external states of the environment, allows us to formulate a swarm learning model (see Figure 2).

The internal states of the Artificial Swarm Intelligence system (3) and its actions (4) work to minimize the free-energy $F(\bar{s}, \mu)$, which is a function of sensory inputs $s(t)$ (5) and a probabilistic representation $q(\Theta|\mu)$ of its causes:

$$ \mu = \operatorname*{argmin}_{\bar{s}} F(\bar{s}, \mu) \qquad (3) $$

$$ \alpha = \operatorname*{argmin}_{\bar{s}} F(\bar{s}, \mu) \qquad (4) $$

$$ \bar{s} = g(\bar{x}, \Theta) + \bar{z} \qquad (5) $$

The environment (6) is described by equation of motion that specify the trajectory of its hidden states:

$$ \dot{\bar{x}} = f(\bar{x}, \alpha, \Theta) + \bar{w} \qquad (6) $$

Where causes $\Theta \supset \{\bar{x}, \Theta, \gamma\}$ of sensory inputs include hidden states $\bar{x}(t)$, parameters $\Theta$ and precision $\gamma$, controlling the amplitude of the random fluctuations $\bar{z}(t)$ and $\bar{w}(t)$.

Free-energy depends on two probability densities: the recognition density $q(\Theta|\mu)$ and the one that generates sensory samples and their causes, $p(\bar{s}, \Theta|m)$. The latter represents a probabilistic generative model (denoted by $m$), the form of which is determined by the controlling ANN of the ASI agent.

The sensory surprise of an ASI agent is a negative log-likelihood [17] associated with its sensory states $\bar{s}(t)$ that have been caused by some unknown quantities $\Theta \supset \{\bar{x}, \Theta\}$:

$$ -\ln p(\bar{s}(t)|m) = -\ln \int p(\bar{s}(t), \Theta) d\Theta \qquad (7) $$

It can be shown [9] that the free-energy can be represented as a surprise $-\ln p(\bar{s}(t)|m)$ plus a *Kullback-Leibler* [16] divergence between the recognition $q(\Theta|\mu)$ and conditional $p(\Theta|\bar{s})$ densities:

$$ F = D_{KL}(q(\Theta|\mu) \parallel p(\Theta|\bar{s})) - \ln p(\bar{s}|m) \qquad (8) $$

In this case, the free-energy can be minimized by training ASI agent model to infer causes of the sensory samples in *Bayes-optimal* fashion.

With another form of free-energy presentation (9), it can be shown that it can be minimized by affecting the environment (action) in accordance with the current internal state of the ASI agent, followed by sampling of sensory data. After that, the ASI agent will reconfigure its sensor circuits to sample inputs, which are predicted by its recognition density in order to minimize prediction error.

$$ F = D_{KL}(q(\Theta|\mu) \parallel p(\Theta)) - \langle \ln p(\bar{s}(\alpha)|\Theta, m) \rangle_q \qquad (9) $$

## V. EXAMPLE OF THE SWARM SYSTEM IMPLEMENTATION

In this part, we will look at how to apply the proposed design of the swarm robotic system to build very basic exploratory system that is able to develop awareness of the spatial environment and accomplish the tasks assigned.

Such a system should at least have the following components (see Figure 3):

* The command-and-control (CNC) module with high power capacity and advanced computational capabilities, which has reliable communication channels with other members of the swarm, as well as with human operators.

* The light-weight mobile (LWM) units, which can be used to quickly explore environment and deploy spatial coordinate grid for independent local positioning system.

* The heavy-weight mobile (HWM) units, that can physically reach various areas of the environment and perform assigned tasks. They will use the spatial coordinate grid provided by the LWM units to confirm their position and direction.

<figure><img src="image_6.png" /><figcaption>Fig. 3: The structure of the Swarm Exploratory System</figcaption></figure>

The role of light-weight mobile units is to quickly explore an unknown environment and occupy certain positions to form spatial grid coordinate system that will be used to facilitate spatial awareness of the swarm. The LWM units must be highly mobile so that they can be deployed as quickly as possible. This means that, depending on the properties of the environment, they can be physically presented as drones, lightweight gyro-boards, etc.

The heavy-weight mobile platforms will complement the LWM units with additional sensor arrays capable of collecting the most comprehensive environmental data. Thus, with the deployment of HWM platforms, the swarm's awareness of environmental dynamics must be sufficient enough to fulfill the task set by the human operators, even in an autonomous mode.

The positions of the HWM platforms will be visually correlated with respect to the spatial grid formed by the LWM units (see Figure 4), which makes the swarm independent of any external positioning system (GPS, GLONAS, etc). Also, the special configuration of the infrared LED arrays placed on the body of the HWM platforms at certain places, will allow to determine their direction relatively to the spatial grid without any reference to the geomagnetic data. Furthermore, it is possible to use other variants of local positioning based, for example, on near-field radio signals. The advantage of using a visual signal is that it is much more difficult to suppress than a radio signal. To further enhance the reliability of the local coordinate system, visual positioning can be supplemented with acoustic or seismic signals, depending on the properties of the environment in which it is deployed. This is especially useful in underground or underwater environments where light and radio signals propagation is limited.
With a local grid coordinate system, the swarm can operate in any environment where there are no other means to obtain reliable positioning from external systems. Thus, swarm can be deployed in many harsh environments: underground, underwater, in the interplanetary space, or in places where signals from an external positioning systems are suppressed.

The CNC module could be implemented as a stationary structure with a powerful battery array and capable processing units. To increase the resilience and survivability of the swarm, it is possible to use multiple CNC systems deployed on HWM platforms, which use a master-slave architecture. The current master CNC module will manage the swarm, and the slave CNC modules will maintain their state by listening to regular updates from the master. In the event of master's failure, a new master will be elected according to a specific election protocol [18], [19].

## VI. CONCLUSIONS

In this paper, we presented our ideas on the development of robotic systems controlled by an *Artificial Swarm Intelligence*. We looked at how swarm can be trained to obtain full sensorimotor awareness about all of its constituent parts (proprioception) endowing it with sense of relative parts positions, movement, and acceleration. We also considered how to support lifelong learning, which is one of the most important features of the ASI system, that allows it to quickly adapt to the conditions of novel environments using transfer-learning methods.

We looked at the *Artificial Swarm Intelligence* as a macro-scale hierarchy of the *Micro-Intelligent Agents* and *Information Channels*, which, according to the theory of casual emergence [24], provides more informative casual model of the environment than micro-scale systems that take into account every bit of input data. I.e. *map* (macro-scale) is better than *territory* (micro-scale) in terms of information models.

We suggested using *evolutionary computational* [20] methods (e.g. *neuro-evolution*) for evolving of specific control neural networks from basic (seed) ANN configurations. We assume that during the evolutionary process, the specific type of control ANN configuration will be evolved, that are able to take full advantage of the sensorimotor capabilities of a particular robotic unit. The overall configuration of the swarm is also the subject of an evolutionary development in which various types of robotic units will co-evolve, complementing each other's characteristics.

We have described a learning model based on the notion of minimizing the sensory surprise of a swarm system when it interacts with the environment. And applied the *free-energy principle* [8] to determine the optimization function, which can be used as a guiding principle during the lifelong training of the ASI model.

Finally, we provided an overview of the implementation of a simple swarm system with the definition of its major constituent parts and their interactions.

Summing up our research, the following key concepts should be considered when designing the *Artificial Swarm Intelligence* system:

<figure><img src="image_7.png" /><figcaption>Fig. 4: The grid structure of the coordinate system is formed by light-weight mobile units. The hexagonal spatial structure allows to generate grid code that determines the coordinates in the local coordinate system relative to the master CNC module.</figcaption></figure>

* The sensorimotor configuration of the swarm system must be constant during the period of primary training. It's important that during maturation the Artificial Swarm Intelligence becomes fully aware of its physical boundaries and learns how to act within them (proprioception). An embodiment allows ASI to build complex models of internal sensory experiences linked with the causes of the specific sensory inputs, which allows the use of *active inference* [7] optimization techniques in order to support *lifelong learning*.

* The ASI system should be designed as a modular hierarchical architecture, in which data collection, data processing, and decision-making procedures are distributed across multiple hierarchical levels and span across multiple physical units of the swarm. The exact architecture, structure and format of communication (information) channels should be evolved in the process of ASI maturation as part of the early *epistemic* experiments with the environment: checking physical boundaries, predicting outcomes of physical actions, etc.

* The dynamics of the swarm should be controlled by the interaction of active and passive units. Active units serve as a *random attractors* [10], which shape the behavior of passive units. By doing this, they reduce the computational load on the command-and-control unit when building the optimal strategy for execution of the current task.

* It is important to have energy- and computationally-rich Command-and-Control units to maintain proprioception of the swarm system and control execution of assigned tasks on the high level.

* Another important aspect of Artificial Swarm Intelligence is to maintain the library of already explored environments, which can be used for transfer-learning when building a model of novel environments.

* The swarm must include various types of robotic units
with different types of sensorimotor configurations and specializations. This provides boost during an ASI evolution and allows the selection of appropriate configurations that are beneficial for a particular environment. Such flexibility allows to use a trained swarm system to explore any type of environment for which it has an appropriate sensorium and actuators.

* The ASI system should evolve local spatial awareness based on its component parts. This implies that it has light-weight mobile units that can be quickly deployed into strategic places to form a local spatial coordinate grid with hexagonal cells. After that, it will have a long-range coordinate system for determining the relative positions of the swarm units without any external cues (see Figure 4). It is important that mentioned hexagons is not uniform but differ in sizes, directions, and shifts relative to one another. Due to such complex geometry they will provide unique coding patterns (grid codes) for each possible spatial coordinates [28] relative to the master CNC module.

## VII. FUTURE WORK

Right now we are working on the implementation of the described swarm system architecture. We already developed many components of the software framework, which can be used to implement control ANN modules.

The software is written in *GO programming language* [27], which makes it highly interoperable that can be executed on wide range of hardware platforms. Also, our emphasis on using neuro-evolution methods to evolve specific topology and configuration of control neural networks, results in the modest hardware requirements exposed.

We open sourced various components of our software framework at GitHub:

* https://github.com/yaricom/goNEAT

* https://github.com/yaricom/goNEAT_NS

* https://github.com/yaricom/goESHyperNEAT

In parallel, we are working on the hardware components of light-weight mobile platform to begin field experiments with autonomous behavior.

## REFERENCES

[1] Marijuán, P.C., Navarro, J., and del Moral, R. (2010). *On prokaryotic intelligence: Strategies for sensing the environment*. Biosystems, 99, 94–103 (2010). https://doi.org/10.1016/j.biosystems.2009.09.004

[2] Lyon, P. (2015). *The cognitive cell: Bacterial behaviour reconsidered*. Frontiers in Microbiology 6, 264 (2015). https://doi.org/10.3389/fmicb.2015.00264

[3] Parr, Thomas and Friston, Karl J. (2018). *The Anatomy of Inference: Generative Models and Brain Structure*. Frontiers in Computational Neuroscience, 12, 90 (2018). https://doi.org/10.3389/fncom.2018.00090

[4] Joel Lehman and Kenneth O. Stanley, (2011). *Novelty Search and the Problem with Objectives*. Genetic Programming: Theory and Practice IX (GPTP 2011), New York, NY: Springer, 2011. http://eplex.cs.ucf.edu/papers/lehmangptp11.pdf

[5] O'Regan, J. K. and Noë, A. (2001). *A sensorimotor account of vision and visual consciousness*. Behavioral and Brain Sciences 24(5): 939–73. https://doi.org/10.1017/S0140525X01000115

[6] David Gamez, (2018). *Human and Machine Consciousness*. Cambridge, UK: Open Book Publishers, 2018. https://doi.org/10.11647/OBP.0107

[7] Adams, R. A., Shipp, S., and Friston, K. J. (2013). *Predictions not commands: active inference in the motor system*. Brain Struct. Funct. 218, 611–643. https://doi.org/10.1007/s00429-012-0475-5

[8] Friston K, Kilner J, Harrison L. (2006). *A free energy principle for the brain*. J. Physiol Paris. 2006 Jul-Sep; 100(1-3):70-87. Epub 2006 Nov 13. https://doi.org/10.1016/j.jphysparis.2006.10.001

[9] Friston K. (2010). *The free-energy principle: a unified brain theory? Nature Reviews Neuroscience — Aop*, published online 13 Jan 2010; volume 11. https://doi.org/10.1038/nrn2787

[10] Crauel, H. and Flandoli, F. (1994). *Attractors for random dynamical systems*. Probab. Theory Relat. Fields 100, 365–393 (1994). https://doi.org/10.1007/BF01193705

[11] K. O. Stanley and R. Miikkulainen, (2004). *Competitive coevolution through evolutionary complexification*. Journal of Artificial Intelligence Research, 21:63–100, 2004. http://dl.acm.org/citation.cfm?id=1622467.1622471

[12] K. O. Stanley, B. D. Bryant, and R. Miikkulainen, (2005). *Real-time neuroevolution in the NERO video game*. IEEE Transactions on Evolutionary Computation Special Issue on Evolutionary Computation and Games, 9(6):653–668, 2005. https://doi.org/10.1109/TEVC.2005.856210

[13] J. C. Bongard, (2002). *Evolving modular genetic regulatory networks*. In Proceedings of the 2002 Congress on Evolutionary Computation, 2002. https://doi.org/10.1109/CEC.2002.1004528

[14] Beni G., Wang J. (1993). *Swarm Intelligence in Cellular Robotic Systems*. In: Dario P., Sandini G., Aebischer P. (eds) Robots and Biological Systems: Towards a New Bionics? NATO ASI Series (Series F: Computer and Systems Sciences), vol 102. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-58069-7_38

[15] Nasuto S., Bishop M. (2008). *Stabilizing Swarm Intelligence Search via Positive Feedback Resource Allocation*. In: Krasnogor N., Nicosia G., Pavone M., Pelta D. (eds) Nature Inspired Cooperative Strategies for Optimization (NICSO 2007). Studies in Computational Intelligence, vol 129. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-78987-1_11

[16] Kullback, S.; Leibler, R. A. (1951). *On Information and Sufficiency*. Ann. Math. Statist. 22 (1951), no. 1, 79–86. https://doi.org/10.1214/oms/1177729694

[17] R. A. Fisher and Edward John Russell. *On the mathematical foundations of theoretical statistics*. **222**. Philosophical Transactions of the Royal Society of London. Series A, Containing Papers of a Mathematical or Physical Character. http://doi.org/10.1098/rsta.1922.0009

[18] H. Attiya and J. Welch, (2004). *Distributed Computing: Fundamentals, Simulations and Advance Topics*. John Wiley & Sons inc., 2004, chap. 3. https://doi.org/10.1002/0471478210

[19] Haeupler, Bernhard & Ghaffari, Mohsen (2013). *Near Optimal Leader Election in Multi-Hop Radio Networks*. Proceedings of the Twenty-Fourth Annual ACM-SIAM Symposium on Discrete Algorithm. https://doi.org/10.1137/1.9781611973105.54
[20] Back, Thomas and Fogel, David B. and Michalewicz, Zbigniew (1997). *Handbook of Evolutionary Computation*. 1st ed. IOP Publishing Ltd. Bristol, UK, UK ISBN: 978-0-7-503-0895-3

[21] Erik P. Hoel, (2017). *When the Map Is Better Than the Territory*. Entropy. 2017; 19(5):188. https://doi.org/10.3390/e19050188

[22] Gagniuc, Paul A. (2017). *Markov Chains: From Theory to Implementation and Experimentation*. USA, NJ: John Wiley & Sons. pp.1–235. ISBN: 978-1-119-38755-8

[23] Cover, T.M.; Thomas, J.A. (2012). *Elements of Information Theory*. John Wiley & Sons: Hoboken, NJ, USA, 2012. ISBN: 978-0-471-24195-9

[24] Hope, L.R.; Korb, K.B. (2005). *An information-theoretic causal power theory*. In Australasian Joint Conference on Artificial Intelligence; Springer: Berlin/Heidelberg, Germany, 2005; pp. 805–811. https://doi.org/10.1007/978-0-387-84816-7_10

[25] Shannon, C.E. (1948). *A mathematical theory of communication*. Bell Syst. Tech. J. 1948, 27, 623–666. https://doi.org/10.1002/j.1538-7305.1948.tb01338.x

[26] Tononi, G.; Sporns, O. (2003). *Measuring information integration*. BMC Neurosci. 2003, 4, 31. https://doi.org/10.1186/1471-2202-4-31

[27] Robert Griesemer, Rob Pike, Ken Thompson, (2009). *The Go Programming Language*. Google GO team, 2009-2018. Retrieved from https://golang.org

[28] Jacob L. S. Bellmund, Peter Gärdenfors, Edvard I. Moser, Christian F. Doeller, (2018). *Navigating cognition: Spatial codes for human thinking*. Science 09 Nov 2018: Vol. 362, Issue 6415, eaat6766. https://doi.org/10.1126/science.aat6766

[29] Neuberg, L. (2003). *CAUSALITY: MODELS, REASONING, AND INFERENCE*, by Judea Pearl. Cambridge University Press, 2000. Econometric Theory, 19(4), 675-685. https://doi.org/10.1017/S0266466603004109