Maximizing Transfer Entropy in Agent Based Models
Ekaterina Sangati1, Georgina Montserrat Reséndiz-Benhumea1,2, Federico Sangati1,
Alexey Yudin1 and Tom Froese1
1Okinawa Institute of Science and Technology Graduate University, Okinawa, Japan
2Institute for Applied Mathematics and Systems Research, National Autonomous University ofMexico, Mexico City, Mexico

IntroductionResults• The entropic brain hypothesis (Carhart-Harris et al., 2014) proposes thatentropy can be viewed as an index of the state of consciousness thatspans the spectrum of cognitive states between high entropy associatedwith flexible cognition and low entropy correlated with inflexible cognition.
Normal waking
consciousness

Comparison between Shannon Entropy and Transfer Entropy• Using artificial two-dimensional data

e.g. Coma,
depression,
sedatione.g. Psychedelicstate, infant
consciousness, REM
sleep/dreaming

Criticality
proper

Flexible states

Inflexible states
Figure adapted from Carhart-Harris et al. (2014).

• This raises the question of which entropy-based measure of neuralcomplexity is maximized at the most adaptive levels of consciousness inthe middle of the spectrum. While simple Shannon entropy seems like aneasy choice, it is likely to be maximal when the system’s behavior is close
to random. It has been argued that a more proper measure is one thatreflects a balance between integration and segregation of the system’scomponents. This measure is often defined in terms of transfer entropy(Mäki-Marttunen et al., 2013; Tononi, Sporns & Edelman, 1994).
• In this work we compare Shannon entropy (SE) and transfer entropy (TE)in terms of:• their general characteristics derived from applying these measures tovarious cases of simulated artificial data
• their downstream effects on behavior when artificial agents areevolved to maximize neural complexity indexed by these twomeasures

Methods• Simulated two-dimensional data
100 samples of artificial 2D data of size N=2000 were generated for each of
the following cases:• uncorrelated random arrays from normal and uniform distributions• constant arrays of a single value ([1.0, 1.0,…], [1.0, 1.0, …])• correlated arrays of constant values ([1.0, 1.0,…], [2.0, 2.0, …])• uniform filling of all 100x100 bins (here N=10,000), in regular andrandom order• 1 random array and 2nd array correlated with the 1st at various levels of
delayed correlation• positions of a coupled spring mass system with various masses, spring
lengths and constants• Interactive agents based on Candadai et al. (2019) model•••

Pairs of embodied agents in a simulated 2dimensional environment. Each agent has
two acoustic sensors, one acoustic emitterand two motors driving wheels.
Neural architecture of an agent: Consistingin 3 layers (sensor, neuron and actuatorlayers) . The neuron layer has two neurons.
Evolutionary algorithm: Optimization of theparameters of the neural controllers tomaximize each agent’s neural entropy(Shannon Entropy or TE).

Plots of mean entropy values for Shannon entropy (SE), transfer entropy obtained separately for original continuous
values using KSG estimator (TEC) and values discretized into 100 bins using discrete estimator (TED).

• Using agent-based modeling of two agents in interactive scenario
Shannon Entropy

Continuous TE

Binned discrete TEBehaviorLow Entropy

Neural activity

High Entropy

Top row: plots of 2 agent movement trajectories. Bottom row: plots of the neural output of both agent’s 2 neurons(top-most N1 of agent 1, then N2 of agent 1, N1 of agent 2, N2 of agent 2).

Conclusions
1. On artificial data
• TEC and TED do not behave in completely parallel manner• TEC is predictably higher for ordered compared to shuffled uniform data
while TED is surprisingly high for shuffled uniform data
• for data in which one time series is random while the other highly
correlated, TEC is more sensitive to higher correlation than TED• dependencies in oscillatory data are characterized as entropic by SE• SE is maximized for random data, especially when drawn from a uniform
distribution• SE is not sensitive to the order of data points
2. On data produced by agent-based modeling• TEC leads to oscillatory behavior in neural activity. However, its range isextremely restricted and does not translate into behavioral complexity. Thisin turn, does not allow the agents to engage in interesting interactive
patterns which could lead to their reciprocal coupling and increase theirneural complexity (see Candadai et al., 2019).
• TED produces neural and behavioral complexity, which seems even higher
than that produced by maximizing SE. Whether this can lead to overall
more adaptive behaviors needs to be further investigated.

References• Carhart-Harris, R. L., Leech, R., Hellyer, P. J., Shanahan, M., Feilding, A., Tagliazucchi, E., Chialvo, D. R. and Nutt, D. (2014). The entropic brain: a theory of
conscious states informed by neuroimaging research with psychedelic drugs. Frontiers in Human Neuroscience, 8:20.
• Candadai, M., Setzler, M., Izquierdo, E. J. and Froese, T. (2019). Embodied Dyadic Interaction Increases Complexity of Neural Dynamics: A Minimal AgentBased Simulation Model. Frontiers in Psychology, 10:540.
• Mäki-Marttunen, V., Diez, I., Cortes, J. M., Chialvo, D. R., and Villarreal, M. (2013). Disruption of transfer entropy and inter-hemispheric brain functional
connectivity in patients with disorder of consciousness. Frontiers in Neuroinformatics, 7:24.
• Tononi, G., Sporns, O. and Edelman, G. M. (1994). A measure for brain complexity: Relating functional segregation and integration in the nervous system.
Proceedings of the National Academy of Sciences of the United States of America, 91(11): 5033-5037.