# Learning and Cognition in Financial Markets: A Paradigm Shift for Agent-Based Models

Johann Lussange, Alexis Belianin, Sacha Bourgeois-Gironde, Boris Gutkin

► To cite this version:

Johann Lussange, Alexis Belianin, Sacha Bourgeois-Gironde, Boris Gutkin. Learning and Cognition in Financial Markets: A Paradigm Shift for Agent-Based Models. Intelligent Systems and Applications Proceedings of the 2020 Intelligent Systems Conference, pp.241-255, 2021, 10.1007/978-3-030-55190-2_19. hal-03055057

HAL Id: hal-03055057

https://hal.science/hal-03055057

Submitted on 10 Nov 2022

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>
# Learning and Cognition in Financial Markets: A Paradigm Shift for Agent-Based Models

Johann Lussange¹(✉), Alexis Belianin², Sacha Bourgeois-Gironde³,⁴, and Boris Gutkin¹,⁵

¹ Laboratoire des Neurosciences Cognitives, INSERM U960, Département des Études Cognitives, École Normale Supérieure PSL University, 29 rue d’Ulm, 75005 Paris, France

johann.lussange@ens.fr

² ICEF, National Research University Higher School of Economics and Primakov Institute for World Economy and International Relations,

Russian Academy of Sciences, 8 Myasnitskaya st., 101000 Moscow, Russia

³ Institut Jean-Nicod, UMR 8129, Département des Études Cognitives, École Normale Supérieure PSL University, 29 rue d’Ulm, 75005 Paris, France

⁴ Laboratoire d’Économie Mathématique et de Microéconomie Appliquée, EA 4442, Université Paris II Panthéon-Assas, 4 rue Blaise Desgoffe, 75006 Paris, France

⁵ Center for Cognition and Decision Making, Department of Psychology, NU University Higher School of Economics,

8 Myasnitskaya st., 101000 Moscow, Russia

**Abstract.** The history of research in finance and economics has been widely impacted by the field of Agent-based Computational Economics (ACE). While at the same time being popular among natural science researchers for its proximity to the successful methods of physics and chemistry for example, the field of ACE has also received critics by a part of the social science community for its lack of empiricism. Yet recent trends have shifted the weights of these general arguments and potentially given ACE a whole new range of realism. At the base of these trends are found two present-day major scientific breakthroughs: the steady shift of psychology towards a hard science due to the advances of neuropsychology, and the progress of reinforcement learning due to increasing computational power and big data. We outline here the main lines of a computational research study where each agent would trade by reinforcement learning.

**Keywords:** Financial markets · Agent-based models · Multi-agent systems · Reinforcement learning

## 1 Past Research

The field of finance and economics has used various approaches to model financial markets dynamics. Among these we can historically distinguish three important
classes of models. The first and most encountered are statistical models which are calibrated to fit times series like past prices history. These can bring interesting results pertaining to general volatility [1,2] or log-return forecasting [3] as long as the variability of the parameters of calibration is not too strong. The second are known as Dynamic Stochastic General Equilibrium (DSGE) models which provide explicit agent-based microfoundations for the sectoral dynamics and aggregate fluctuations [4]. Modern developments of DSGE models strive to add realism to the basic model structure, accounting for agent heterogeneity, bounded rationality and imperfect learning, and (in the New Keynesian versions) replace the rational expectations hypothesis by market rigidities and exogenous stochastic shocks to emulate true market environment conditions [5-7]. These two classes of models have shown a variety of promising results over the years. However if we consider a top-down approach to system inference, we can say that they are based on rough approximations of reality [8,9], and will not explain the wealth and diversity of price microstructure traditionally seen in markets. This leads to a third class of models called Agent-Based Models (ABM) or sometimes Multi-Agent Systems (MAS) to probe and emulate markets from a pure bottom-up approach [10-12], and considering them as the complex systems [13] that they truly are. Among financial ABM models, we can also include order book models [14,15] even though some may see those as a midway approach. In a financial ABM, market investors or traders are modelled as agents trading together via an order book (such as a double auction order books [16]). This is a discrete-time algorithm taking in the trading bids at *t* and offers of specific securities from all agents, and matching them at transaction prices which then collectively define the price of the market for such securities at the time step *t* + 1.

ABM have been used in many scientific disciplines [17–19]. In economics, these models have emerged by way of psychological learning models [20], evolutionary biology [21,22], and especially game theory [23–27]. In recent years, ABM also became popular as a tool to study macroeconomics [28–31]—specifically, the impact of trading taxes, market regulatory policies, quantitative easing, and the general role of central banks [32]. ABM can also play an important role in analysis of the impact of the cross-market structure [33].

From a regulatory point of view, this implies a general stronger role for ABM to play [34]. Jean-Claude Trichet declared for instance in 2010: “As a policy-maker during the crisis, I found the available models of limited help. In fact, I would go further: in the face of the crisis, we felt abandoned by conventional tools. [...] Agent-based modelling dispenses with the optimisation assumption and allows for more complex interactions between agents. Such approaches are worthy of our attention.” A decade after the financial crisis, the structural causes to the repetition of such systemic risks in financial markets are far from being eliminated. One could hence say that their social and political implications makes ABM research all the more relevant today as it was a decade ago.

Even if ABM are often designed with many parameters and hence subject to the delicate issue of overfitting, one of their biggest advantages is that they
require fewer assumptions (e.g normal distribution of returns, no-arbitrage) than
top-down models [35]. Added to this, ABM display the famous complexity emer-
gence proper to bottom-up approaches [36] and can hence show completely new
regimes of transitions, akin to phase transitions in statistical physics [37]. How-
ever, being models, ABM are of course imperfect and need a thorough and
lengthy cross-market validation [38]. Yet at the same time, one should keep in
mind that such a general and cautious validation of ABM shall be in fact appli-
cable and necessary to any other model as well [39].

From now on we consider the application of ABM to financial markets. We shall note that among financial ABM some exclusively pertain to high-frequency trading [40,41], while other take both high- and low-frequency into account [42–44]. Another popular topic of literature in financial ABM is the emulation of the widespread Minority Game [45,46], which formally is not a financial market problem, but a general game theory problem which can be related to the financial issues of pricing and forecasting.

In order to generate a dynamic trading activity in financial ABM, a basic
economic assumption is that the agents disagree on the present security price or
trade at different frequencies (which possibility is sometimes explicitly denied in
economics literature [47]), and are hence willing to long or short a same security
at different prices. In other words, there must be some sort of price disagreement
happening and an original pricing mechanism at the discretion of each individual
agent. In the literature, this mechanism of pricing in financial ABM has in general
been designed according to two basic mechanisms of trading: in some models at
least a part of the agents trade in a random way as ‘noise traders’ [40, 41, 48–52],
and in other models agents use realistic trading strategies known to real financial
markets, depending on the variability and stochasticity of the market [53–56].

2 Accuracy

Over the years, economic research (and especially econophysics research) has gradually discovered a certain number of non-trivial statistical features of or stylised facts about financial times series. These stylised facts are based on variations in prices that have universal statistical properties in common from market to market, over different types of instruments, and time periods. Among these, those pertaining to returns distribution or volatility clustering for example were gradually discovered during the nineties: Kim-Markowitz [57], Levy-Levy-Solomon [58–64], Cont-Bouchaud [65], Solomon-Weisbuch [66], Lux-Marchesi [53,67], Donangelo-Sneppen [68–71], Solomon-Levy-Huang [72]. It was also not before this time that ABM started to emulate these stylised facts.

The importance of the universality of stylised facts to really gauge finan-
cial markets comes from the fact that the price evolutions of different markets
may have very different exogenous or endogenous causes. As a consequence they
highlight general underlying financial mechanisms that are market-independent,
and which can in turn be exploited for ABM architecture design. From a sci-
entific point of view, stylised facts are hence extremely interesting and their
faithful emulation has been an active topic of research in the past fifteen years or so [73,74]. Their definite characteristics has varied ever so slightly over the years and across literature, but the most widespread and unanimously accepted stylised facts can in fact be grouped in three broad, mutually overlapping categories:

*Non-gaussian returns*: the returns distribution is non-gaussian and hence asset prices should not be modeled as brownian random walks [75,76], despite what is taught in most text books, and applied in sell-side finance. In particular the real distributions of returns are dissimilar to normal distributions in that they are: (i) having fatter tails and hence more extreme events, with the tails of the cumulative distribution being well approximated [75,77] by a power law of exponent belonging to the interval [2, 4] (albeit this is still the subject of a discussion [78,79] famously started by Mandelbrot [80] and his Levy stable model for financial returns), (ii) negatively skewed and asymmetric in many observed markets [81] with more large negative returns than large positive returns, (iii) platykurtic and as a consequence having less mean-centered events [82], (iv) with multifractal *k*-moments so that their exponent is not linear with *k*, as seen in [83–86].

*Clustered volatilities*: market volatility tends to aggregate or form clusters [2]. Therefore compared to average, the probability to have a large volatility in the near-future is greater if it was large also in the near-past [73, 87, 88]. Regardless of whether the next return is positive or negative, one can thus say that large (resp. small) return jumps are likely followed by the same [80], and thus display some sort of long memory behaviour [89]. Because volatilities and trading volumes are often correlated, we also observe a related volume clustering.

*Decaying auto-correlations*: the auto-correlation function of the returns of financial time series are basically zero for any value of the auto-correlation lag, except for very short lags (e.g. half-hour lags for intraday data) because of a mean-reverting microstructure mechanism for which there is a negative auto-correlation [81,89]. This is sometimes feeding the general argument of the well-known Efficient Market Hypothesis [90,91] that markets have no memory and hence that one cannot predict future prices based on past prices or information [87,88]. According to this view, there is hence no opportunity for arbitrage within a financial market [77]. It has been observed however that certain non-linear functions of returns such as squared returns or absolute returns display certain steady auto-correlations over longer lags [89].

Since then, ABM of financial markets have steadily increased in realism and can generate progressively more robust scaling experiments. We can specifically highlight the potential of these simulations to forecast real financial time series via reverse-engineering. A promising recent perspective for such use of ABM has been highlighted in the field of statistics by [92–94]: the agent-based model parameters are constrained to be calibrated and fit real financial time series and then allowed to evolve over a given time period as a basic forecast measure on the original time series used for calibration. With this, one could thus say
that ABM are now reaching Friedman’s [95] methodological requirement that a
theory must be “judged by its predictive power for the class of phenomena which
it is intended to explain.”

3 Calibration

Just as any other model, the parameters of the ABM must be calibrated to real
financial data in order to perform realistic emulation. This part of calibration is
together with architecture design the most technical and crucial aspect of the
ABM [96]. Yet at the same time in the literature most calibration techniques
are done by hand, so that the stylised facts are re-enacted in a satisfactory way.
Therefore so far the ABM calibration step is often performed in way that is
sub-optimal [48,97,98].

On the other hand, an efficient methodology for calibration would need two
important steps. First a fully automated meta-algorithm in charge of the cali-
bration should be incorporated, so that a decently large amount of financial data
could be treated and the aforementioned scope of validity of ABM studied via
cross-market validations [48,99]. This is important as the robustness of a cali-
bration always relies on many runs of ABM simulations [100]. Second, this cali-
bration meta-algorithm should be working through the issues of overfitting and
underfitting, which may constitute a severe challenge due to the ever-changing
stochastic nature of financial markets.

Part of this calibration problem is to thoroughly and cautiously define the
parameter space. This step is particularly sensitive, since it can lead to poten-
tially problematic simplifications. For instance, what should be the size of the
time step of the simulation? ABM with a daily time tick will of course produce
time series that are much coarser than those coming from real financial data,
which include a wealth of intraday events [101–103].

4 Trends

As previously said, emergence and recent progress of two separate fields of
research will likely have a major upcoming impact on economic and financial
ABM. The first one is the recent developments in cognitive neuroscience and neu-
roeconomics [104–108], which has revolutionised behavioural economics with its
ever lower cost experimental methods of functional magnetic resonance imaging
(fMRI), electro-encephalography (EEG), or magneto-encephalography (MEG)
applied to decision [109, 110] and game theories [111]. The second one concerns
the recent progress of reinforcement learning which has reach in some tasks
superhuman performance [112, 113]. Among the multiple reinforcement learning
research fields, we can highlight in particular the recent progress of self-play
reinforcement learning [113, 114], end-to-end reinforcement learning and artifi-
cial neural networks [115–117], reinforcement learning and Monte Carlo tree
search [118], multi-agent learning [119–121], not to mention new types of unsu-
pervised algorithms [122, 123].
To the ABM field, this implies that the realism of the economic agents can be greatly increased via these two recent technological developments: the agents can be endowed with numerous cognitive and behavioural biases emulating those of human investors for instance, but also their trading strategies can be more faithful to reality in the sense that they can be dynamic and versatile depending on the general stochasticity or variability of the market, thanks to the fact that via reinforcement learning they will learn to trade and invest. In this respect, we shall mention with recent ABM literature [124] the recent attempts to design order book models with reinforcement learning [125], and the study of market making by reinforcement learning in a zero-intelligence ABM framework [126]. At a time where the economy and financial markets are progressively more and more automated, this impact of reinforcement learning on ABM should thus be explored. One should keep in mind that the central challenge (and in fact hypotheses) of ABM are the realism of the agents, but also the realism of the economic transactions and interactions between the agents.

# 5 Cognition and Behaviour

The reinforcement learning framework proper to all agents gives the possibility to implement certain traits or biases in the agents' cognition and behaviour, that are similar to those of human investors. The correspondance between reinforcement learning and decision making in the brain is an active field of research [127-130]. One could then reverse-calibrate these agents' traits and biases implemented in the reinforcement learning architecture of the autonomous agents in order to quantitatively gauge their impact on financial market dynamics at the macro-level. As financial stocks markets are increasingly impacted by the role played by algorithmic trading and the automation of transaction orders, the relevance of such a study hangs on the tight portfolio management constraints (e.g. risk management, liquidity preferences, acceptable drawdown levels, etc.) imposed by human investors, which algotrading strategies take as cost-functions. We hence propose here a set of a dozen of particularly interesting and important cognitive and behavioural biases, and their possible implementation within reinforcement learning algorithmics. In such a framework, each agent would be initialised at the beginning of the simulation with some or all of the following cognitive and behavioural biases, according to specific biases distributions in the population of agents:

*Belief Revision:* Defined as to changing one's belief insufficiently in the face of new evidence. Each agent is naturally endowed with a parameter relevant to this bias in reinforcement learning, called the reinforcement learning rate.

*Hyperbolic Discounting:* Defined as having greater economic utility for immediate rewards, rather than delayed ones. This economic value could follow a quasi-hyperbolic discount function following [131]. It could be modeled through a certain amount of the agents having a shorter investment horizon, or via a quasi-hyperbolic function accordingly weighting the returns of the agents.
*Loss Aversion:* Defined as demanding much more to sell an asset than to buy it. Because of loss aversion, the agent would for instance favour an ask price much higher than the bid price, in its transaction order sent to the order book.

*Illusory Superiority (Resp. Inferiority):* Defined as overestimating (resp. underestimating) one's own abilities compared to others. This could be implemented in two ways: because market volatility is generally considered in portfolio management as a main indicator of risk or uncertainty, the agent's illusory superiority (resp. inferiority) could first be modelled via an alteration of the agent's state, by decreasing (resp. increasing) its perceived stock price volatility. Another possible implementation would be to enhance (resp. lessen) the agent's past returns. These updates could be performed with varying degrees of frequency.

*Fear and Greed:* These are the two main driving forces of economic investors. For a long-only equity strategy implemented in the agent, it would display fear (resp. greed) by being more prone to short (resp. long) equity. One could also implement such fear and greed in the agents by varying the type of order sent to the order book, e.g. limit vs. market orders. These updates could be performed with varying degrees of frequency.

*Exaggerated (Resp. Lowered) Expectation:* Also called regressive bias, this is defined as overestimating (resp. underestimating) high values and likelihoods while underestimating (resp. overestimating) low values and likelihoods. This could be implemented in two ways: the agent state relevant to the perceived stock price trend (weather it would increase, decrease, or remain stable) could be modified accordingly, or the stock price volatility perceived by the agent could be modified accordingly. These updates could be performed with varying degrees of frequency.

*Negativity (Resp. Positive) Bias:* Defined as better recalling unpleasant (resp. pleasant) memories than pleasant (resp. unpleasant) ones. This could easily be modelled by amending the agent's past returns accordingly. This is an easy implementation in the reinforcement learning framework applied to portfolio optimisation, since the return accomplished by the agent is fully known at the end of its investment time horizon, and would not require a computationally heavy operation on the agent policy and Q-function to forget and relearn the associated impact of its returns. These updates could be performed with varying degrees of frequency.

*Egocentric Bias:* Defined as remembering the past as better or worse than it was in a self-serving manner. This could be modelled as above.

# 6 Application

Besides studying the agents’ collective trading interactions with one another, an ABM stock market simulation could be used to probe the following specific fields of study:
*Market Macrostructure:* one could study how human cognitive and behavioural biases change agents’ behaviour at a level of market macrostructure. The two main topics of market macrostructure that would be of interest are naturally all those pertaining to systemic risk [132,133] (bubble and crash formation, problem of illiquidity), and also those related to herding-type phenomena [134] (information cascades, rational imitation, reflexivity). A key aspect of this work would be also to carefully calibrate and compare the stylised facts to real financial data (we need to see standardised effects such as volatility clustering, leptokuric log-returns, fat tails, long memory in absolute returns). Indeed some of these macro-effects have been shown to arise from other agent-based market simulators [11,135], however it is not yet fully understood how these are impacted by the agents learning process nor their cognitive and behavioural biases such as risk aversion, greed, cooperation, inter-temporality, and the like.

*Price Formation:* one could study how these biases change the arbitrage possibilities in the market via their blatant violation of the axioms of the Efficient Market Hypothesis [136]. Indeed agent-based market simulators so far have often relied on the use of the aforementioned 'noise traders' in order to generate the necessary conditions for basic business activity [12, 33, 137]. The novel aspect here would be to go one step further and replace this notion of purely random trading by implementing specific neuroeconomic biases common to real human behaviours. Another problem of past research in agent-based stock market simulators is that these have often relied on agents getting their information for price forecasting from a board of technical indicators common to all agents [138]. In contrast, we should develop agent models that allow each agent to be autonomous and use and ameliorate its own forecasting tools. This is a crucial aspect in view of the well-known fact that information is at the heart of price formation [141]. We hypothesise that such agent learning dynamics is fundamental in effects based on market reflexivity and impact on price formation, and hints to the role played by fundamental pricing versus technical pricing of assets. Of major interest is the issue of global market liquidity provision and its relation to the law of supply and demand and bid-ask spread formation [139, 140, 142].

*Credit Risk:* one could study the effect of market evolutionary dynamics when the agents are allowed to learn and improve their trading strategy and cognitive biases via reinforcement learning. In other words, it would be interesting to see the agents population survival rates [143–145] (cf. credit risk), and overall price formation with respect to the arbitrage-free condition of markets when we increase the variability of the intelligence in the trading agents [146]. Indeed, recent studies [137, 147, 148] suggest that arbitrage opportunities in markets arise mainly from the collective number of non-optimal trading decisions that shift the prices of assets from their fundamental values via the law of supply and demand. Another topic would be to compare such agent survivability with Zipf’s law [145].
# 7 Preliminary Results

We have recently started to develop an ABM stock market simulator with autonomous agents trading by reinforcement learning, whose general architecture is described in [149]. In such a simulator, whose parameters are calibrated to real data coming from the London Stock Exchange between 2008 and 2018, a number *I* of agents trade over *T* time steps a quantity of *J* different stocks. These agents first learn by reinforcement how to forecast future prices, with actions as econometric parameters set for mean-reverting, averaging, or trend- following, and states depending on the market volatilities at different time scales, and rewards as the mismatch between such predictions and the realised market price. A second reinforcement learning algorithm is used by each agent to trade according to this econometric output and learned information, with its action being to sell, buy or hold its assets, and at what price, with the reward being defined as the realised cashflow consequent to this decision. At each time step of the simulation, agents thus send their trading orders to a centralised order book, which shuffles and clears them by matching orders. The latest transaction sets the market price of the stock for the next time step, which in turn is used by all agents to update their state. We show below some early results from the simulations of this ABM model.

<figure><img src="image_2.png" /><figcaption>Fig. 1. (a) Absolute logarithmic price returns of the simulated stock market as a function of time. (b) Price volatility at two-weeks intervals of the simulated stock market as a function of time. (c) Trading volumes of the simulated stock market as a function of time. The simulation is for 500 agents over 2875 time steps.</figcaption></figure>
We first want to check on the model capacity to emulate the aforementioned clustering activity of the agents. We show this on Fig. 1, as an output of the simulation for 500 agents trading a given stock over 2875 time steps wrt. to absolute logarithmic price returns, price volatilities at a two weeks rolling interval, and trading volumes.

We then want to see as a preliminary result whether the agents learn correctly, and if their performance can be revealed by de-trending their profits from market prices. We show on Fig. 2 the performances of four randomly selected agents over 2875 time steps, de-trended from stock price. We can notice, after a number of time steps corresponding to about half the total simulation time, a real performance and hence learning process for the second agent.

<figure><img src="image_3.png" /><figcaption>**Fig. 2.** Example of certain agents' profit, de-trended from market performance and as a function of time. The simulation is for 500 agents over 2875 time steps.</figcaption></figure>

# 8 Conclusion

We have thus highlighted new possible exciting perspectives for financial ABM, where the agents would be designed with neuroeconomic biases and having trading or investment strategies updated by reinforcement learning. One should recall that the main argument against ABM, and indeed their main challenge, has always been about the realism of the agents, albeit one should also consider the realism of the economic transactions. We thus argue that these recent trends should set a totally new level of realism for financial ABM.

In particular, whereas early financial ABM generations increased their realism of emulation of real stock markets by re-enacting stylised facts gradually during the late nineties, and whereas the issue of calibration is still undergoing the process of automation so that ABM may be validated on large scales of data, we expect these trends to bring in several revolutionary breakthroughs, and the emergence and recognition of ABM as the relevant tools that they are in finance and economics.

**Acknowledgment.** This work was supported by the RFFI grant nr. 16-51-150007 and CNRS PRC nr. 151199, and received support from FrontCog ANR-17-EURE-0017. A preliminary preprint of this work was uploaded to arXiv [150].
References

1. Bollerslev, T.: CREATES research paper 2008, p. 49 (2008)

2. Engle, R.F.: Econometrica 50(4), 987 (1982)

3. Brownlees, C.T., Engle, R.F., Kelly, B.T.: *J. Risk* **14**(2), 3 (2011)

4. Sbordone, A.M., Tambalotti, A., Rao, K., Walsh, K.J.: Econ. Policy Rev. **16**(2) (2010)

5. Evans, G.W., Honkapohja, S.: Learning and Expectations in Macroeconomics. Princeton University Press, Princeton (2001)

6. Eusepi, S., Preston, B.: Am. Econ. Rev. **101**, 2844 (2011)

7. Massaro, D.: *J. Econ. Dyn. Control* **37**, 680 (2013)

8. Farmer, J.D., Foley, D.: Nature **460**(7256), 685 (2009)

9. Grauwe, P.D.: Public Choice **144**(3-4), 413 (2010)

10. Tesfatsion, L., Judd, K.L.: Handbook of Computational Economics: Agent-Based Computational Economics, vol. II. Elsevier, Amsterdam (2006)

11. Samanidou, E., Zschischang, E., Stauffer, D., Lux, T.: Rep. Prog. Phys. **70**(3), 409 (2007)

12. LeBaron, B.: Building the Santa Fe Artificial Stock Market (2002)

13. Bonabeau, E.: Harvard Bus. Rev. **80**(3), 109 (2002)

14. Smith, E., Farmer, D.J., Gillemot, L., Krishnamurthy, S.: Quant. Finance **3**, 481 (2003)

15. Huang, W., Lehalle, C.-A., Rosenbaum, M.: *J. Am. Stat. Assoc.* **110**, 509 (2015)

16. Mota, R., Larralde, H.: arXiv:1601.00229 (2016)

17. Macal, C.M., North, M.J.: *J. Simul.* **4**, 151 (2010)

18. Axelrod, R.M.: The Complexity of Cooperation: Agent-Based Models of Competition and Collaboration. Princeton University Press, Princeton (1997)

19. Grimm, V., Berger, U., Bastiansen, F., Eliassen, S., Ginot, V., Giske, J., Goss-Custard, J., Grand, T., Heinz, S.K., Huse, G., et al.: Ecol. Model. **198**(1), 115 (2006)

20. Bush, R.R., Mosteller, F.: Stochastic Models for Learning. Wiley, Oxford (1955)

21. Smith, J., Price, D.: Nature **246**, 15 (1973)

22. Taylor, P.D., Jonker, L.B.: Math. Biosci. **40**, 145 (1978)

23. Mookherjee, D., Sopher, B.: Games Econ. Behav. **7**, 62 (1994)

24. Erev, I., Roth, A.E.: Am. Econ. Rev. **88**, 848 (1998)

25. Erev, I., Roth, A.E.: PNAS **111**, 10818 (2014)

26. Camerer, C.F., Ho, T.H.: PNAS **67**, 827 (1999)

27. Fudenberg, D., Levine, D.: The Theory of Learning in Games. MIT Press, Cambridge (1998)

28. Colander, D., Howitt, P., Kirman, A., Leijonhufvud, A., Mehrling, P.: Am. Econ. Rev. 236-240 (2008)

29. Dosi, G., Fagiolo, G., Napoletano, M., Roventini, A.: *J. Econ. Dyn. Control* **37**(8), 1598 (2013)

30. Gualdi, S., Tarzia, M., Zamponi, F., Bouchaud, J.-P.: *J. Econ. Interact. Coord.* 1-31 (2016)

31. Gualdi, S., Tarzia, M., Zamponi, F., Bouchaud, J.-P.: *J. Econ. Dyn. Control* **50**, 29 (2015)

32. Westerhoff, F.H.: Jahrbucher Fur Nationalokonomie Und Statistik **228**(2), 195 (2008)

33. Xu, H.-C., Zhang, W., Xiong, X., Zhou, W.-X.: Math. Prob. Eng. **2014**, 563912 (2014)
34. Boero, R., Morini, M., Sonnessa, M., Terna, P.: Agent-Based Models of the Economy, From Theories to Applications. Palgrave Macmillan, New York (2015)

35. LeBaron, B.: Agent-based computational finance. In: The Handbook of Computational Economics, vol. 2. Elsevier, Amsterdam (2005)

36. Heylighen, F.: Complexity and Self-Organization. CRC Press, Boca Raton (2008)

37. Plerou, V., Gopikrishnan, P., Stanley, H.E.: Nature **421**, 130 (2003)

38. Hamill, L., Gilbert, N.: Agent-Based Modelling in Economics. Wiley, Hoboken (2016)

39. Wilcox, D., Gebbie, T.: arXiv:1408.5585 (2014)

40. Hanson, T.A.: Midwest finance association 2012 annual meetings paper (2011)

41. Bartolozzi, M.: Eur. Phys. J. B **78**, 265 (2010)

42. Wah, E., Wellman, M.P.: Proceedings of the Fourteenth ACM Conference on Electronic Commerce, pp. 855–872 (2013)

43. Paddrik, M.E., Hayes, R.L., Todd, A., Yang, S.Y., Scherer, W., Beling, P.: SSRN 1932152 (2011)

44. Aloud, M., Tsang, E., Olsen, R.: Business science reference, Hershey (2013)

45. Challet, D., Marsili, M., Zhang, Y.-C.: Minority Games: Interacting Agents in Financial Markets. Oxford University Press, Oxford (2005)

46. Martino, A.D., Marsili, M.: J. Phys. A **39**, 465 (2006)

47. Kyle, A.S., O. A.: Econométrica forthcoming (2016)

48. Preis, T., Golke, S., Paul, W., Schneider, J.J.: Europhys. Lett. **75**(3), 510 (2006)

49. Farmer, J.D., Patelli, P., Zovko, I.I.: Proc. Natl. Acad. Sci. U.S.A. **102**(6), 2254 (2005)

50. Maslov, S.: Phys. A **278**(3), 571 (2000)

51. Challet, D., Stinchcombe, R.: Quant. Finance **3**(3), 155 (2003)

52. Schmitt, T.A., Schfer, R., Mnnix, M.C., Guhr, T.: Europhys. Lett. 100 (2012)

53. Lux, T., Marchesi, M.: J. Theor. Appl. Finance **3**, 67 (2000)

54. Cont, R.: Volatility clustering in financial markets: empirical facts and agent-based models. Springer (2007)

55. Bertella, M.A., Pires, F.R., Feng, L., Stanley, H.E.: PLoS ONE **9**(1), e83488 (2014)

56. Alfi, V., Cristelli, M., Pietronero, L., Zaccaria, A.: Eur. Phys. J. B **67**(3), 385 (2009)

57. Kim, G., Markowitz, H.M.: J. Portfolio Manag. **16**, 45 (1989)

58. Levy, M., Solomon, S.: Int. J. Mod. Phys. C **7**, 595 (1996)

59. Levy, M., Levy, H., Solomon, S.: Econ. Lett. **45**, 103 (1994)

60. Levy, M., Levy, H., Solomon, S.: J. Phys. I **5**, 1087 (1995)

61. Levy, M., Solomon, S.: Int. J. Mod. Phys. C **7**, 65 (1996)

62. Levy, M., Persky, N., Solomon, S.: Int. J. High Speed Comput. **8**, 93 (1996)

63. Levy, M., Levy, H., Solomon, S.: Phys. A **242**, 90 (1997)

64. Levy, M., Levy, H., Solomon, S.: Microscopic Simulation of Financial Markets. Academic Press, New York (2000)

65. Cont, R., Bouchaud, J.P.: Macroecon. Dyn. **4**, 170 (2000)

66. Solomon, S., Weisbuch, G., de Arcangelis, L., Jan, N., Stauffer, D.: Phys. A **277**(1), 239 (2000)

67. Lux, T., Marchesi, M.: Nature **397**, 498 (1999)

68. Donangelo, R., Hansen, A., Sneppen, K., Souza, S.R.: Phys. A **283**, 469 (2000)

69. Donangelo, R., Sneppen, K.: Phys. A **276**, 572 (2000)

70. Bak, P., Norrelykke, S., Shubik, M.: Phys. Rev. E **60**, 2528 (1999)

71. Bak, P., Norrelykke, S., Shubik, M.: Quant. Finance **1**, 186 (2001)
72. Huang, Z.F., Solomon, S.: Eur. Phys. J. B **20**, 601 (2000)

73. Lipski, J., Kutner, R.: arXiv:1310.0762 (2013)

74. Barde, S.: School of economics discussion papers 04. University of Kent (2015)

75. Potters, M., Bouchaud, J.-P.: Phys. A **299**, 60 (2001)

76. Plerou, V., Gopikrishnan, P., Amaral, L.A., Meyer, M., Stanley, H.E.: Phys. Rev. E **60**(6), 6519 (1999)

77. Cristelli, M.: Complexity in Financial Markets. Springer, Cham (2014)

78. Weron, R.: Int. J. Mod. Phys. C **12**, 209 (2001)

79. Eisler, Z., Kertesz, J.: Eur. Phys. J. B **51**, 145 (2006)

80. Mandelbrot, B.: J. Bus. 394-419 (1963)

81. Cont, R.: Quant. Finance **1**, 223 (2001)

82. Bouchaud, J., Cont, R., Potters, M.: Scale invariance and beyond. In: Proceedings of CNRS Workshop on Scale Invariance. Springer, Les Houches (1997)

83. Ding, Z., Engle, R., Granger, C.: J. Empir. Finance **1**, 83 (1993)

84. Lobato, I.N., Savin, N.E.: J. Bus. Econ. Stat. **16**, 261 (1998)

85. Vandewalle, N., Ausloos, M.: Phys. A **246**, 454 (1997)

86. Mandelbrot, B., Fisher, A., Calvet, L.: A multifractal model of asset returns. Cowles Foundation for Research and Economics (1997)

87. de Vries, C., Leuven, K.: Stylized facts of nominal exchange rate returns. Working Papers from Purdue University, Krannert School of Management Center for International Business Education and Research (CIBER) (1994)

88. Pagan, A.: J. Empir. Finance **3**, 15 (1996)

89. Cont, R.: Volatility clustering in financial markets: empirical facts and agent-based models. In: Kirman, A., Teyssiere, G. (eds.) Long Memory in Economics. Springer (2005)

90. Fama, E.: J. Finance **25**, 383 (1970)

91. Bera, A.K., Ivliev, S., Lillo, F.: Financial Econometrics and Empirical Market Microstructure. Springer, Cham (2015)

92. Wiesinger, J., Sornette, D., Satinover, J.: Comput. Econ. **41**(4), 475 (2012)

93. Andersen, J.V., Sornette, D.: Europhys. Lett. **70**(5), 697 (2005)

94. Zhang, Q.: Disentangling financial markets and social networks: models and empirical tests. Ph.D. thesis, ETH Zurich (2013)

95. Friedman, M.: Essays in Positive Economics. Chicago University Press, Chicago (1953)

96. Canova, F., Sala, L.: J. Monetary Econ. **56**(4), 431 (2009)

97. Chiarella, C., Iori, G., Perello, J.: J. Econ. Dyn. Control **33**, 525 (2009)

98. Leal, S.J., Napoletano, M., Roventini, A., Fagiolo, G.: J. Evol. Econ. **26**, 49 (2016)

99. Fabretti, A.: J. Econ. Interact. Coord. **8**, 277 (2013)

100. Axtell, R.: Center on social and economic dynamics working paper 17 (2000)

101. Gilli, M., Winker, P.: Comput. Stat. Data Anal. **42**, 299 (2003)

102. Farmer, J.D., Joshi, S.: J. Econ. Behav. Organ. **49**, 149 (2002)

103. Kirman, A.: Epidemics of opinion and speculative bubbles in financial markets. In: Money and Financial Markets. Macmillan, New York (1991)

104. Glimcher, P.W., Camerer, C.F., Fehr, E., Poldrack, R.A.: Neuroeconomics: Decision Making and the Brain. Academic Press, Cambridge (2009)

105. Camerer, C.: J. Econ. Lit. **51**(4), 1155 (2013)

106. Martino, B.D., Doherty, J.P.O., Ray, D., Bossaerts, P., Camerer, C.: Neuron **79**(6), 1222 (2013)

107. Camerer, C.: Ann. Rev. Econ. **5**, 425 (2013)

108. Camerer, C.: Neuroscience, game theory, monkeys. TEDx talk (2013)
109. Kahneman, D., Tversky, A.: Econometrica **47**(2), 263 (1979)

110. Frydman, C., Barberis, N., Camerer, C., Bossaerts, P., Rangel, A.: NBER working paper 18562 (2012)

111. Camerer, C.F.: Behavioral Game Theory: Experiments on Strategic Interaction. Princeton University Press, Princeton (2003)

112. Silver, D., Hubert, T., Schrittwieser, J., Antonoglou, I., Lai, M., Guez, A., Lanctot, M., Sifre, L., Kumaran, D., Graepel, T., et al.: Science **362**, 1140 (2018)

113. Silver, D., Schrittwieser, J., Simonyan, K., Antonoglou, I., Huang, A., Guez, A., Hubert, T., Baker, L., Lai, M., Bolton, A., et al.: Nature **550**, 354 (2017)

114. Doll, B.B., Duncan, K.D., Simon, D.A., Shohamy, D.S., Daw, N.D.: Nat. Neurosci. **18**, 767 (2015)

115. Goodfellow, I., Bengio, Y., Courville, A.: Deep Learning. MIT Press, Cambridge (2016)

116. Schmidhuber, J.: Neural Netw. **61**, 85 (2015)

117. Turchenko, V., Beraldi, P., Simone, F.D., Grandinetti, L.: The 6th IEEE International Conference on Intelligent Data Acquisition and Advanced Computing Systems: Technology and Applications (2011)

118. Silver, D., Huang, A., Maddison, C.J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., et al.: Nature **529**, 484 (2016)

119. Tuyls, K., Weiss, G.: AI Mag. Fall (2012)

120. Heinrich, J., Silver, D.: AAAI Workshop (2014)

121. Heinrich, J., Silver, D.: IJCAI (2015)

122. Reed, S., Akata, Z., Yan, X., Logeswaran, L., Lee, H., Schiele, B.: ICML (2016)

123. Lerer, A., Gross, S., Fergus, R.: ICML (2016)

124. Biondo, A.E.: J. Econ. Interact. Coord. **14**(3) (2018)

125. Spooner, T., Fearnley, J., Savani, R., Koukorinis, A.: Proceedings of the 17th AAMAS (2018)

126. Ganesh, S., Vadori, N., Xu, M., Zheng, H., Reddy, P., Veloso, M.: arXiv:1911.05892 (2019)

127. Lefebvre, G., Lebreton, M., Meyniel, F., BourgeoisGironde, S., Palminteri, S.: Nat. Hum. Behav. **1**, 1 (2017)

128. Duncan, K., Doll, B.B., Daw, N.D., Shohamy, D.: Neuron **98**, 645 (2018)

129. Momennejad, I., Russek, E., Cheong, J., Botvinick, M., Daw, N.D., Gershman, S.J.: Nat. Hum. Behav. **1**, 680–692 (2017)

130. Palminteri, S., Khamassi, M., Joffily, M., Coricelli, G.: Nat. Commun. 1–14 (2015)

131. Laibson, D.: Q. J. Econ. **112**(2), 443 (1997)

132. The financial crisis inquiry report. Official government edition (2011)

133. Fouque, J.-P., Langsam, J.A.: Handbook on Systemic Risk. Cambridge University Press, Cambridge (2013)

134. Bikhchandani, S., Sharma, S.: Int. Monetary Fund. **47**(3) (2001)

135. Bikhchandani, S., Hirshleifer, D., Welch, I.: J. Polit. Econ. **100**(5), 992 (1992)

136. Fama, E.: J. Bus. **38**, 34 (1965)

137. Sornette, D.: arXiv:1404.0243v1 (2014)

138. da Costa Pereira, C., Mauri, A., Tettamanzi, A.G.B.: IEEE Computer Society WIC ACM (2009)

139. Kyle, A.S.: Econometrica **53**, 1315 (1985)

140. Sanford, G.J., Miller, M.H.: J. Finance **43**, 617 (1988)

141. Grossman, S.J., Stiglitz, J.E.: Am. Econ. Rev. **70**, 393 (1980)

142. Cason, T.N., Friedman, D.: Exp. Econ. **2**, 77 (1999)
143. Evstigneev, I.V., Hens, T., Schenk-Hopp, K.R.: Evolutionary finance. In: Handbook of Financial Markets, Dynamics and Evolution. North-Holland, Elsevier (2009)

144. Saichev, A., Malevergne, Y., Sornette, D.: Theory of Zipf's Law and Beyond. Lecture Notes in Economics and Mathematical Systems, vol. 632. Springer, Heidelberg (2010)

145. Malevergne, Y., Saichev, A., Sornette, D.: J. Econ. Dyn. Control **37**(6), 1195 (2013)

146. Hasanhodzic, J., Lo, A.W., Viola, E.: Quant. Finance **11**(7), 1043 (2011)

147. Malkiel, B.G.: A Random Walk Down Wall Street: The Time-Tested Strategy for Successful Investing, 10th revised edn. W. W. Norton and Company (2012)

148. Black, F.: J. Finance **41**(3), 529 (1985)

149. Lussange, J., Bourgeois-Gironde, S., Palminteri, S., Gutkin, B.: arXiv:1909.07748 (2019)

150. Lussange, J., Belianin, A., Bourgeois-Gironde, S., Gutkin, B.: arXiv:1801.08222 (2018)