Original: Evology: an Empirically-Calibrated Market Ecology Agent-Based Model for Trading Strategy Search
Revised: Evology: an Empirically-Calibrated Market Ecology Agent-Based Model for Trading Strategy Search

Original: Anonymous Authors

Original: Abstract
Revised: Abstract

Original: Market ecology views financial markets as ecosystems of diverse, interacting and evolving trading strategies.
Revised: Market ecology views financial markets as ecosystems of diverse, interacting and evolving trading strategies.

Original: We present a heterogeneous, empirically calibrated multi-agent market ecology agentbased model.
Revised: We present a heterogeneous, empirically calibrated multi-agent market ecology agentbased model.

Original: We outline its potential as a valuable and challenging training ground for optimising trading  strategies using machine learning algorithms and defining research tasks
Revised: We outline its potential as a valuable and challenging training ground for optimising trading and investment strategies using machine learning algorithms.

Original: 1. Introduction 1.1. Motivation
Revised: 1. Introduction 1.1. Motivation

Original: The theory of market ecology (Farmer, 2002; LeBaron, 2002; Musciotto et al., 2018; Lo, 2019; Levin & Lo, 2021;
Revised: The theory of market ecology (Farmer, 2002; LeBaron, 2002; Musciotto et al., 2018; Lo, 2019; Levin & Lo, 2021;

Original: Scholl et al., 2021) borrows concepts from ecology and biology to study financial markets.
Revised: Scholl et al., 2021) borrows concepts from ecology and biology to study financial markets.

Original: Trading strategies are analogous to biological species: they exploit market inefficiencies and compete for survival or profit.
Revised: Trading strategies are analogous to biological species: they exploit market inefficiencies and compete for survival or profit.

Original: (Scholl et al., 2021) has highlighted the nature of interactions between common trading strategies and the strong density dependence of their returns. 
Revised: (Scholl et al., 2021) has highlighted the nature of interactions between common trading strategies and the strong density dependence of their returns. 1.2. Related work This article is in the continuity of the rich area of financial agent-based models, and market selection with heterogeneous beliefs (Blume & Easley, 1992; 2006).

Revised: For example, several ABMs have recently been introduced for marketmaking optimisation (Spooner et al., 2018), understanding flash crashes (Paulin et al., 2019) and providing sophistiA. Vi ´ e designed and implemented the modelling and performed the experiments.

Revised: M. Scholl gathered and compiled the data used for calibration.

Revised: MS and D. Farmer provided guidance and help for the modelling.

Original: 1.2. Contribution
Revised: All authors wrote/revised the article. 1 Mathematical Institute, University of Oxford, Oxford, United Kingdom 2 Institute of New Economic Thinking, University of Oxford, United King- dom 3 Department of Computer Science, University of Oxford, United

Revised: 4 Santa Fe Institute, Santa Fe, New

Revised: United States of

Revised: Correspondence to: Aymeric Vi´e < vie maths ox ac

Original: In this article, we present Evology: an empirically-calibrated market ecology agent-based model (ABM), in the continuity ofthe market ecology perspective.
Revised: > . ICML authors. cated financial architectures for trading training (Byrd et al., 2019). We attempt to develop the complementary approach of market ecology (Scholl et al., 2021) by focusing on the ecological interactions between the different types of agents and strategies.

Original:  We contribute to the market ecology literature  by featuring heterogeneous strategy types and sub-strategies: pessimistic and optimistic noise traders and value investors, and trend followers trading over various time horizons.
Revised: Our contribution to the existing literature is thus on the interactivity and heterogeneity of the financial agents in a more simplified, low-frequency financial environment with a low-dimensional action space. 1.3. Contribution In this article, we present Evology: an empirically-calibrated market ecology agent-based model (ABM), in the continuity of the market ecology perspective. We contribute to the market ecology literature (Farmer, 2002; Scholl et al., 2021) by featuring heterogeneous strategy types and sub-strategies: pessimistic and optimistic noise traders and value investors, and trend followers trading over various time horizons.

Original: We improve the realism of the resulting wealth flows by validating the model with stylised facts and calibrating investment dynamics and short interest.
Revised: We improve the realism of the resulting wealth flows by validating the model with stylised facts and calibrating investor behaviour.

Original: We describe how some particular results of the market ecology model provide a new, exciting challenge for optimising trading strategies using machine learning algorithms
Revised: We describe how some particular results of the market ecology model provide a new, exciting challenge for optimising trading strategies using machine learning algorithms.

Original: Such simulation-based training can account for interactions and density dependence effects that could be significant and overlooked by traditional time-series training.
Revised: Such simulation-based training can account for interactions and density dependence effects that could be significant and overlooked by traditional time-series training.

Original: 2. Model
Revised: 2. Model

Original: We consider a population of n agents, acting as investment funds, who trade shares of a single representative asset.
Revised: We consider a population of n agents, acting as investment funds, who trade shares of a single representative asset and cash in the form of a bond.

Original: Every period-day t , the funds can buy, sell and short-sell shares of the asset in constant supply.
Revised: Every period-day t , the funds can buy, sell and short-sell shares of the asset in constant supply.

Original: Asset shares pay daily dividends D ( t ) following an autocorrelated Geometric Brownian Motion.
Revised: Asset shares pay daily dividends δ ( t ) following an autocorrelated Geometric Brownian Motion.

Original: Cash yields an interest rate r paid daily.
Revised: Cash/bond yields an interest rate r paid daily.

Original: We build over the foundational model of (Scholl et al., 2021), with notable improvements in the heterogeneity of strategies and empirical calibration, as well as adding a new strategy evolution component described in Section 5. 
Revised: We build over the foundational model of (Scholl et al., 2021), with notable improvements in the heterogeneity of strategies and empirical calibration and adding a new strategy evolution component described in Section 5. We show an example simulation run in the Supplementary Information.

Original: Initialise
Revised: Initialise

Original: Trading signal
Revised: Trading signal

Original: Excess demand
Revised: Demand

Original: Market clearing
Revised: Market clearing

Original: Earnings
Revised: Dividends/interest

Original: Strategy evolution
Revised: Strategy evolution

Original: Investment Terminate
Revised: Investment Terminate

Original: For T max periods
Revised: For T max periods

Original: After T max periods
Revised: After T max periods

Original: 2.1. Trading strategies and signals
Revised: 2.1. Trading strategies and signals

Original: Like real markets, our financial market model features a diverse sample of stylized versions of the most common funds’ trading strategies (Scholl et al., 2021), adding heterogeneous sub-strategies. 
Revised: Like real markets, our financial market model features a diverse sample of stylised versions of the most common funds’ trading strategies (Scholl et al., 2021), adding heterogeneous sub-strategies. Value investors (VI) form heterogeneous subjective valuations

Original:  Noise traders (NT) trade on individual-specific, mean-reverting Ornstein Uhlenbeck processes X ( t ) , mimicking crowd dynamics not driven by financial fundamentals.
Revised: V i of the asset based on discounted sums of dividends. Noise traders (NT) trade on a similar valuation perturbed by a mean-reverting Ornstein Uhlenbeck process X ( t ) , mimicking exogenous sentiment dynamics.

Original: We calibrate the parameters of the Ornstein Uhlenbeck process to match empirical excess volatility (Scholl et al., 2021).
Revised: We calibrate the parameters of the Ornstein Uhlenbeck process to match empirical excess volatility (Scholl et al., 2021).

Original: Value investors (VI) form heterogeneous subjective valuations

Original: V i of the asset based on discounted sums of dividends.

Original: Trend followers (TF) trade on the existence of trends in the asset price over various time horizons.
Revised: Trend followers (TF) trade on the existence of trends in the asset price over various time horizons.

Original: Agents’ trading strategies are represented by their trading signals ϕ ( t ) .
Revised: Agents’ trading strategies are represented by their trading signals ϕ ( t ) .

Original: [Equation]
Revised: [Equation]

Original: 2.2. Excess demand
Revised: 2.2. Asset demand

Original: The funds’ daily trading signals are inputs of theexcess demand function for the asset.
Revised: The funds’ daily trading signals are inputs of the demand function for the asset (Scholl et al., 2021).

Original: Theexcess demand function expresses the demand of the fund for the asset as a function of the unknown price p ( t ) . 
Revised: The demand function expresses the demand of the fund for the asset as a function of the unknown price p ( t ) . Fund wealth W is the sum of agent cash, present value of asset shares and liabilities.

Original: The tanh function smooths and bounds the trading signal in the feasible trading range [ − 1 , 1] . In the case of indifference ( ϕ ( t ) = 0 ), the 0 .
Revised: Our demand function features maximum leverage λ and strategy aggression β .

Original: 5 term targets an equal split of wealth between asset shares and cash. Fund wealth W is the sum of agent cash, present value of asset shares and liabilities.
Revised: Our demand function simply represents an investor with budget λW ( t ) spending a share ˜ ϕ ( t ) of her budget on the asset, and 1 − ˜ ϕ ( t ) on the bond (Poledna et al., 2014; Scholl et al., 2021).

Original: [Equation]
Revised: [Equation]

Revised: For ˜ ϕ

Revised: ( t ) =

Revised: tanh( βϕ ( t )) : the tanh function smooths and bounds the trading signal in the range [ − 1 , 1] , so that the demand never exceeds the agent budget including leverage.

Revised: This demand function is continuous, allows short-selling, enforces deleveraging & margin calls 1 , and always deliver orders that respect the budget constraint.

Original: 2.3. Market-clearing
Revised: 2.3. Market-clearing

Original: The market-clearing process finds theroot of the aggregate excess demand, i.e. the price for which the sum of the funds’excess demands equals 0 , demand matching supply.
Revised: The market-clearing process finds the price for which the sum of the funds’ demands equals the fixed asset supply Q , demand matching supply (Poledna et al., 2014). This is equivalent to the alternative market-clearing procedure of finding the root of the excess demand function (Scholl et al., 2021).

Original: When this is not possible, we minimize the difference between demand and supply.
Revised: The market-clearing condition here is thus: (cid:80)

Original: Earnings
Revised: [Equation] 2.4. Dividends, interest and investment flows

Original: After computing the clearing price, funds execute the resultingexcess demand orders.
Revised: After computing the clearing price, funds execute the resulting demand orders.

Original: Agents receive earnings: the dividends D ( t ) and interest r corresponding to their new positions.
Revised: Agents receive earnings: the dividends δ ( t ) and interest r corresponding to their new positions.

Original: The excess demand can be negative: agents then (short) sell shares.

Original: 2.5. Investment

Original: The actions of external investors play an essential role in the wealth dynamics of mutual funds.
Revised: The actions of external investors play an essential role in the wealth dynamics of mutual funds.

Original: Depending on the performance of the funds,  investors can choose to redeem their shares or invest.
Revised: Depending on the performance of the funds, external investors can choose to redeem their shares or invest.

Original: Our model models those inflows and outflows in the investment module according to empirical data on fund flows.
Revised: Our model models those inflows and outflows in the investment module according to empirical data on fund flows.

Original: 2.6. Bankruptcy
Revised: 2.5. Bankruptcy

Original: Funds with negative wealth enter bankruptcy and exit the market.
Revised: Funds with negative wealth enter bankruptcy and exit the market.

Original: An administrator slowly liquidates their shares.
Revised: An administrator slowly liquidates their shares.

Original: The wealthiest fund will split into several identical, equal-sized entities to fill the vacant spot.
Revised: The wealthiest fund will split into several identical, equal-sized entities to fill the vacant spot.

Original: This mechanism keeps the number of funds and asset shares constant and limits market perturbations due to insolvencies. 
Revised: This mechanism keeps the number of funds and asset shares constant and limits market perturbations due to insolvencies. 2.6. Strategy evolution

Revised: We distinguish two regimes in our simulation based on the role of strategy evolution.

Original: 3. Calibration and Validation
Revised: Environment 1 lets funds’ strategies fixed: funds are not adaptive and will not change their strategy type (NT, VI, TF) or their sub-strategy (e.g. the time horizon). Strategy optimisation entails understanding the inner dynamics of the ecology. This first environment is simpler but not fully static: as funds interact, wealth is reallocated between funds and (sub)strategies, constituting a form of evolution (Farmer, 2002).

Revised: The asymptotic wealth distributions evolved in market ecologies is an important topic beyond this article’s subject

Revised: This non-adaptive setting has some empirical justification since real funds usually are committed to their strategy toward their investors, and managers seldom change their investment style.

Revised: However, a small minority of funds display such adaptive behaviour.

Revised: Environment 2 features a small percentage of funds displaying adaptive behaviour.

Revised: Indeed, actively managed and hedge funds are more likely than other fund types to change or discover trading strategies.

Revised: From the perspective of trading strategy optimisation of a single fund, Environment 2 offers a much more complicated problem to solve, as the environment shows coevolution.

Revised: There are many ways to represent this adaptive behaviour, from simple random choice models to deep neural networks within each adaptive fund.

Revised: A simple and satisfying way to model this online adaptation may use Genetic Algorithms with agents optimising the sub-strategy parameter (e.g. bullishness for VI or time horizon for TFs) or Genetic Programming to simulate strategy innovation beyond user-defined types (Subramanian et al., 2006;

Revised: Fern ´ andez-Blanco et al., 2008; de Almeida & Neves, 2022; Long et al., 2022).

Original: Calibration and validation of agent-based models (ABMs) are crucial (Paulin et al., 2018).
Revised: Calibration and validation of agent-based models (ABMs) are crucial (Paulin et al., 2018).

Original: A common criticism of
Revised: A common criticism of

Original: ABMs is that they often have too many parameters and risk being unrealistic.
Revised: ABMs is that they often have too many parameters and risk being unrealistic.

Original: By calibration , we set model input parameters (short position size and investment) using real data to reduce the dimensionality of the parameter space.
Revised: By calibration , we tune the external investment system to represent realistic investor behaviour.

Original: We validate the model showing that it reproduces the stylised financial properties of asset returns (Cont, 2001).
Revised: We validate the model showing that it reproduces the stylised financial properties of asset returns (Cont, 2001; 2007).

Original: 3.1. Reproducing stylised facts of financial markets
Revised: 3.1. Reproducing stylised facts of financial markets

Original: Generating the so-called financial “stylised facts” is a popular requirement for validating financial ABMs.
Revised: Generating the so-called financial “stylised facts” is a popular requirement for validating financial ABMs.

Original: Our model reproduces all main stylised facts  (Cont, 2001).
Revised: Our model reproduces the main stylised facts of asset prices (Cont, 2001).

Original: Our log prices display intermittency and volatility clustering: irregular bursts in volatility that tend to cluster together, separated by periods of low volatility.
Revised: Our log prices display intermittency.

Original: The log price returns do not show significant linear autocorrelations past trivial frequencies.
Revised: The log price returns do not show significant linear autocorrelations past trivial frequencies.

Original: Returnsthemselves show heavy tail distributions, displaying excess kurtosis compared to a normal distribution.
Revised: Returns show heavy tail distributions with excess kurtosis compared to a normal distribution.

Original: We can also reproduce the leverage effect -negative correlation between price returns and volatility- a positive volume-volatility correlation and slow decay of autocorrelation in absolute returns. 
Revised: We can also reproduce the leverage effect -negative correlation between price returns and volatility- a positive volumevolatility correlation and slow decay of autocorrelation in absolute returns. We provide more details and comment on improving this calibration with simulation-based inference in the supplementary information.

Original: 3.2. Calibrating Investment flows
Revised: 3.2. Calibrating Investment flows

Original: We calibrate investment flows to ensure that funds’ returns lead to realistic redemption flows, as these flows represent a significant share of funds’ net value.
Revised: We calibrate investment flows to ensure that funds’ returns lead to realistic redemption flows, as these flows represent a significant share of funds’ net value.

Original: To correctly assessthe profitability of a given strategy, including a realistic mapping from returns to investment flows is thus necessary.
Revised: To correctly assess a given strategy’s profitability, a realistic mapping from returns to investment flows is thus necessary.

Original: To understand the empirical relationship between fund performance measured by returns and redemption flows, we analysed reporting data from the Securities and Exchange

Original: Commission. Investment companies are subject to reporting requirements of their assets, redemptions, sales, and other indicators through the form N-PORT.
Revised: Investment companies are subject to reporting requirements of their assets, redemptions, sales, and other indicators through various SEC forms.

Original: Net redemption flows appear negative for good funds and positive for funds with negative returns at the one-quarter horizon. Although some previous literature identified a  positive association (Chevalier & Ellison, 1997), other analyses have highlighted strong contrarian or profit-taking behaviours in mutual fund investors (Goetzmann & Massa, 2002; Rakowski & Yamani, 2021). The sign of the flows changes non-linearly with respect to the lagged return horizon and fund’s performance, following an S-shape approximated by a sigmoid function.
Revised: (Ha & Ko, 2019) analysed N-SAR reports and fund returns and identified a linear, positive relationship between funds’ excess return and investment flow, suggesting that external investors are chasing returns, confirming earlier results (Chevalier & Ellison, 1997).

Original: We model the share of redemption flows over the  net value of the assets as being distributed as a double Gumbel distribution. Our investment module is calibrated onthose empirical distributions to provide a realistic mapping between funds’ returns and external investments. 3.3. Calibrating Short Interest The size of short positions for the stock impacts ecology wealth dynamics.
Revised: We estimated the linear relationship between fund returns and net flows based on the combined data and a noise term calibrated on the data variance to introduce this data-driven investment flow module into our simulation.

Original: Some strategies might accumulate substantial short positions, be vulnerable to price movements and become extinct.

Original: The objective of this calibration of short interest is thus to bound the financial system in a realistic operating window.

Original: We ensure that our single asset is representative of the most popular stocks traded in empirical markets, considering short interest: how many shares of a stock are sold short compared to the total amount of shares.

Original: Using stock exchange data sourced by Yahoo Finance, we found that the average short ratio days to cover of the ten most popular NASDAQ stocks (i.e. with the highest market capitalisation) is 1.124.

Original: Compared to the number of shares available for trading, the average short-selling volume represents 1.17% of the float.

Original: In our model, we have restricted the maximum size of funds’ short positions accordingly.

Original: 4. Results
Revised: 4. Results

Original: We present the dynamics of the Evology ABM: the asymptotic distribution of wealth between strategies.
Revised: We present some vanilla dynamics of the Evology ABM without strategy evolution: the asymptotic distribution of wealth between strategies.

Original: We are interested in their dependency on the initial wealth distribution.
Revised: We are interested in their dependency on the initial wealth distribution.

Original: On each point of a uniform sample of 500 points in the three-dimensional simplex, we run the simulation 10 times for 150 years of trading.
Revised: On each point of a uniform sample of 500 points in the three-dimensional simplex, we run the simulation 10 times for 150 years of trading.

Original: We measure the average wealth share of the last 10 , 000 days of trading.
Revised: We measure the average wealth share of the last 10 , 000 days of trading.

Original: This sampling ensures convergence in wealth distributions and accounts for stochasticity.
Revised: This sampling ensures convergence in wealth distributions and accounts for stochasticity.

Original: The returns and wealth shares of the strategies significantly vary with their position in the simplex, showing density-dependence (Farmer, 2002; Scholl et al., 2021).
Revised: The returns and wealth shares of the strategies significantly vary with their position in the simplex, showing density-dependence (Farmer, 2002; Scholl et al., 2021).

Original: Experiment parameters ensure that we observe the asymptotic wealth shares after convergence.
Revised: Experiment parameters ensure that we observe the asymptotic wealth shares after convergence.

Original: Figure 2 provides an example of the price and dividend series generated during a single simulation run, and Table 1 presents the performance of the base strategies in the example run.
Revised: Figure 2 provides an example of the price and dividend series generated during a single simulation run, and Table 1 presents the performance of the base strategies in the example run.

Original: Figures 3-5 show the final wealth shares of each strategy after 150 years, depending on the initial condition.
Revised: Figures 3-5 show the final wealth shares of each strategy after 150 years, depending on the initial condition.

Original: Outside a specific corner of the simplex, the share of noise traders goes down to small levels of 5 to 20%.
Revised: Outside a specific corner of the simplex, the wealth percentage of noise traders goes down to negligible levels of 5 to 20%.

Original: Value investors are almost absent from the left boundary but dominate most of the simplex configurations with a majority of 70% of the wealth.
Revised: Value investors are almost absent from the left boundary but dominate most of the simplex configurations with a majority of 70% of the wealth.

Original: Trend followers dominate the unstable top region and aretotally absent around the bottom axis.
Revised: Trend followers dominate the unstable top region and are absent around the bottom axis.

Original: Early extinction of the other strategies characterises this unstable top region: if initialised in high proportion to other strategies, TFs arevery detrimental to the other species.
Revised: Early extinction of the other strategies characterises this unstable top region: if initialised in high proportion to other strategies, TFs are detrimental to the other species.

Original: This stability and diversity are desirable: actual markets feature a diversity of profitable strategies.
Revised: This stability and diversity are desirable: actual markets feature a variety of profitable strategies.

Original: 5. Trading Strategy Optimisation in Evology
Revised: 5. Trading Strategy Optimisation in Evology

Original: The trading strategy optimisation problem

Original: Optimisation problem Given a set of market conditions

Original: M , the action space available to our adaptive fund is a Tdimensional vector Φ of trading signal values.

Original: On each day t , the fund decides if it is buying or selling asset shares.

Original: [Equation]

Original: Dynamic strategy training can lead to dynamic strategies: the function ϕ could change at every step.

Original: Static strategy training optimizes a single function ϕ that will generate the list of actions ϕ ( t ) .

Original: The objective function for the trading strategy optimisation problem is typically a measure of profit, such as the Sharpe ratio (geometric mean of returns divided by the standard deviation of returns) or wealth multipliers (ratio of increase of the funds’ wealth after T simulation steps).

Original: On a metalevel, this simulator also allows studying the performance of the optimisation objectives, checking which metrics lead to the most profitable strategies and possibly searching for an optimal combination of objectives.

Original: Lucas’ critique and changes in market conditions It has been known since the Lucas critique that changes in economic policies can lead to structural changes in modelling and optimal behaviour.

Original: Since we are in a simulation environment, we can alter market conditions, and training can include those changes.

Original: It is easy to implement changes in the interest rates, dividend policies and other interventions in an ABM.

Original: For example, Central Banks’ quantitative easing can be modelled as a fictitious agent with a constant positive excess demand for the asset.

Original: 5.2. Simulation training and time series training

Original: Trading strategy search is a popular topic for applying machine learning.
Revised: Trading strategy search is a popular topic for applying machine learning.

Original: Quantitative trading systems driven by linear & logistic regression, support vector machines, reinforcement learning, deep neural networks, random forests, genetic algorithms, and genetic programming have been successful in creating profitable strategies (Allen & Karjalainen, 1999; Dempster & Jones, 2001; Zhang & Maringer, 2016;
Revised: Quantitative trading systems driven by linear & logistic regression, support vector machines, reinforcement learning, deep neural networks, random forests, genetic algorithms, and genetic programming have successfully created profitable strategies (Allen & Karjalainen, 1999; Dempster & Jones, 2001; Zhang & Maringer, 2016;

Original: Ta et al., 2018; Hasan et al., 2020).
Revised: Ta et al., 2018; Hasan et al., 2020).

Original: Derivative-free methods are relevant since the profit objective usually cannot be reduced to an explicit, derivable objective function.
Revised: Derivative-free methods are relevant since the profit objective usually cannot be reduced to an explicit, derivable objective function.

Original: Evolution strategies and program synthesis can also be particularly performant for this task.
Revised: Evolution strategies and program synthesis can also be particularly performant for this task.

Original: The returns of a trading strategy depend on the wealth dis- tribution of the market, i.e. the wealth shares owned by the different strategies (Scholl et al., 2021), challenging optimisation.
Revised: The returns of a trading strategy depend on the wealth distribution of the market, i.e. the wealth shares owned by the different strategies (Scholl et al., 2021), challenging optimisation.

Original: Strategies experience crowding: their returns decrease as their size grows and exceeds the carrying capacity of their niche (Farmer, 2002; Scholl et al., 2021).
Revised: Strategies experience crowding: their returns decrease as their size grows and exceeds the carrying capacity of their niche (Farmer, 2002; Scholl et al., 2021).

Original: Offline tuning, which assumes no interactions between the strategy and the training data, can overlook this effect.
Revised: Offline tuning, which assumes no interactions between the strategy and the training data, can overlook this effect.

Original: These additional difficulties invite reassessing the performance of those popular machine learning approaches for trading strategy search in a simulation environment.
Revised: These additional difficulties invite reassessing the performance of those popular machine learning approaches for trading strategy search in a simulation environment.

Original: In addition, provided that the simulation environment is realistic enough, training on the data generating process behind market dynamics can lead to more robust strategies than training on single sample paths of this market process.
Revised: In addition, provided that the simulation environment is realistic enough, training on the data generating process behind market dynamics can lead to more robust strategies than training on single sample paths of this market process.

Original: 5.3. Benchmark  tasks
Revised: 5.1. Benchmark learning tasks

Original: Here, we present some tasks to advance trading strategy optimisation in Evology. Evolve static/dynamic strategies that are better than thebase strategies .
Revised: General challenges Beyond the tasks described below, the strategies evolved in Evology should satisfy some higherend goals.

Original: First, the optimisation algorithms should be capable of finding statistically significantly more profitable strategies than those already present in the model and some trivial strategies, such as the typical buy and hold or the interest rate oncash. The objective would be to evolve realistic, high-performing strategies without human input, e.g. the Kelly criterion.
Revised: We do not desire the machine learning algorithms to result in incomprehensible, over-fitting strategies to maximise profits. For the trained strategies to be interesting, they need to be interpretable -display some level of economic insight- and robust -successfully operate under various market conditions-. We develop in more detail those general challenges and provide more details on the tasks mentioned below in the supplementary information.

Original: Setting a small percentage of thefund population to be adaptive, trading strategy search under coevolution could be an interesting task. In real markets, actively managed funds and hedge funds are adaptive. Coevolution makes trading strategy optimisation more challenging because the evolved strategies may be copied or face stronger opponents in the simulation, impacting their profitability.
Revised: Environments As described in Section 2.6, we can consider two different environments: an environment of stable strategies where the strategies of all the agents (except the optimising agent) stay fixed over time. The alternative is a coevolution setting where a small fraction of the population is adaptive, imitating the activity of actively managed/hedge funds.

Original: Setting a benchmark for simulation performance is difficult as this benchmark is new. However, obtaining Sharpe ratios superior to empirical hedge fund levels, i.e. 10 or 20, would be a good starting point. Evolve robust, interpretable strategies generalisable to real
Revised: This adaptation could use an extensive range of models from the intensity of choice and imitation to more sophisticated evolutionary or machine learning approaches.

Original: markets For the trained strategies to be inter - esting
Revised: This coevolution setting brings an additional challenge to strategy optimisation.

Original: , they need to be interpretable and robust . Al- though interpretability in this context requires further definition, we can start by observing that real-world strategies usually rest on some economic reasoning.
Revised: Task 1: Trading strategy optimisation We are interested in an individual fund optimising its trading strategy ϕ

Original: For example, trend followers assume that price trends are persistent. Interpretable strategies should exhibit a generalisable “model” of the financial market. As for robustness, the strategies can readily train on various market initial conditions or face unseen policy interventions and structural model changes, obstacles that are easy to implement in the ABM. Finally, the 
Revised: ( t ) to maximise profits during a T -period market run, using a profit measure such as the cumulative return or the Sharpe ratio.

Original: Evology environment would be  more useful if it could train strategiesthat could be used in real markets, an additional challenge to robustness and ABM realism.
Revised: The baseline levels to achieve would be i) become more profitable than the base strategies and ii) become more profitable as empirical strategies.

Original: Continuous development and calibration of the simulation platform The end goal for this simulation could be to become an easy-to-use toolkit for developing machine learning algorithms, just like the Open AI gym (Brockman et al., 2016). It is thus essential that the environment offers sufficient complexity to be of interest to the machine learning community and is realistic enough for its insights to be potentially transferable to the real world.
Revised: Task 2: Investment strategy optimisation We are interested in an individual investor optimising its investment strategy.

Original: The simulation should be efficient enough to allow training, e.g. through the development of a GPU implementation. Calibration and validation efforts should continue to emulate market environments, mainly populating the ecology with realistic strategies reconstructed from actual portfolios (Scholl et al., 2021). Thanks to  the SEC data publicly available, model calibration constitutes a benchmark task of interest.
Revised: The investor is learning how to invest, i.e. a function ν mapping the various fund characteristics (return, size...) to positive or negative investment amounts, intending to maximise the profitability of their investments.

Original:  Conclusionand next steps
Revised: 6. Conclusion

Original:  We present Evology, an empirically calibrated financial agent-based model grounded on the market ecology perspective.
Revised: Conclusion We present Evology, an empirically calibrated financial agent-based model grounded on the market ecology perspective.

Original: In particular, Evology features heterogeneous trading strategies, a representative asset and realistic investment flows.
Revised: In particular, Evology features heterogeneous trading strategies, a representative asset and realistic investment flows.

Original: The complexity of strategies’ interactions and the density-dependence of returns make this specific optimisation problem challenging for search algorithms: dynamic, deceptive, and no clear optimal solution.
Revised: The complexity of strategies’ interactions and the density-dependence of returns make this specific optimisation problem challenging for search algorithms: dynamic, deceptive, and no clear optimal solution.

Original: This multi-agent model has the potential to become a new training ground of interest for trading strategy search with machine learning methods.
Revised: This multi-agent model has the potential to become a new training ground of interest for trading strategy search with machine learning methods.

Original: We outline several directions for developing the financial ABM environment and propose a set of  tasks to advance ABM-based trading strategy optimisation: evolving better strategies than the base ones  and under coevolution, improving strategies’ interpretability and robustness.
Revised: We outline several directions for developing the financial ABM environment. We propose a set of strategy tasks in Evology for trading and investment. We define the first baseline as evolving better strategies than the base ones in static and coevolution cases and under coevolution, improving strategies’ interpretability and robustness and obtaining real-world level strategy performance. We further discuss the model limitations and next steps to address them in the supplementary information.

Original: Software and Data
Revised: Software and Data

Original: The Evology ABM is available open-source 1 .
Revised: The Evology ABM is available open-source 2 .

Original: For simulation efficiency and object-oriented design, it is coded in
Revised: For simulation efficiency and object-oriented design, it is coded in

Original: Python and Cython (Behnel et al., 2010).
Revised: Python and Cython (Behnel et al., 2010).

Original: Calibration procedures use SEC Form 13F and Form N-PORT data that are publicly available 2 .
Revised: Calibration procedures use SEC Form 13F and Form N-PORT data that are publicly available 3 .

Original: Acknowledgements
Revised: Acknowledgements

Original: This publication is based on work supported by the EPSRC Centre for Doctoral Training in Mathematics of
Revised: This publication is based on work supported by the EPSRC Centre for Doctoral Training in Mathematics of

Original: Random Systems: Analysis, Modelling and Simulation (EP/S023925/1).
Revised: Random Systems: Analysis, Modelling and Simulation (EP/S023925/1).

Original: The authors thank Christian Schroeder de Witt for advice.
Revised: The authors thank Christian Schroeder de Witt for useful advice, the Mathematical Instiute of the University of Oxford for the use of computing servers, members of Oxford INET for advice and three anonymous referees for their valuable suggestions. Supplementary information .1. Model validation - Empirical stylised facts of asset prices (Cont, 2001) The following figures related to stylised facts of asset prices use data from a single simulation run, starting from the [1 / 3 , 1 / 3 , 1 / 3] initial condition on wealth shares.

Revised: The interest rate equals 0 .

Revised: 01 , external investment is inactive, and the random seed is equal to 0 for reproducibility of the dividend and noise processes.

Original: Supplementary information
Revised: Absence of autocorrelations Autocorrelations of asset returns should be insignificant except for very small time scales, in which the microstructure has some impact. Heavy tails The unconditional distribution of returns should display a power-law or a Pareto tail, with finite variance, excluding the normal distribution. Gain/loss asymmetry One should observe large drawdowns in stock prices without equally large upward movements. Aggregational Gaussianity As we increase the timescale for calculating the returns, their distribution should look more and more like a normal distribution.

Revised: Intermittency Returns should display high variability, visible by the presence of irregular bursts in time series of volatility estimators.

Revised: Volatility clustering and slow decay of autocorrelation in absolute returns Different volatility measures should display a positive autocorrelation over several days, showing that high-volatility events tend to cluster over time.

Revised: We can observe this volatility clustering from the slow decay of the autocorrelation function of absolute returns.

Revised: Leverage effect Measures of asset volatility should negatively correlate with the asset returns.

Revised: Over our simulation run, this Pearson correlation is negative and equal to − 0 .

Revised: Volume/volatility correlation Trading volume should correlate with volatility measures.

Revised: Over our simulation run, this

Revised: Pearson correlation is positive and equal to 0 .

Revised: In addition, volume positively correlates with price returns ( 0 . 001 ).

Revised: .2. Simulation run example

Revised: [Table]

Revised: .3. Supplementary details on the benchmark learning tasks .4. Interpretability and robustness

Revised: Although interpretability in this context requires further definition, we can start by observing that real-world strategies usually rest on some economic reasoning.

Revised: For example, trend followers assume that price trends are persistent.

Revised: Interpretable strategies should exhibit a generalisable “model” of the financial market.

Revised: As for robustness, the strategies can readily train on various market initial conditions or face unseen policy interventions and structural model changes, which are easy to implement in the ABM.

Revised: For example, in the current context of rising interest rates, we could train a strategy on a probability distribution of interest rates rather than on a single interest rate value.

Revised: Finally, the Evology environment would be more useful if its insights could generate useful signals for trading in real markets, an additional challenge to robustness and ABM realism.

Revised: Since the Lucas critique, we know that changes in economic policies can lead to structural changes in modelling and optimal behaviour.

Revised: Since we are in a simulation environment, we can alter market conditions, and training can include those changes.

Revised: It is easy to implement changes in the interest rates, dividend policies and other interventions in an ABM.

Revised: For example, Central Banks’ quantitative easing can involve a fictitious agent with a constant positive excess demand for the asset.

Revised: .5. Task 1: trading strategy

Revised: The trading is simplified to arbitrage between the asset and the bond/cash.

Revised: Given a set of market conditions M , the action space available to our adaptive fund is a T -dimensional vector Φ of trading signal values.

Revised: On each day t , the fund decides if it is buying or selling asset shares.

Original: [Table]
Revised: [Equation]

Revised: The optimisation problem faced by the adaptive fund consists in finding the sequence of trading signals Φ that maximises some measure of performance P , typically a measure of profits such as the Sharpe ratio (geometric mean of returns divided by the standard deviation of returns) or wealth multipliers (ratio of increase of the funds’ wealth after T simulation steps).

Revised: [Equation]

Revised: The baselines to beat are first the performance of the base strategies introduced in the model.

Revised: The first definition of success for the adaptive fund optimising a trading strategy is to achieve better performance than the NT, VI and TF strategies present in the model, under the same bounded rationality limitations.

Revised: This environment requires the agent to understand its market impact and the returns landscape and to learn to mitigate the effect of its size on its returns.

Revised: Once we achieve this target, setting a benchmark for simulation performance is more difficult as this benchmark is new.

Revised: However, obtaining Sharpe ratios superior to empirical hedge fund levels, i.e. 10 or 20, would be a good starting point.

Revised: There are several interesting extensions of this task.

Revised: The first is to evaluate the performance of different optimisation goals P .

Revised: Does profit maximisation lead to the highest profits, or is the problem so deceptive that other metrics, multiobjective fitness, or even novelty search (Lehman & Stanley, 2011) can lead to better strategies?

Revised: We can also consider two sub-tasks for Task 1.

Revised: The optimised trading strategy could be a single trading signal function ϕ ( t ) kept fixed during each evaluation: this would be a static task.

Revised: In a more dynamic approach, the agent could instead evolve a sequence of different trading signal functions.

Revised: Learning thus would focus on the meta-strategy that governs the adoption at any period of a specific trading signal.

Revised: Task 2: investment strategy optimisation

Revised: The investor is learning what characteristics of the funds should determine their inflows and outflows so that the investor achieves the highest return on their investment.

Revised: Using a similar choice of the performance measure P , the investor maximisation problem is:

Revised: [Equation]

Revised: Where ν is a function that maps an investment decision (the quantity of money to invest in or out of a fund) based on some fund characteristics x , including measures of fund return at various timescales, the statistical significance of those returns, fund size, fund strategy...

Revised: It will be fascinating to consider how this evolved investor behaves concerning timescales (Scholl et al., 2021), compare its performance to typical investment firms and how statistically significant are their actions.

Revised: The first baseline will be to achieve a higher success (e.g. profitability or rationality) than the average investor behaviour implemented in the model and derived from empirical data (Ha & Ko, 2019).

Revised: .6. Supplementary conclusion

Revised: Limitations The model presented here presents several limitations concerning market behaviour that future updates will tackle.

Revised: Built on improvements to previous market ecology and financial ABMs, it is a stepping stone towards a more realistic and complex market ecology simulation.

Revised: First, the single-asset setup limits the dimensionality of the action space.

Revised: Future updates should extend the model to several assets with different classes (e.g. small, mid, big cap) and attached to companies with dividend life cycles.

Revised: Several classes of strategies representing significant fractions of market capitalisation are missing from the model, such as market making, passive investing, dividend/yield, growth, growth & income.

Revised: Future research will identify more scalable representations of trading strategies to avoid an explosion of types that undermines the simulation development and analysis.

Revised: Our funds lack market entry, ageing and exit dynamics beyond mere bankruptcies, which may make the ecology more stable than it should be.

Revised: Monetary policy and dividend policies are currently exogenous: the interest rate and dividend are exogenous and disconnected from the financial market dynamics, which invites future updates to consider a more macroeconomic integrated and dynamical way of setting those model inputs.

Revised: Leverage, substrategies and activity timescales, which vary considerably among agents (Preis et al., 2011), should also be calibrated to the corresponding strategies in future updates.

Revised: It is possible that this push for realism also necessitates switching from market-clearing to a limit-order book system.

Revised: Next steps A potential goal for this simulation could be to become an easy-to-use toolkit for developing machine learning algorithms, just like the Open AI gym (Brockman et al., 2016).

Revised: It is thus essential that the environment offers sufficient complexity to be of interest to the machine learning community and is realistic enough for its insights to be potentially transferable to the real world.

Revised: The simulation should be efficient enough to allow training, e.g. through developing a GPU implementation.

Revised: Calibration and validation efforts should continue to emulate market environments, mainly populating the ecology with realistic strategies reconstructed from actual portfolios (Scholl et al., 2021).

Revised: Thanks to publicly available SEC data, model calibration constitutes a benchmark task of interest.

Revised: One task of potential interest that is not mentioned yet in the model is the optimisation of financial regulation: we could easily add to the model described in Figure 1 an additional regulation component in which a policy-maker observes the market and attempts to limit volatility, inflation or other measures of concern, by restricting agent behaviour through maximum leverage, price interventions such as quantitative easing or tightening.

Revised: This addition would open a new task in which a machine learning algorithm can attempt to optimise financial regulation to achieve those goals.

Revised: Next-level calibration with simulation-based inference

Revised: Although we have reduced the number of free parameters in the model, we are left with several free parameters of importance.

Revised: We assumed that the various heterogeneous sub-strategies (e.g. time horizon θ i , required rates of return ˜ r i ) are uniform within arbitrary ranges.

Revised: The leverage of the various strategies is also significant to the ecology dynamics as they impact the system’s position in the profit landscape.

Revised: Several of the abovementioned updates may also bring more free parameters to tackle.

Revised: A poor choice of summary statistics can lead to losing information from the empirical data and reduce the quality of calibration (Dyer et al., 2022), which motivates alternatives such as simulation-based inference using graph neural networks.

Revised: These procedures will likely require further exploration of empirical data, such as the N-PORT forms and 13F filings.