MAS-GAN: Adversarial Calibration of Multi-Agent Market Simulators.

Abstract: We look at the problem of how the simulation of a financial market should be configured so that it most accurately emulates the behavior of a real market.  In particular, we address agent-based simulations of markets that are composed of many hundreds or thousands of trading agents. A solution to this problem is important because it provides a credible test bed for evaluating potential trading algorithms (e.g., execution strategies). Simple backtesting of such algorithms suffers from a critical weaknesses, chiefly that the overall market is not responsive to the candidate trading algorithm. Multi-agent simulations address this weakness by simulating {\it market impact} via interaction between market participants. Calibration of such multi-agent simulators to ensure realism, however, is a challenge. In this paper, we present MAS-GAN -- a multi-agent simulator calibration method that allows to tune simulator parameters and to support more accurate evaluations of candidate trading algorithm. Our calibration focus is on high level parameters such as the relative proportions of the various types of agents that populate the simulation.
      MAS-GAN is a two-step approach: first, we train a discriminator that is able to distinguish between ``real'' and ``fake'' market data as a part of GAN with self-attention, and then utilize it within an optimization framework to refine simulation parameters. The paper concludes with quantitative examples of applying MAS-GAN to improve simulator realism.

Reviews and author responses:

This paper proposes using a GAN to learn a discriminator and then use the discriminator to tune a non-differentiable simulator. The authors use this idea in the context of stock market simulation.

Pros:
The idea of using the discriminator of a GAN to tune parameters of a simulator is interesting.

Cons:
The experiments appear weak to me because of the following reasons:
-- The goal stated upfront is "we need to calibrate parametrized multiagent limit order book market simulator to be as close as possible to real market data". Why is realism measured only as mid-price return every minute? The price changes every second. Shouldn't there be richer statistics - for example as in [Li. et al. 2020]? Moreover, the only aspect that is calibrated is the number of type of agents N,M. There are so many parameters in the simulator that can be configured (trader arrival rate, params in the Ornstein-Uhlenbeck process). Why not calibrate these?
-- The simulator used appears to be rich, but clearly still an approximation of real behavior in the market. The simulator uses well-known trader models but these are after all approximations of behavior. Plus there are other well-known models other than the 3 types such as heuristic belief learning model of traders [Steven Gjerstad. The competitive market paradox, 2007]. This makes me more uncomfortable with just calibrating N,M - this is possibly a forced fitting that matches mid-prices relying on assumptions that may not hold in real stock markets.
--- Are there quantitative numbers for comparing 5(c) and 2(a)?

Presentation cons:
Plot the distributions to compare in one graph, not possible to visually compare across two graphs. I cannot compare 5(c) and 2(a) - 5(c) is also just too small to read. The images are also spilling over the column width. SimGAN is a name already used by another well-known paper (https://arxiv.org/pdf/1612.07828v1.pdf), please consider another name.

--------------Post response-------------------
Stock market is complex and the evaluation with few statistics is unrealistic and too simple. At least the evaluation should be more fine grained than prior work. The goal of interpretability is nice, but not sure the paper gets there by only tuning few parameters.
Overall, I like the idea but needs better execution of those ideas.

We want to thank the reviewer for the feedback. 

We disagree with reviewer’s summary of the paper being “A multi-agent market simulator using GANs are proposed”. We do not simulate the market using GANs, but rather use GAN discriminator to calibrate a market simulator constructed using agent-based approach.

We outline the contributions of this paper in Section 1.3 (titled “Our contributions”). Additionally, comparison to existing work is presented in Section 1.2 (titled “Background and related work”),

To summarize our contributions, we present a novel two-step method for multi-agent market simulator calibration: we first learn a calibration objective as a discriminator portion of GAN (it is generally a difficult problem to write such calibration objective  explicitly), and then use the learnt calibration objective given by a GAN discriminator for simulator parameter optimization. To the best of our knowledge, this is the first method that proposes to use adversarially trained discriminator as an objective function for multi-agent system optimization. Additionally, we use conv1D and self-attention without recurrence in the GAN architecture to generate synthetic time series  —  which is also novel. 

We also improved the figure layout of the paper in the revised manuscript (see revised manuscript attached, changes marked in blue).

We want to thank the reviewer for the feedback. Below are our comments:

First, thank you for bringing our attention to the clash of the naming convention used in the paper - we were previously not aware of existing method called SimGAN, hence, we renamed our method to MAS-GAN (MAS stands for Multi Agent Simulator). We’ve made respective changes in the manuscript (see revised manuscript attached, changes marked in blue), and also refer to our method as MAS-GAN further in this rebuttal.

Various stylized facts about limit order books quantities can be used as realism metrics. In Section 1.1 of the paper, we discuss different stylized facts proposed in the literature — they include stylized facts about properties of asset return distributions, volumes and order flow, non-stationary patterns, etc. The input to MAS-GAN are both mid price and traded volume time series (not just mid price time series). We specifically evaluate the generator quality both for mid price and volume stylized facts (Figures 2(a-c) mid-price returns; Figures 2(d-f) volume/volatility correlations). In Section 3.3.3, when we evaluate MAS-GAN calibration performance, we present stylized facts for both 1-minute and 10-minute mid price return distributions as well as for volume/volatility correlation distribution (previously only evaluation on price returns was included in the paper).
For your reference, we now made all charts in Figures 6, 7, 8 larger. We reach the conclusion that the configurations highlighted as most realistic by MAS-GAN also display closer price return and volume/volatility correlation similarity (figures 6, 7, 8).

Generally, MAS-GAN can be applied to time series at any time resolution. Similarly, MAS-GAN can apply to calibration of both high level(e.g., number of agents) and low level simulator parameters(e.g., agent arrival rates, order sizes) as well as the parameters of Ornstein-Uhlenbeck fundamental (in order to define the most realistic the market trajectory that agents will follow).  Unlike Li et. al. 2020 paper where the goal is to generate a realistic  stock market order stream with no specific requirements for interpretability of the generated data, our goal is to find the most realistic multi-agent simulator configuration given a predefined agent universe in order to retain explainability of the agent-based system.  Thus, our goal is to choose the most realistic simulator configuration that consists of agents from a given agent universe.  

In this paper, apply our method to limit order book mid price and traded volume time series sampled at one minute. With that, we calibrate the simulator over a rectangular grid of noise and value agents (the market maker agent is present in all configurations). We chose this time resolution and agent universe in order to demonstrate the feasibility of our method. We are planning to look at different time resolutions and agent universe extension by different well-known types of agents such momentum traders and heuristic belief learners for future work. We specifically mention these extensions in the conclusion of the revised manuscript.

We want to thank the reviewer for the feedback. We revised discussion in sections 1.1 and 1.2 to include only work that is immediately related to results presented in the paper (revised manuscript attached, changes marked in blue).

It is true that the GAN generator and discriminator are biased towards a historical dataset that they are trained on, and unbiased towards synthetic data coming for the simulator. We clarify explanation of it in the revised manuscript. Also, as we mention in the conclusion, one of the directions for future work would be to apply the proposed method to calibrate the simulation of low vs. high volatility regimes (such as COVID market shock).

We would like to thank anonymous reviewers for the insightful comments. Attached is the revised manuscript - all revisions are marked in blue. Each individual reviewer questions are answered below.

- Summary: A multi-agent market simulator using GANs are proposed.
- Quality: I see many inappropriate layouts that make it difficult to read and hard to focus on the technical aspects.
- Clarity: The paper is not clearly written and not self-contained.
- Originality & Significance
  - Sec. 2 is supposed to describe the proposed algorithm, however, it is hard to see any novel contents. If the novel contents are written in another section, then it should be rearranged so that the readers would easily catch them.
- Cons: Hard to find out due to unclearly written paper. I see no comparison to conventional works.
- Layout: I see many inappropriate layouts. I recommend the authors to fix it.
  - I recommend that every figure has a title even if there are subtitles (Fig. 1 and 4)
  - Fig. 1(a) and Fig. 1(b) are independent contents. I recommend that the subfigures be separated. (I guess that that's why the authors didn't write the title of Fig. 1). Also, the two figures in Fig. 1(b) can be shown as two subfigure.
  - Please modify the figure layout properly (size, location, width, etc).
  - I see the bad layout in Figure 3.
  - The contents of Figure 4(c) would better be shown in Table not figure.
  - When I print the paper, some part of Figure 5 (c) is not printed since it is outside of the page. The size of the figure should be adjusted.
- Typo: I also see some grammar errors. Any grammar checking program would find those errors. I recommend the authors check the grammar of the whole text again.
  - In the first paragraph of Sec. 1.3, please add space in front of "Both".
  - What is the blank in the last paragraph of Sec. 2? (simulation parameters out of the ___________)

- The authors have presented a new technique to create multi-agent market simulation using GANs. It is a good step forward in applying GANs as a tool for simulation engineering. As there is already quite a bit of research done in the same domain, so distilling the paper shows that there are two novelties
    1 - A two step method is used with the help of GANs to train a time series generator and discriminator.
    2 - Sefl attention is used in the design of generator and discriminator. 
    3 -  A relatively new trend of using GANs for parameter optimization is followed.
- There are some problems with the presentation of the article. Section 1.1 and 1.2 contain too much information that is not directly related to the contribution.  Sometimes it seems that a sentence is added in section 1.2 to cite an article. For example "From a game-theoretic perspective, a simulated environment is more realistic when it reaches a Nash equilibrium, where every player’s parameters are configured so that each can do no better by unilaterally changing its strategy. Agents that learn to maximize their long term rewards by reinforcement from empirical equilibrium environments have been discussed in Schvartzman & Wellman (2009)." Nash's equilibrium has profound impact on the market research, but here in this article it seems spurious to mention it, as that is not the focus of the research. Similarly, few other sentences can be omitted from section 1.1 and 1.2 to improve focus and clarity.
- The statement does not seem to be right, "Because SIM-GAN only uses historical, and not simulated data for training, once the discriminator is trained – it can be used to calibrate any time series model (not necessarily multi-agent) that represents a given historical dataset". If this is true, then where does the fake time series data come from? If this is wrong, then the claim that discriminator is unbiased is also wrong. Also it is safe to say that the generator and discriminator are biased towards the specific portion of the historical data selected for training. For example, it would have been interesting to see how the simulator would predict a shock like COVID-19 on the market. That would be a good test of its (un)bias. Apparently, the end of section 3.1 is pointing towards the same fact. 
-  Mistake in the sentence "We then perform grid-based optimization with respect to the discriminator-defined objective to find
most realistic simulation parameters out of the . "


Meta-review: This paper tackles an important problem of generating a synthetic data for stock market agents behavior. In particular, GAN is trained to distinguish stock market time series from synthetic ones. Then market agent parameters can be scored by running time series generated using these parameters through a GAN. This way one can optimize the parameters to find realistic ones.

The reviewers appreciate authors response, however, they found that their concerns has not been fully addressed:

1. Clarity. The reviewers believe that the presentation need to be significantly improved and clarified before the paper is ready for acceptance (see reviewers comments for details).
2. The metrics used to evaluate the performance are not sufficient. While "stylized facts" statistics can be a good sanity check it does not mean that the agents that have been produced are useful in any way. Presumably, the goal of calibration such agents is to use them in simulation to generate synthetic trading for the purpose of building a predictive model. If so then it seems that the best way to judge the success of the calibration by the success of downstream task.

Given this, the paper can not be accepted for the presentation in its current form.

Decision: withdrawn-rejected-submissions