Agent Trading Arena: A Study on Numerical Understanding in LLM -Based Agents (2025)

Abstract: Large language models (LLMs) have demonstrated remarkable capabilities in natural language tasks, yet their performance in dynamic, real-world financial environments remains underexplored. Existing approaches are confined to historical backtesting, where trading actions cannot influence market prices, and agents train on static data. To overcome this limitation, we present the Agent Trading Arena, a virtual zero-sum stock market in which LLM-based agents engage in competitive, mult-agent trading and directly impact price dynamics. By simulating realistic bid-ask interactions, our platform enables agents to train in scenarios that closely mirror live markets, thereby narrowing the gap between training and evaluation. Experiments show that LLMs struggle with numerical reasoning when given plain-text data, tending to overfit local patterns and recent values. In contrast, chart-based visualizations significantly boost both numerical reasoning and trading performance. Moreover, integrating a reflection module yields further improvements, especially with visual inputs. Finally, evaluations of the NASDAQ and CSI datasets demonstrate the superiority of our method, particularly under high volatility. All code and data are available at https://github.com/wekjsdvnm/Agent-Trading-Arena.

## introduction in recent years, large language models (llms) have demonstrated strong capabilities in contextual understanding and generation across tasks such as natural language processing @xcite , data analysis, and reasoning @xcite . their applications are rapidly extending to complex decision-making and interactive scenarios. llm-based autonomous agents # these authors contributed equally. have emerged as a promising paradigm, showing notable progress in communication @xcite , social simulation @xcite @xcite , gaming @xcite , and multi-agent collaboration @xcite @xcite . these agents are increasingly regarded as a pathway toward general artificial intelligence, with growing potential in dynamic decision-making and interactive environments. despite these advances, the application of llm-based agents in financial markets remains limited. financial ecosystems naturally form complex multi-agent environments in which participants interact through dynamic market mechanisms. however, most existing studies @xcite rely on historical backtesting, as illustrated in fig. 1 , where agents cannot influence market prices and thus fail to capture interactivity and adaptive behavior. this gap underscores the need for a framework that supports real-time agent interactions and realistic market feedback. to address these limitations, we present the agent trading arena, a virtual zero-sum stock market in which llm-based agents engage in competitive trading and directly affect price dynamics through bid-ask interactions. our platform generates continuously evolving numerical data, forcing agents to adapt to shifting market conditions; any temporarily optimal strategy is quickly countered, rewarding adaptability and robust reasoning over memorization. by simulating bid-ask order matching in real time, the arena faithfully reproduces liquidity dynamics and price impact, enabling agents to experience authentic market friction and slippage. unlike static backtests, the arena continuously updates its state based on agents' collective actions, creating an ever-shifting landscape that prevents overfitting to historical patterns. because agents directly influence subsequent price trajectories, the arena closes the loop between training and evaluation, effectively bridging the "train-test" gap of traditional financial simulations. its modular design further supports seamless integration of new market features, such as varying transaction costs, order types, or macroeconomic shocks, making it a versatile testbed for financial ai research. our experiments show that llms struggle with numerical reasoning when processing plain-text data: they focus on absolute values, overlook percentage changes and relational patterns, and overemphasize recent trends even when earlier data are emphasized. in contrast, visual representations, such as line charts, bar graphs, and scatter plots, substantially improve both numerical reasoning and trading performance. as shown in we further integrate a reflection module @xcite , which amplifies the benefits of visual data by enabling agents to iteratively refine their reasoning. this module is particularly effective with structured visual inputs, facilitating deeper pattern recognition and more strategic decision-making. finally, we validate our approach on real-world datasets from nasdaq and csi, demonstrating consistent performance gains over baselines, particularly under high-volatility market regimes. our contributions are summarized threefold: ## agent trading arena to overcome the limitations of static backtesting and capture dynamic interactions among market participants, we propose the agent trading arena, a closed-loop, zero-sum economic system that simulates complex quantitative scenarios in real markets @xcite . the overall architecture is shown in fig. 2 and detailed in appendix a. within this environment, agents invest in assets, earn dividends on their holdings, and pay daily expenses using virtual currency. ## proposed method our agent, arenatrader, integrates textual and visual modalities to form a more comprehensive understanding of market conditions. it employs a structured three-stage framework, market analysis, trade execution, and strategic reflection, to effectively leverage this multi-modal representation. this design enables informed and adaptive decision-making based on data-driven insights and iterative feedback. ## conclusion we presented the agent trading arena, a closedloop, zero-sum simulation framework designed to model complex market dynamics and evaluate llms on numerical reasoning tasks. our experiments show that while llms struggle with plain-text numerical data, they achieve significantly higher returns when supported by visual representations, underscoring the importance of charts and graphs in complex decision-making. integrating a reflection module further improves performance by enabling iterative analysis and strategic refinement. validation on both nasdaq and csi confirms that vision-enhanced llm agents consistently outperform text-only counterparts without additional training. overall, this work highlights both the capabilities and limitations of llms in numerical reasoning and establishes the agent trading arena as a robust testbed for advancing research in adaptive, real-world financial applications.