StockBench: Can LLM Agents Trade Stocks Profitably In Real-world
  Markets?
Authors: name: Yanxu Chen; name: Zijun Yao; name: Yantao Liu; name: Jin Ye; name: Jianing Yu; name: Lei Hou; name: Juanzi Li
Published: 2025-10-02 16:54:57

Abstract: Large language models (LLMs) have recently demonstrated strong capabilities
as autonomous agents, showing promise in reasoning, tool use, and sequential
decision-making. While prior benchmarks have evaluated LLM agents in domains
such as software engineering and scientific discovery, the finance domain
remains underexplored, despite its direct relevance to economic value and
high-stakes decision-making. Existing financial benchmarks primarily test
static knowledge through question answering, but they fall short of capturing
the dynamic and iterative nature of trading. To address this gap, we introduce
StockBench, a contamination-free benchmark designed to evaluate LLM agents in
realistic, multi-month stock trading environments. Agents receive daily market
signals -- including prices, fundamentals, and news -- and must make sequential
buy, sell, or hold decisions. Performance is assessed using financial metrics
such as cumulative return, maximum drawdown, and the Sortino ratio. Our
evaluation of state-of-the-art proprietary (e.g., GPT-5, Claude-4) and
open-weight (e.g., Qwen3, Kimi-K2, GLM-4.5) models shows that while most LLM
agents struggle to outperform the simple buy-and-hold baseline, several models
demonstrate the potential to deliver higher returns and manage risk more
effectively. These findings highlight both the challenges and opportunities in
developing LLM-powered financial agents, showing that excelling at static
financial knowledge tasks does not necessarily translate into successful
trading strategies. We release StockBench as an open-source resource to support
reproducibility and advance future research in this domain.