LMSYS-Chat-1M: A Large-Scale Real-World LLM Conversation Dataset
Authors: name: Lianmin Zheng; name: Wei-Lin Chiang; name: Ying Sheng; name: Tianle Li; name: Siyuan Zhuang; name: Zhanghao Wu; name: Yonghao Zhuang; name: Zhuohan Li; name: Zi Lin; name: Eric. P Xing; name: Joseph E. Gonzalez; name: Ion Stoica; name: Hao Zhang
Published: 2023-09-21 12:13:55

Abstract: Studying how people interact with large language models (LLMs) in real-world
scenarios is increasingly important due to their widespread use in various
applications. In this paper, we introduce LMSYS-Chat-1M, a large-scale dataset
containing one million real-world conversations with 25 state-of-the-art LLMs.
This dataset is collected from 210K unique IP addresses in the wild on our
Vicuna demo and Chatbot Arena website. We offer an overview of the dataset's
content, including its curation process, basic statistics, and topic
distribution, highlighting its diversity, originality, and scale. We
demonstrate its versatility through four use cases: developing content
moderation models that perform similarly to GPT-4, building a safety benchmark,
training instruction-following models that perform similarly to Vicuna, and
creating challenging benchmark questions. We believe that this dataset will
serve as a valuable resource for understanding and advancing LLM capabilities.
The dataset is publicly available at
https://huggingface.co/datasets/lmsys/lmsys-chat-1m.