Qwen Technical Report
Authors: name: Jinze Bai; name: Shuai Bai; name: Yunfei Chu; name: Zeyu Cui; name: Kai Dang; name: Xiaodong Deng; name: Yang Fan; name: Wenbin Ge; name: Yu Han; name: Fei Huang; name: Binyuan Hui; name: Luo Ji; name: Mei Li; name: Junyang Lin; name: Runji Lin; name: Dayiheng Liu; name: Gao Liu; name: Chengqiang Lu; name: Keming Lu; name: Jianxin Ma; name: Rui Men; name: Xingzhang Ren; name: Xuancheng Ren; name: Chuanqi Tan; name: Sinan Tan; name: Jianhong Tu; name: Peng Wang; name: Shijie Wang; name: Wei Wang; name: Shengguang Wu; name: Benfeng Xu; name: Jin Xu; name: An Yang; name: Hao Yang; name: Jian Yang; name: Shusheng Yang; name: Yang Yao; name: Bowen Yu; name: Hongyi Yuan; name: Zheng Yuan; name: Jianwei Zhang; name: Xingxuan Zhang; name: Yichang Zhang; name: Zhenru Zhang; name: Chang Zhou; name: Jingren Zhou; name: Xiaohuan Zhou; name: Tianhang Zhu
Published: 2023-09-28 17:07:49

Abstract: Large language models (LLMs) have revolutionized the field of artificial
intelligence, enabling natural language processing tasks that were previously
thought to be exclusive to humans. In this work, we introduce Qwen, the first
installment of our large language model series. Qwen is a comprehensive
language model series that encompasses distinct models with varying parameter
counts. It includes Qwen, the base pretrained language models, and Qwen-Chat,
the chat models finetuned with human alignment techniques. The base language
models consistently demonstrate superior performance across a multitude of
downstream tasks, and the chat models, particularly those trained using
Reinforcement Learning from Human Feedback (RLHF), are highly competitive. The
chat models possess advanced tool-use and planning capabilities for creating
agent applications, showcasing impressive performance even when compared to
bigger models on complex tasks like utilizing a code interpreter. Furthermore,
we have developed coding-specialized models, Code-Qwen and Code-Qwen-Chat, as
well as mathematics-focused models, Math-Qwen-Chat, which are built upon base
language models. These models demonstrate significantly improved performance in
comparison with open-source models, and slightly fall behind the proprietary
models.