Qwen2 Technical Report
Authors: name: An Yang; name: Baosong Yang; name: Binyuan Hui; name: Bo Zheng; name: Bowen Yu; name: Chang Zhou; name: Chengpeng Li; name: Chengyuan Li; name: Dayiheng Liu; name: Fei Huang; name: Guanting Dong; name: Haoran Wei; name: Huan Lin; name: Jialong Tang; name: Jialin Wang; name: Jian Yang; name: Jianhong Tu; name: Jianwei Zhang; name: Jianxin Ma; name: Jin Xu; name: Jingren Zhou; name: Jinze Bai; name: Jinzheng He; name: Junyang Lin; name: Kai Dang; name: Keming Lu; name: Keqin Chen; name: Kexin Yang; name: Mei Li; name: Mingfeng Xue; name: Na Ni; name: Pei Zhang; name: Peng Wang; name: Ru Peng; name: Rui Men; name: Ruize Gao; name: Runji Lin; name: Shijie Wang; name: Shuai Bai; name: Sinan Tan; name: Tianhang Zhu; name: Tianhao Li; name: Tianyu Liu; name: Wenbin Ge; name: Xiaodong Deng; name: Xiaohuan Zhou; name: Xingzhang Ren; name: Xinyu Zhang; name: Xipin Wei; name: Xuancheng Ren; name: Yang Fan; name: Yang Yao; name: Yichang Zhang; name: Yu Wan; name: Yunfei Chu; name: Zeyu Cui; name: Zhenru Zhang; name: Zhihao Fan
Published: 2024-07-15 12:35:42

Abstract: This report introduces the Qwen2 series, the latest addition to our large
language models and large multimodal models. We release a comprehensive suite
of foundational and instruction-tuned language models, encompassing a parameter
range from 0.5 to 72 billion, featuring dense models and a Mixture-of-Experts
model. Qwen2 surpasses most prior open-weight models, including its predecessor
Qwen1.5, and exhibits competitive performance relative to proprietary models
across diverse benchmarks on language understanding, generation, multilingual
proficiency, coding, mathematics, and reasoning.
  The flagship model, Qwen2-72B, showcases remarkable performance: 84.2 on
MMLU, 37.9 on GPQA, 64.6 on HumanEval, 89.5 on GSM8K, and 82.4 on BBH as a base
language model. The instruction-tuned variant, Qwen2-72B-Instruct, attains 9.1
on MT-Bench, 48.1 on Arena-Hard, and 35.7 on LiveCodeBench. Moreover, Qwen2
demonstrates robust multilingual capabilities, proficient in approximately 30
languages, spanning English, Chinese, Spanish, French, German, Arabic, Russian,
Korean, Japanese, Thai, Vietnamese, and more, underscoring its versatility and
global reach.
  To foster community innovation and accessibility, we have made the Qwen2
model weights openly available on Hugging Face1 and ModelScope2, and the
supplementary materials including example code on GitHub3. These platforms also
include resources for quantization, fine-tuning, and deployment, facilitating a
wide range of applications and research endeavors.