An Empirical Study of Instruction-tuning Large Language Models in C hinese (2023)

Abstract: The success of ChatGPT validates the potential of large language models (LLMs) in artificial general intelligence (AGI). Subsequently, the release of LLMs has sparked the open-source community’s interest in instruction-tuning, which is deemed to accelerate ChatGPT’s replication process. However, research on instruction-tuning LLMs in Chinese, the world’s most spoken language, is still in its early stages. Therefore, this paper makes an in-depth empirical study of instruction-tuning LLMs in Chinese, which can serve as a cookbook that provides valuable findings for effectively customizing LLMs that can better respond to Chinese instructions. Specifically, we systematically explore the impact of LLM bases, parameter-efficient methods, instruction data types, which are the three most important elements for instruction-tuning. Besides, we also conduct experiment to study the impact of other factors, e.g., chain-of-thought data and human-value alignment. We hope that this empirical study can make a modest contribution to the open Chinese version of ChatGPT. This paper will release a powerful Chinese LLM that is comparable to ChatGLM. The code and data are available at https: //github.com/PhoebusSi/Alpaca-CoT.

## introduction the emergence of chatgpt gives humanity a real sense of hope for agi for the first time, and inspires researchers to realize the importance of llm research. however, the closed source of llms (e.g., @xcite and palm @xcite ) coupled with the requirement for massive computing resources to build the exclusive llm has deterred researchers from reaching the llm training stage. subsequently, a series of "api research" based on gpt-3 and chatgpt are con-stantly emerging, which stimulate the specific capabilities of frozen llms (e.g., chain-of-thought @xcite @xcite ) or guide them to complete specific tasks @xcite , by calling openai interfaces and carefully designing prompts without model training. the unexpected disclosure of the pre-trained llama @xcite model changes this situation, and has sparked a surge of excitement in the llm research community. this is the first open llm with competitive performance. recently, alpaca @xcite uses self-instruct @xcite and chatgpt to generate 52k instructions, which can enable llama to respond to various human instructions like chatgpt. this open project verifies the important role of instructiontuning @xcite open llms in replicating the chatgpt process. given the open llm llama and alpaca's high-quality instruction data, there is still a challenge for researchers: even the instruction-tuning of the 7b model still requires high computational resources. to address this problem, alpaca-lora extends the parameter-efficient method lora to llama, which further reduces the computing cost of instruction-tuning. it further sparks extensive research in the open-source community on instruction-tuning for llms. on this basis, more llms (e.g, bloom @xcite , gpt-j @xcite ) are shown to have significant improvements in instruction-following performance with instruction-tuning. on the other hand, more instruction data is constantly being proposed, e.g., belle @xcite constructs chinese instructions in the same way, and sharegpt collects a large number of real human-chatgpt conversations. however, research on instruction-tuning llms in chinese, the world's most spoken language, is still in its early stages. llm bases, parameterefficient methods, and instruction data are three essential elements for customizing chinese chatgptlike llms. there are no tutorials in the academic community on them yet. some important questions have not yet been explored and answered: 1) "which open llm is more suitable as a foundation for chinese instruction-tuning?", 2) "how do parameter-efficient methods other than lora affect llms?" and 3) "what is the impact of various types of instruction datasets?" to answer these questions, we collect a range of llms, parameterefficient methods, and instruction datasets. besides, we consider the agi (instruction-following) capability and professional knowledge reserve (human exams) of models, and correspondingly select two benchmarks belle-eval @xcite and mmcu @xcite for comprehensively evaluation. we also conduct experiments to explore several other factors that may affect the final performance. specifically, we find that tuning with chain-of-thought (cot) data can improve the ability to respond to complex reasoning questions. different llms may be suitable for different language prompts (excluding instruction parts) in instructiontuning. human-value alignment results into slight performance drop. on the basis of the above findings, this paper carefully instruction-tunes a powerful chinese llms that is comparable to chatglm. the contributions can be summarized as follows: (1) we are the first to systematically study on instruction-tuning in chinese through adequate experiments, which can serve as a cookbook that provides valuable findings for customizing chinese version of chatgpt. (2) we release a powerful chinese llm that is comparable to chatglm. ## conclusion this paper is the first to conduct a thorough empirical study on instruction-tuning open large language models in chinese, with a detail discussion of a range of large language models, parameter-efficient methods, and chinese instruction datasets. in addition, we explore several other important factors, including cot, vocabulary, language of prompts and human-value alignment. based on the empirical exploration, we publicly release a llm, that is rival to chatglm, with detailed implementation details.